Blogger.create { :name =>'Matt Aimonetti',
:location => 'San Diego, Ca',
:email => mattaimonetti AT gmail.com,
:linkedin => Matt's Linkedin page,
:recommend_me => HERE,
:contractor => true}

Avoid using metaprogramming (seriously!)

Written by matt on May 4th, 2008

Ruby is sexy, Ruby is cool and its metaprogramming potential offers some really cook features. However you might not realize that your cleverness is slowing down your code.

Today I was working on cleaning up merb_helper a Merb plugin that brings a lot of the stuff Rails developers are used to. In Merb we aim for speed and try to avoid magic.

merb_plugin didn't receive a lot of love from the main contributors but few features were added by different contributors and the code became hard to maintain.

Looking at the code I quickly found this bad boy:

(Old Merb Time DSL using metaprogramming)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

   
  module MetaTimeDSL

    {:second => 1, 
     :minute => 60, 
     :hour => 3600, 
     :day => [24,:hours], 
     :week => [7,:days], 
     :month => [30,:days], 
     :year => [364.25, :days]}.each do |meth, amount|
      define_method "m_#{meth}" do
        amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
        self * amount
      end
      alias_method "m_#{meth}s".intern, "m_#{meth}"
    end

  end
  Numeric.send :include, MetaTimeDSL

The above code looks awful to me and I decided to rewrite it a way I thought would be more efficient:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

  module TimeDSL

    def second
      self * 1
    end
    alias_method :seconds, :second

    def minute
      self * 60
    end
    alias_method :minutes, :minute

    def hour
      self * 3600
    end
    alias_method :hours, :hour

    def day
      self * 86400
    end
    alias_method :days, :day

    def week
      self * 604800
    end
    alias_method :weeks, :week

    def month
      self * 2592000
    end
    alias_method :months, :month

    def year
      self * 31471200
    end
    alias_method :years, :year

  end
  Numeric.send :include, TimeDSL

To make sure I was right, I run the following benchmarks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127

require 'benchmark'
TIMES = (ARGV[0] || 100_000).to_i

Benchmark.bmbm do |x|
  
  x.report("metaprogramming 360.seconds") do
    TIMES.times do    
      360.m_seconds
    end
  end
  
  x.report("no metaprogramming 360.hours") do
    TIMES.times do
      360.seconds
    end
  end
  
  x.report("metaprogramming 360.minutes") do
    TIMES.times do    
      360.m_minutes
    end
  end
  
  x.report("no metaprogramming 360.minutes") do
    TIMES.times do
      360.minutes
    end
  end
  
  x.report("metaprogramming 360.hours") do
    TIMES.times do    
      360.m_hours
    end
  end
  
  x.report("no metaprogramming 360.hours") do
    TIMES.times do
      360.hours
    end
  end
  
  x.report("metaprogramming 360.days") do
    TIMES.times do    
      360.m_days
    end
  end
  
  x.report("no metaprogramming 360.days") do
    TIMES.times do
      360.days
    end
  end
  
  x.report("metaprogramming 360.weeks") do
    TIMES.times do    
      360.m_weeks
    end
  end
  
  x.report("no metaprogramming 360.weeks") do
    TIMES.times do
      360.weeks
    end
  end

  x.report("metaprogramming 18.months") do
    TIMES.times do    
      18.m_months
    end
  end
  
  x.report("no metaprogramming 18.months") do
    TIMES.times do
      18.months
    end
  end
  
  x.report("metaprogramming 7.years") do
    TIMES.times do    
      7.m_years
    end
  end
  
  x.report("no metaprogramming 7.years") do
    TIMES.times do
      7.years
    end
  end
  
end


 Rehearsal ------------------------------------------------------------------
metaprogramming 360.seconds      0.130000   0.000000   0.130000 (  0.133164)
no metaprogramming 360.hours     0.050000   0.000000   0.050000 (  0.042655)
metaprogramming 360.minutes      0.130000   0.000000   0.130000 (  0.133327)
no metaprogramming 360.minutes   0.040000   0.000000   0.040000 (  0.042401)
metaprogramming 360.hours        0.140000   0.000000   0.140000 (  0.134312)
no metaprogramming 360.hours     0.040000   0.000000   0.040000 (  0.043125)
metaprogramming 360.days         0.130000   0.000000   0.130000 (  0.134949)
no metaprogramming 360.days      0.050000   0.000000   0.050000 (  0.043745)
metaprogramming 360.weeks        0.130000   0.000000   0.130000 (  0.135581)
no metaprogramming 360.weeks     0.050000   0.000000   0.050000 (  0.043544)
metaprogramming 18.months        0.130000   0.000000   0.130000 (  0.135234)
no metaprogramming 18.months     0.050000   0.000000   0.050000 (  0.044354)
metaprogramming 7.years          0.140000   0.000000   0.140000 (  0.144062)
no metaprogramming 7.years       0.050000   0.000000   0.050000 (  0.044392)
--------------------------------------------------------- total: 1.260000sec

                                     user     system      total        real
metaprogramming 360.seconds      0.130000   0.000000   0.130000 (  0.132567)
no metaprogramming 360.hours     0.040000   0.000000   0.040000 (  0.042777)
metaprogramming 360.minutes      0.140000   0.000000   0.140000 (  0.132554)
no metaprogramming 360.minutes   0.040000   0.000000   0.040000 (  0.043193)
metaprogramming 360.hours        0.130000   0.000000   0.130000 (  0.133027)
no metaprogramming 360.hours     0.050000   0.000000   0.050000 (  0.042613)
metaprogramming 360.days         0.130000   0.000000   0.130000 (  0.138637)
no metaprogramming 360.days      0.050000   0.000000   0.050000 (  0.043213)
metaprogramming 360.weeks        0.130000   0.000000   0.130000 (  0.134049)
no metaprogramming 360.weeks     0.040000   0.000000   0.040000 (  0.043713)
metaprogramming 18.months        0.140000   0.000000   0.140000 (  0.134941)
no metaprogramming 18.months     0.040000   0.000000   0.040000 (  0.043980)
metaprogramming 7.years          0.150000   0.000000   0.150000 (  0.143389)
no metaprogramming 7.years       0.040000   0.000000   0.040000 (  0.044585)
 0.136591)

The metaprogramming version of the same implementation is almost 3 times slower!

Moral of the story: only use metaprogramming if you really have to or if you don't care about speed of execution.



Comments

  • Henrik N on 04 May 02:40

    This example is definitely not the place for metaprogramming. Even if it would run equally fast, the metaprogramming version of the code is pretty hard to read.

    One thing I think I would change about your code is to make the methods more readable by defining them in terms of one another:

    def week 7.days * self end

    and so on. Especially when it comes to month and year where someone is likely to check the implementation to see just what numbers are used.

  • Henrik N on 04 May 02:43

    Oh, and the 364.25 (as opposed to 365.25) in the very first code snippet is a bug and/or typo.

  • dohzya on 04 May 03:19

    try to put @amount = amount.isa?(Array) ? amount[0].send(amount[1]) : [email protected] out of the definemethod. It must be compute one time only, not at each call of method... Your can try to replace your @[email protected] by an @eval( "", binding )@ too (more slow one time only)

  • Matt Aimonetti on 04 May 20:38

    Matt Jones had a really good post, but my blog indicated that textile was enabled while only Markdown is enabled.

    Here is Matt's post:

    dohzya's comment is on target - the overhead here is in restoring the context (stack frame) that was captured by definemethod, and handling the amount.isa?(Array) case.

    Just lifting the Array case above definemethod fails pretty badly, as the methods don't get defined on Numeric until the end. (Also note: the meta-programming code relies on the TimeDsl code in this example - mdays calls TimeDsl's version of hours)

    What's really needed (if you want to go meta) is something closer to ActiveRecord's method:

    see pastie

    Please look at the above pastie to see how Matt got really good perf using metaprogramming. Really interesting stuff!

  • Ben on 05 May 01:00

    I seem to remember a cartoon from the 80s - GI Joe, maybe? - that always had a moral at the end; like the one in this post, however, the moral presented then often seemed to be the wrong one given the experiences the Joe team had in the preceding half-hour.

    Isn't the correct moral here that you should benchmark your code, metaprogrammed or otherwise?

  • Matt Aimonetti on 05 May 01:35

    Good point! At least if you care about performances.

    Now, if the benchmarks were close and that the slightly slower version was easier to read/understand, I would probably go for readability over performance.

  • allen on 05 May 07:43

    Metaprogramming is used to add methods and features to classes at load time, not on each instantiation. Of course, doing this will require more work to dynamically create and install these methods on your Ruby classes.

    While the metaprogramming can be slower on load time, it it not significant considering it is only loaded once (in the production environment, though after every change to those classes in development).

    This should be the same in merb as rails. In a script/runner invocation or an application off the rails (CGI, command line, etc.) will have a slower load time as the class will be loaded each time your app starts up.

    In the end, it does provide more readable/maintainable and DRY code, if written well. Are you optimizing prematurely?

  • DaveS on 05 May 08:06

    You might as well go back to assembly programming...

  • Ken Miller on 05 May 08:21

    It's not metaprogramming per se that makes this slow, it's the use of definemethod. methods so defined are always slower to call than methods defined the usual way. If you just use classeval and a string containing a standard method definition, you'll get results more in line with the written out version.

    Of course, then it's even harder to read. Metaprogramming should really only be used if it's the only way to solve a problem, as you say, but performance is not really a reason not to.

  • Ken Miller on 05 May 08:27

    Amendment to above: in this case, it's also that you're doing the array work every time. That could easily be moved into the generator code and out of the instance method, leaving the instance methods defined identically to the hand-coded methods.

    But this really supports the overall point -- metaprogramming is harder to write and understand, and contains more opportunities for bugs and sneaky performance pitfalls. But if you really need it, it IS possible to make it perform just as well as hand-coded ruby.

  • Tim Harper on 05 May 11:16

    The following code is just as fast, and I would write it this way over the long way :) It's pretty clear to me.

    module MetaTimeDSL SECOND = 1 MINUTE = SECOND * 60 HOUR = MINUTE * 60 DAY = HOUR * 24 WEEK = DAY * 7 MONTH = DAY * 30 YEAR = DAY * 364.25

    %w[SECOND MINUTE HOUR DAY WEEK MONTH YEAR].each do |const_name| meth = const_name.downcase class_eval <<-EOF def m_#{meth} self * #{constget(constname)} end alias m#{meth}s m#{meth} EOF end end

  • Tim Harper on 05 May 11:16

    The following code is just as fast, and I would write it this way over the long way :) It's pretty clear to me.

    module MetaTimeDSL
      SECOND  = 1
      MINUTE  = SECOND * 60
      HOUR    = MINUTE * 60
      DAY     = HOUR * 24
      WEEK    = DAY * 7
      MONTH   = DAY * 30
      YEAR    = DAY * 364.25
    
      %w[SECOND MINUTE HOUR DAY WEEK MONTH YEAR].each do |const_name|
        meth = const_name.downcase
        class_eval <<-eof>
  • Rob Kaufman on 06 May 13:24

    It is worth noting that Tim Harpers method only gets called when the class instantiates (which is why it is equally as fast) as Matt's cleaned up version. The most important thing about meta-programming is knowing when to use it and when not to and to only do as little as possible. The reasons are speed, readability and maintainability. And remember there is nothing wrong with writing code that writes code (generators)

Comments are closed