Rails on the Run

Wow, it’s been a while since I blogged. With all the cool kids saying that spending time reading RSS feeds is overrated (see Defunkt’s keynote for instance) I even wonder if people will ever read this post!

Anyways, I have been quite busy preparing courses for classes I gave to a bunch a great Engineers at one of the Fortune 100 companies based in San Diego. I was also planning my big vacation trip to Europe and wrapping up few projects.

However, during my exile overseas, I came to the conclusion that Rubyists don’t scale. Since Twitter became stable again, we don’t hear many people ranting about Rails not scaling anymore. With one of my clients’ app handling around 7 million requests/day I can tell you Ruby/Merb do scale quite well! But ruby developers don’t seem to scale for some reason.

Maybe saying that we(Rubyists) don’t scale isn’t technically correct but that’s basically what one of my client told me.

Let’s go back in time a little bit and follow my client who we will call clientX.

• ClientX has a great concept and wants to conquer the internet.
• ClientX hears that Rails is the way to go.
• ClientX hires a contractor/mercenary/freelancer/guns for hire/consultant (aka Me)

• Me builds a killer app using Merb (killing framework)

• ClientX raises loads of $$$

• ClientX wants to hire a team because Me doesn’t want to become a FTE

• ClientX and Me look for Rubyists wanting to relocate and get a decent salary

• ClientX *can’t find someone they consider good enough and who would accept their package

• Many JAVA guys are available on location and accept lower packages

• Ruby app gets ported over to JAVA
• Me sad

So is it really the Rubyists’ fault if we don’t want to relocate and only accept higher packages? Should I blame Obie for telling people to charge more and teaching how to hustle? Or should we just tell clients that it’s time to get used to working remotely?

Honestly, I don’t think any of the above explanations are valid. Ruby is the new/hot technology and very few people have the skills and experience to lead major projects. These people make a good living and enjoy their “freedom” and dream of building their own products. Most of them/us value their work environment, family and are reluctant to move.

At the same time, companies do need people locally(at least a core team) and can’t always afford the cool kids.

ClientX, quite frustrated by the whole hiring process told me once: “you Ruby folks are too unavailable and difficult to work with! We need a committed team that actually cares about the company/product.”

That hurts when you worked hard on a project and just can’t satisfy the client by finding guys willing to relocate and work for them. It gets even more painful when your code gets entirely ported over to JAVA!

But at the same time I understand ClientX’s motivation, PHP guys are cheaper, JAVA guys are more available, why in the word did we go with Ruby and are now struggling finding people?

Once again, there is positive and negative side in everything, by choosing Ruby and a “great contractor” ClientX was able to catch up with the competition and even pass them in no time. They quickly raised good money and got everything they needed to become #1. I don’t believe it would have been possible to do the same thing so quickly with JAVA for instance. However choosing a cutting edge technology means you need to look harder for talented people.

It’s too bad the code gets rewritten in a different language but at the same time, I do my best to facilitate the process and to keep a good relation with my client. There was nothing personal in the decision, it’s just too bad we were not able to keep on using the latest/coolest/awesomess technology available

To finish on a positive note, here is the solution to scale your Ruby task force provided to you by the #caboose wisdom:

Based on my conversations with other #caboosers who hire other devs, the word in the street is that you just need to get one or two great ruby guys (who will probably cost you a lot) and find a bunch of smart people to train. You’ll end up with an awesome team of scalable rubyists

In a previous article I took an example of bad metaprogramming and I pushed people to think twice before using metaprogramming.

My main points were that:

• you might make your code way slower if you don’t know what you are doing
• readability might drop considerably
• maintainability can become an issue

People left some very good comments about how to write the same module using metaprogramming and keep things fast.

Today Wycats pinged me about this post and told me that the issue was definemethod and that classeval is effectively the same as regular code, it gets evaluated in eval.c, just like regular Ruby code. On the other hand, defined_method has to marshall the proc.

I cleaned up my benchmarks using rbench, added some of the solutions provided to me and obtained the following results:

Here is the original/bad metaprogramming example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  module MetaTimeDSL

    {:second => 1, 
     :minute => 60, 
     :hour => 3600, 
     :day => [24,:hours], 
     :week => [7,:days], 
     :month => [30,:days], 
     :year => [364.25, :days]}.each do |meth, amount|
      define_method "#{meth}" do
        amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
        self * amount
      end
      alias_method "#{meth}s".intern, "#{meth}"
    end

  end
  Numeric.send :include, MetaTimeDSL

The no metaprogramming module is available there

Refactored:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

  module RefaMetaTimeDSL

  {:second => 1, 
   :minute => 60, 
   :hour => 3600, 
   :day => [24,:hours], 
   :week => [7,:days], 
   :month => [30,:days], 
   :year => [364.25, :days]}.each do |meth, amount|
    self.class_eval <<-RUBY
      def r_#{meth}
        #{amount.is_a?(Array) ? "#{amount[0]}.#{amount[1]}" : "#{amount}"}
      end
      alias_method :r_#{meth}s, :r_#{meth}
    RUBY
  end

end
Numeric.send :include, RefaMetaTimeDSL

the refactor 2 or eval based solution provided by Matt Jones which uses class_eval like the previous refactor.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  module EvalMetaTimeDSL
    def self.included(base)
      base.class_eval do
        [ [:e_second, 1], 
          [:e_minute, 60], 
          [:e_hour, 3600], 
          [:e_day, [24,:e_hours]], 
          [:e_week, [7,:e_days]], 
          [:e_month, [30,:e_days]], 
          [:e_year, [365.25, :e_days]]].each do |meth, amount|
            amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
            eval "def #{meth}; self*#{amount}; end"
            alias_method "#{meth}s", meth
          end
      end
    end
  end
  Numeric.send :include, EvalMetaTimeDSL

and finally, the “better metaprogramming” version:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

 module GoodMetaTimeDSL

  SECOND  = 1
  MINUTE  = SECOND * 60
  HOUR    = MINUTE * 60
  DAY     = HOUR * 24
  WEEK    = DAY * 7
  MONTH   = DAY * 30
  YEAR    = DAY * 364.25
  
  %w[SECOND MINUTE HOUR DAY WEEK MONTH YEAR].each do |const_name|
      meth = const_name.downcase
      class_eval <<-RUBY 
        def g_#{meth} 
          self * #{const_name} 
        end 
        alias g_#{meth}s g_#{meth} 
      RUBY
  end
end
Numeric.send :include, GoodMetaTimeDSL

Looking at the refactored version by Wycats, you can see he’s right and the major issue with the original version was definemethod. Using classeval does make things almost as fast and even faster than the no metaprogramming version.

Interesting enough, the benchmarks show that some methods from the meta modules are faster than the ones from the no meta module. Overall, an optimized metaprogramming can be more or else as fast as a non meta code. Of course, with the new VMs coming up, things might change a little bit depending on the language implementation.

In conclusion, metaprogramming can be as fast as no metaprogramming but that won’t help your code readability and maintainability, so make sure to only use this great trick when needed!

p.s: here is the benchmark file if you don’t believe me

I realized I haven’t updated this blog in a while. Here is a quick update on what’s happened and on things to come:

• RailsConf 08. Great conference, probably my last Rails Conf though. I’ll be in Orlando for Ruby Conf 08 and I’ll focus on 1 or 2 local conferences (probably mountain west and another one).

• MerbCamp 08 in San Diego this Fall organized by SD Ruby. Details are not finalized yet but Yehuda Katz announced it during his Merb talk at RailsConf.

• Moved this blog to a new Joyent accelerator with git support and finally have the possibility to use Ambition! (planning on moving from Mephisto to Feather)

• Launched a client’s Merb app and getting around 3 million hits/day. Merb is just awesome. (more info when the client’s app gets out of beta)

• I’ll join Gregg Pollack from http://railsenvy.com/ during Qcon and take part in the Ruby for the Enterprise track. My talk will focus on Merb usage in real life.

• Renamed my github username, new repo url: http://github.com/mattetti (sorry about that)