Rails on the Run

If you follow my blog, you already know what Merb is.

I love Rails and I truly believe it has changed web development. At least it has changed the way I do web development.

But Merb looks slick, apparently is way faster than Rails, and has less “fluff” and less magic.

Now that we are getting really close to a Merb 1.0 (scheduled for Rails Conf ’08) it’s time to evaluate if Merb is the good choice for some of my clients’ projects.

However, according to Merb’s author, Ezra, at MountainWest RubyConf 2008, Rails will get you there faster. In a client’s case, they don’t need to build a huge app but need a lot of speed and the ability to easily handle a heavy load right away without using caching. Also most of the traffic will go through an API so we won’t have to manage too many views.

Let’s see how fast Merb really is.

To test Merb’s speed, I built the very same prototype using Merb 0.9.2 and Rails Edge (pre 2.1). Both apps use ActiveRecord and are connected to a UTF8 MySQL database, both apps have exactly the same views. (Note that Merb would run way faster using DataMapper, but I don’t feel that DM 0.9x is production ready yet, also, using a rack handler would certainly be way faster but my goal was really to compare ActionPack vs Merb.)

Both apps use the same ActiveRecord class, their controllers are a bit different but basically do the same thing.

Here is what was tested:

• The Merb/Rails app should receive a GET request with a JSON object in the query.

• The Merb/Rails app should route the request to a controller and pass the JSON object to an AR class.

• The AR class should parse the JSON object (which contains an array of objects), extract each object, and try to find them in the database using one of the attributes. If the object isn’t found, it should be created, otherwise it should return the AR object. The amount of hits should be incremented by 1 and the object should be saved back to the database.

• A simple HTML view should be rendered

Quick Merb benchmark

I setup Merb to run locally on my MacBook 2.16Ghz Core Duo 2, 2Gb Ram. To test the raw performance, Merb is started in production mode.

I then used httperf to make 10000 connections to the server at a rate of 500 (–rate=500 –send-buffer=4096 –recv-buffer=16384 –num-conns=10000 –num-calls=1)

Here are the results:

Maximum connect burst length: 29
Total: connections 4377 requests 4221 replies 2932 test-duration 41.629 s
Connection rate: 105.1 conn/s (9.5 ms/conn, <=1022 concurrent connections)
Connection time [ms]: min 41.0 avg 1920.4 max 35390.8 median 898.5 stddev 4887.3
Connection time [ms]: connect 2118.1
Connection length [replies/conn]: 1.000

Request rate: 101.4 req/s (9.9 ms/req)
Request size [B]: 321.0

*Reply rate [replies/s]: min 0.0 avg 73.3 max 143.0 stddev 65.8 (8 samples)*
Reply time [ms]: response 809.0 transfer 18.1
Reply size [B]: header 121.0 content 557.0 footer 0.0 (total 678.0)
*Reply status: 1xx=0 2xx=2932 3xx=0 4xx=0 5xx=0*

CPU time [s]: user 0.35 system 36.54 (user 0.8% system 87.8% total 88.6%)
Net I/O: 78.4 KB/s (0.6*10^6 bps)

Errors: total 7068 client-timo 0 socket-timo 0 connrefused 0 connreset 1445
Errors: fd-unavail 5623 addrunavail 0 ftab-full 0 other 0

What we care about is the reply rate/s. We have an average of 73.3 requests per second with a standard deviation of 65.8 using 8 samples.

We also make sure that all the replies were successful. (status == 2xx)

I also checked the database, made sure my AR object was created and that the hits were increased. AR object hits: 2932, which matches the amount of replies reported by httperf.

We don’t care so much about the rest of the httperf. Let’s move on to the Rails benchmark.

---

Quick Rails benchmark

Rails is set the same way, running locally in production mode, same httperf settings.

Here are the results:

Maximum connect burst length: 44

Total: connections 2923 requests 2825 replies 1672 test-duration 37.418 s

Connection rate: 78.1 conn/s (12.8 ms/conn, <=1022 concurrent connections)
Connection time [ms]: min 382.7 avg 5635.4 max 36384.5 median 1887.5 stddev 10103.1
Connection time [ms]: connect 3631.2
Connection length [replies/conn]: 1.000

Request rate: 75.5 req/s (13.2 ms/req)
Request size [B]: 319.0

*Reply rate [replies/s]: min 0.0 avg 43.4 max 75.2 stddev 30.8 (7 samples)*
Reply time [ms]: response 1568.1 transfer 36.7
Reply size [B]: header 471.0 content 581.0 footer 0.0 (total 1052.0)
*Reply status: 1xx=0 2xx=1672 3xx=0 4xx=0 5xx=0*

CPU time [s]: user 0.25 system 31.31 (user 0.7% system 83.7% total 84.4%)
Net I/O: 69.4 KB/s (0.6*10^6 bps)

DB hits: 1672

First thing, the database object was created properly and the hits incremented to 1672 which matches the amount of replies reported by httperf.

Then, we notice that on this test, we only got 7 samples, that’s more than enough though. The standard deviation is 30.8 which is better than Merb’s 65.8. That means that in our benchmarks, the reply speed difference in Merb’s requests was bigger than Rails’. Not a big deal, this is not a scientific test but it’s good to acknowledge it.

What we really care about is the average reply rate: 43.4

Let’s also note that all the replies had a 2xx status, so everything went well.

Results

Based on this really basic benchmark, my Merb app had an average reply rate of 73.3 requests per second against Rails’ 43.4 requests per second.

That means that in this very specific case,

Merb is 69% faster than Rails! Sexy!

In other words, my Merb prototype could handle 69% more requests than the Rails prototype in the same amount of time.

I heard people reporting than Merb was 3 to 5 times faster than Rails. Honestly, it really depends on what you do. By using ActiveRecord on both prototypes, I limited the speed difference since AR is not multithread and therefore Merb can’t run as fast as it would using Sequel or DataMapper. By actually hitting the database on every single request, I also made sure to really compare ActionPack vs Merb.

Conclusion

The conclusion is simple, I recommended that my client go with Merb. Merb 1.0 is almost ready, the public API has been frozen. My client needs speed and simplicity. Using Merb I get exactly what I need and nothing more. Actually, we’ll probably increase the performance by writing a rack handler and bypassing the entire framework for API calls (that should be wicked fast!). Also, as soon as DataMapper becomes production ready, we’ll switch to DM and should get way better performance!

Am I suggesting to give up Rails and switch to Merb?
Absolutely not! First off, Merb is a “lower level” framework. It requires a deeper understanding of Web Development in general and being more than just ‘acquainted’ with the Ruby language. So, unless you are an advanced developer or have time to learn, I would suggest to keep on using Rails (start using Merb on personal projects, it’s a perfect way of learning).
If you have a lot of views and/or use loads of AJAX, RJS, built-in helpers, you probably want to stick to Rails and start looking at how you can do all of that from scratch. By default Rails uses nasty helpers that create inline javascript, and is something you really want to avoid. RJS is fun, but it goes against Merb’s philosophy, so you need to make sure you can live without it (note that you can reproduce the same behavior in Merb rendering JS, it just requries more work). If you rely a lot of Rails plugins, you might want to delay your switch, Merb is pretty new and doesn’t have a mass-load of plugins yet.

Finally, Merb doesn’t have a lot of documentation and changed a lot when 0.9 got released. To understand how Merb works, you will need to go through the source code, specs, Google, and ask on the Merb IRC channel.

It turns out that in our case we have experienced developers, a great need for speed, not too many views and are following Merb’s development really closely . I honestly think it’s the best choice for my client and I’m excited they accepted to use Merb.

Merb is addressing different issues than Rails and doing it well. I think there is a bright future for Merb. And don’t even think that Rails is going away, that won’t happen anytime soon!

Recently, Sony Playstation even posted a job post looking for a Rails/Merb developer. This is very promising for the Merb community!

My recent and certainly provocative post generated a lot of noise. Just by reading some of the comments on my blog or on reddit I could only notice that some people took offense and some people tried to push the metaphor way too far. At the same time, it generated a lot of interesting discussions even outside of our small Ruby community.

A few clarifications:

• Rails is an awesome framework, and to be honest I really like the fact that it’s accessible to newbies while offering great tools

• Rails Plugins that I like calling Duplo blocks are great and I use a bunch in most of my projects

• I, myself wrote a bunch of plugins and gems never as great as the one mentioned in my post though

• No, I don’t think you are dumb if you don’t have a clue how Rails or Rails plugins work

I also agree that the metaphor is very limited and doesn’t stretch… but who cares.. it’s funny.

What did I mean to say

• Rails plugins are useful, use them and ABUSE them. Even though most of them are very well written I strongly believe you would become a better developer if you understand how they work.

• A Duplo doesn’t do exactly what you want it to do, don’t bitch about it. Try submitting a patch or if you can write your own plugin.

• If you are using a plugin on a regular basis, give it some love. Submit some documentation, write a blog post, send a thank you email.

• A great framework and a bunch of nice plugins are awesome tools for developers but they won’t replace your brain.

So, there you go, this time around I tried to be more straight forward and I even apologize if I offended you.

Finally, I don’t believe the Rails community is a bunch of obnoxious-arrogant-wannabe-rockstars, and certainly not a ghetto. We all started as newbies and I hope a lot of newbies will join the fun and learn through Rails and later provide the community with a lot of awesome Duplos!

I’m sure, at least once in your life you played with Duplos.

Duplo bricks are eight times the size in volume, twice the length, height and width of traditional Lego bricks, and are easier to handle for younger children. Despite their size, they are still compatible with traditional Lego brick.

Duplos are great to introduce kids to the concept of Lego bricks and to get them to think about building their own creations. However you would freak out if your kid would grow up and not want to start playing with Legos and building more advanced/custom stuff.

Unfortunately, that’s exactly what’s going on in the Rails community right now. We created a generation of Duplo developers.

Rick Olson, AKA Technoweenie fathered a great majority of this Duplo generation. Rick is a Rails core member and a prolific Rails plugin developer. He has written very popular plugins and Rails apps such as:

• restful-authentication
• attachment_fu
• acts_as_versioned
• permalink_fu
• mephisto blog engine
• Beast forum

If Rick is the father, the mother of this generation would obviously be David Heinemeier Hansson, creator of the Ruby on Rails framework. David has always wanted to make our lives easier, providing us with tools to avoid repeating ourselves and a mass-load of tools to create web apps in no time.

Rick, David and others worked hard to provide the community with tools that cut our development times by 20% to 30% and that’s just awesome. They basically took their meccano applications and extracted Duplo blocks you can play with.

from to

The problem is that a generation of Rubyists has grown up being used to getting everything pre written for them. They haven’t yet passed the “Duplo stage” and basically write applications putting a few blocks together, only writing 10 to 20% and barely understand 5%.

On top of that, what really annoys me is that, these very same developers complain about the existing plugins, always ask for more and don’t give anything back to the community.

The problem is that it’s always the same people giving and helping. Relatively quickly, the community grows and people supporting it get tired. I won’t go as far as Zed and his funny rant but we need to wake up. We need to evolve, learn how Rails magic works, give up the cargo culting and start giving back.

The first thing would be to stop complaining about plugins you use on a daily basis and write your own or fork existing ones. GitHub is a good place to start forking existing projects, if you are a Rails/Ruby beginner, you can help with documentation or submit small patches. Josh Susser wrote a nice tutorial on how to commit changes (patches/documentation).

Why not blog about issues you have just faced and how you resolved them. Start writing small plugins/gems. Try helping people on the various mailing lists.

And finally, drop the Duplos and start playing with Legos – don’t use plugins just because they are available to you, make sure you fully understand what the plugins you use do! Learn more about Rails guts and start using it in a way that makes sense to you.

Why not even switch to meccano and take a look at Merb