Rails on the Run http://railsontherun.com Rails experiments by Matt Aimonetti Tue, 23 Feb 2010 07:28:00 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 Speed up your Rails XML responses http://railsontherun.com/2010/02/22/speed-up-your-rails-xml-responses/ http://railsontherun.com/2010/02/22/speed-up-your-rails-xml-responses/#comments Tue, 23 Feb 2010 07:25:09 +0000 Matt Aimonetti http://railsontherun.com/?p=1734 At work, we have an XML API that gets quite a lot of traffic. Last week I looked into improving its performance since we are expecting more traffic soon and want to make sure our response time is optimized.

My first thought was to make sure we had  an optimized ActiveSupport’s xmlmini backend. Rails 2.3.5 fixed some issues when using Nokogiri as a xmlmini so I switched to my favorite Ruby XML lib:

ActiveSupport::XmlMini.backend = 'Nokogiri'

I run some benchmarks using ab, httperf and jmeter but the results were not that great. I was so sure that switching from rexml to nokogiri would give me awesome results that I was very disappointed.

I was about to call Aaron Patterson (Nokogiri’s author) to insult him, blame him for _why’s disappearance and tell him that all his pro bono efforts were useless since my own app was not running much faster when switched to his library. As I was about to dial his number on my iPhone I had a crazy thought… maybe it was not Aaron’s fault, maybe it was mine.

So I took a break went to play some fuzzball and as I was being ridiculed by Italian coworker, Emanuele, I realized that most of our API calls were just simple HTTP requests with no XML payload, just some query params. However, we were generating a lot of XML to send back to the client and AS::XmlMini only takes care of the XML parsing, not the rendering.

The XML rendering is done by Jim Weirich’s pure Ruby builder library which is vendored in Rails. Builder does a good job, but I thought that maybe a C based library might improve the speed. A coworker of mine (James Bunch) recommended to look into faster-builder, a drop-in Builder replacement based on libxml. Unfortunately, the project doesn’t seem to be maintained and I decided to look into using Nokogiri XML builder instead. (Also, faster-builder’s author doesn’t like me very much while Aaron knows he’s one of my Ruby heroes so asking for help could be easier)

Some people reported having tried using Nokogiri as a XML builder but didn’t see much speed improvement. Because of the amount of magic required to render a rxml template, I was not really surprised but I decided to contact Aaron and ask him if he tried using his lib instead of builder in a Rails app. Aaron told me he gave it a try a while back and he helped me get my Rails app setup to render xml templates using Nokogiri.

The next step was simple, create a benchmark app and benchmark Builder vs Nokogiri using various templates. Here are the results I got using Ruby 1.9.1 (the Ruby version we use in production) and two sets of templates:

Builder small template, time per request: 15.507 [ms] (mean)

$ ab -c 1 -n 200 http://127.0.0.1:3000/benchmarks/builder_small
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Finished 200 requests

Server Software:        nginx/0.7.65
Server Hostname:        127.0.0.1
Server Port:            3000

Document Path:          /benchmarks/builder_small
Document Length:        216 bytes

Concurrency Level:      1
Time taken for tests:   3.101 seconds
Complete requests:      200
Failed requests:        0
Write errors:           0
Total transferred:      114326 bytes
HTML transferred:       43200 bytes
Requests per second:    64.49 [#/sec] (mean)
Time per request:       15.507 [ms] (mean)
Time per request:       15.507 [ms] (mean, across all concurrent requests)
Transfer rate:          36.00 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    11   15   8.8     12      47
Waiting:        3   15   8.9     12      47
Total:         11   15   8.8     12      47

Percentage of the requests served within a certain time (ms)
  50%     12
  66%     12
  75%     13
  80%     13
  90%     35
  95%     36
  98%     38
  99%     41
 100%     47 (longest request)

Nokogiri small template, time per request: 15.354 [ms] (mean)

$ ab -c 1 -n 200 http://127.0.0.1:3000/benchmarks/noko_small
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Finished 200 requests

Server Software:        nginx/0.7.65
Server Hostname:        127.0.0.1
Server Port:            3000

Document Path:          /benchmarks/noko_small
Document Length:        238 bytes

Concurrency Level:      1
Time taken for tests:   3.071 seconds
Complete requests:      200
Failed requests:        0
Write errors:           0
Total transferred:      118717 bytes
HTML transferred:       47600 bytes
Requests per second:    65.13 [#/sec] (mean)
Time per request:       15.354 [ms] (mean)
Time per request:       15.354 [ms] (mean, across all concurrent requests)
Transfer rate:          37.75 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    11   15   8.6     12      39
Waiting:       11   15   8.6     12      39
Total:         11   15   8.6     12      39

Percentage of the requests served within a certain time (ms)
  50%     12
  66%     12
  75%     12
  80%     13
  90%     35
  95%     36
  98%     37
  99%     38
 100%     39 (longest request)

Running the benchmarks many times showed that Nokogiri and Builder were taking more or less the same amount of time to builder a small template.

I then decided to try a bigger template, closer to what we have in production, here are the results:

Nokogiri longer template, time per request: 31.252 [ms] (mean)

$ ab -c 1 -n 200 http://127.0.0.1:3000/benchmarks/noko
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Finished 200 requests

Server Software:        nginx/0.7.65
Server Hostname:        127.0.0.1
Server Port:            3000

Document Path:          /benchmarks/noko
Document Length:        54398 bytes

Concurrency Level:      1
Time taken for tests:   6.250 seconds
Complete requests:      200
Failed requests:        0
Write errors:           0
Total transferred:      10951200 bytes
HTML transferred:       10879600 bytes
Requests per second:    32.00 [#/sec] (mean)
Time per request:       31.252 [ms] (mean)
Time per request:       31.252 [ms] (mean, across all concurrent requests)
Transfer rate:          1711.00 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    24   31  11.3     26      62
Waiting:       23   30  11.3     24      61
Total:         24   31  11.3     26      62

Percentage of the requests served within a certain time (ms)
  50%     26
  66%     27
  75%     27
  80%     29
  90%     54
  95%     55
  98%     58
  99%     59
 100%     62 (longest request)

Builder, longer template, Time per request: 140.725 [ms] (mean)

$ ab -c 1 -n 200 http://127.0.0.1:3000/benchmarks/builder
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Finished 200 requests

Server Software:        nginx/0.7.65
Server Hostname:        127.0.0.1
Server Port:            3000

Document Path:          /benchmarks/builder
Document Length:        54376 bytes

Concurrency Level:      1
Time taken for tests:   28.145 seconds
Complete requests:      200
Failed requests:        0
Write errors:           0
Total transferred:      10947000 bytes
HTML transferred:       10875200 bytes
Requests per second:    7.11 [#/sec] (mean)
Time per request:       140.725 [ms] (mean)
Time per request:       140.725 [ms] (mean, across all concurrent requests)
Transfer rate:          379.83 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:   127  141  24.6    132     331
Waiting:      126  139  23.6    130     328
Total:        127  141  24.6    132     331

Percentage of the requests served within a certain time (ms)
  50%    132
  66%    138
  75%    147
  80%    149
  90%    156
  95%    169
  98%    193
  99%    311
 100%    331 (longest request)

Wow, @tenderlove’s Nokogori just brought us a 4.5X speed improvement for this specific template.  100ms per request is probably not a big deal for most people and I have to say that Jim did a great job with Builder. However in my specific case, 100ms on a request being called thousands of times per hour is quite important.

(The benchmark app is available on github, feel free to fork it and benchmark your own templates)

Who would have thought that a man like this could save the day?!

Aaron 'Tenderlove' Patterson





The moral of the story is that adding a bit of tenderlove to your Ruby code can make it perform much much better!

Thank you Aaron ‘Tenderlove’ Patterson!

]]>
http://railsontherun.com/2010/02/22/speed-up-your-rails-xml-responses/feed/ 5
Googlecharts 1.5.1 http://railsontherun.com/2010/02/03/googlecharts-1-5-1/ http://railsontherun.com/2010/02/03/googlecharts-1-5-1/#comments Thu, 04 Feb 2010 05:18:23 +0000 Matt Aimonetti http://railsontherun.com/?p=1722 To celebrate the relaunch of this site and since we are waiting for Rails 3.0 beta to be released, I figured I should share with you what I worked on the other night.

I merged patches, refactored and released a new version of googlecharts, my Gem to create graphs using Google Chart API.

sudo gem install googlecharts

Here is a quick example of how the API works when dealing with a complex graph:

require 'gchart' # or require 'googlecharts' if you prefer to use the Googlecharts constant.
title = "Player Count"
size = "575x300"
data = [85,107,123,131,155,172,173,189,203,222,217,233,250,239,256,267,247,261,275,295,288,305,322,307,325,347,331,346,363,382,343,359,383,352,374,393,358,379,396,416,377,398,419,380,409,426,453,432,452,465,436,460,480,440,457,474,501,457,489,507,347,373,413,402,424,448,475,488,513,475,507,530,440,476,500,518,481,512,531,367,396,423,387,415,446,478,442,469,492,463,489,508,463,491,518,549,503,526,547,493,530,549,493,520,541,564,510,535,564,492,512,537,502,530,548,491,514,538,568,524,548,568,512,533,552,577,520,545,570,516,536,555,514,536,566,521,553,579,604,541,569,595,551,581,602,549,576,606,631,589,615,650,597,624,646,672,605,626,654,584,608,631,574,597,622,559,591,614,644,580,603,629,584,615,631,558,591,618,641,314,356,395,397,429,450,421,454,477,507,458,490,560,593]
Gchart.line(:title => title, :size => size, :data => data, :axis_with_labels => 'x,y', :line_color => '1e60cc', :axis_labels => [(1.upto(24).to_a << 1)], :max_value => 700, :custom => 'chg=10,15,1,0')

Which provides you with the url or image tag (or downloaded file) that produces the following graph:

Google Chart

This release works great with Ruby 1.9 and MacRuby, lots of bugs got fixed and some new features were added. Something a lot of people complained was that the gem was called googlecharts but that the main class was called Gchart. The problem was that I wrote my gem and called it Gchart and when I went to register the rubyforge project page, the name was already taken. In this release, I fixed this problem by allowing users to require and use the constant name they want, Gchart or Googlecharts. I also spent quite a lot of time cleaning up the old code which I was a bit ashamed of. Class variables are now removed and overall, the code should be a bit more sane.

The source code can be found in my GitHub accout and the documentation there.

]]>
http://railsontherun.com/2010/02/03/googlecharts-1-5-1/feed/ 0
Rails and Merb core team working together http://railsontherun.com/2008/12/24/rails-and-merb-core-team-working-together/ http://railsontherun.com/2008/12/24/rails-and-merb-core-team-working-together/#comments Wed, 24 Dec 2008 08:47:00 +0000 Matt Aimonetti http://railsontherun.com/2008/12/24/rails-and-merb-core-team-working-together This is huge!

While people still try to find some drama an in hypothetical war between rails and merb.

The Rails team and the Merb team announced working together on a joined version of the 2 frameworks. This is so exciting, nobody believed it could ever happen (I personally, had my serious doubt).

Yehuda had a great post laying down the plan and explaining things in details. Check out David’s post explaining why he wants us to work together and his vision of a better Ruby world.

I have to say that I have been impressed by the Rails core team and especially David (DHH).

I’ve known David for few years now and we had long/heated discussions on topics like i18n/l10n for Rails. David is known to be a very opinionated person. But if you come up with the right arguments, he can be convinced and when that happens, he is willing to move forward and shake things up.

This merge is a concrete example that David and the rest of the Rails team cares about Rails and the Ruby community more than we usually give them credit for. As a unified team, we are going to change the way web development in Ruby is done!

But what does it mean for you?

I put together a FAQ video here is the transcript:

Hi, I’m Matt Aimonetti from the merb core team and as you might have heard, a big announcement was made earlier today.

I did this video to hopefully answer the questions you might have.

Q: So what’s the big news?

  • merb and rails team will work together on the next version of their frameworks
  • merb 2.0 and rails 3.0 share the same common endpoint
  • we realized we now have the same objectives and agreed on all the key principles.
  • merb will provide Rails with agnosticism, modularity, improved performance and a public API.
  • The end product will be called Rails 3.0 but what real matters is that it’s a huge gain for the entire community.

Q: What??? I thought there was a war going on between Rails and merb, what happened?

  • There was no war between rails and merb
  • We created merb because rails was not fitting us
  • We wanted better performance, more choices/ more modularity and a Public API.
  • The Rails team now embraces these concepts and want Rails to follow them, so why not work together?

Q: Wait, does that mean that merb is dead?

  • Absolutely not!
  • merb development won’t stop, we are going to keep on releasing updates until rails 3.0
  • clear migration path, and upgrading to rails 3.0 will be as trivial as upgrading from rails 2.x to Rails 3.0

Q: What does the timeline look like?

We just started getting together and discuss the technical details. We are shooting for a release at RailsConf 2009. However it’s hard to estimate this kind of things so again, that’s just an estimate :)

Q: I just started a merb project, so what now?

I’m sure you had valid reasons to use merb, you needed modularity, performance and a solid API.

Keep on using Merb, we won’t let you down. The good news is that the next version of merb (rails 3.0) will be even awesomer!

Q: What about my client who was looking at using merb for his new project?

If your client is going to be using merb for valid reasons (and not, just because it’s not rails) he should still use merb, but with the full understanding that he/she will end up using Rails in 6 months or so. Again, Rails 3.0 will have what pushed you to use merb.

Q: I’ve been involved with the merb-book, what will happen with this project?

  • rails 3.0 won’t get released right away
  • still need awesome documentation
  • if we look at rails 3.0 as merb 2.0, we can easily imagine how the current work can be extended to the new version.
  • rails team will also include an evangelism team which I will be part of, so will be able to focus more on projects like the book.

Q: I’ve been working on a merb plugin, what should I do?

Keep working on it! We’ll assist you with the migration process and the new solid API.

Q: What if I still have questions?

Come talk with me, or any members from the new team. We’ll be open to hear your questions, worries, frustrations.

merb always valued its developers and we will continue to do so but at a bigger scale.


Concretely, nothing changes for Merb users. People loving Merb should not worry. The way you build merb apps won’t change until merb2.0/rails3.0. We will still work on your favorite framework and improve it.

Lori Holden worked on merb_sequel and we should release a 1.0 release of the plugin in a few days.

I’m sure this news comes as a shock for many of you, but try to not see Rails 3.0 as the way Rails is right now. Imagine a version of Rails with true modularity and agnosticism (sequel, DM and AR will still be supported) and the same type of performance as what you get with Merb. In other words, the rails world is about to discover the power of merb!

Here is what Yehuda explicitly says in his blog post:

  • Rails will become more modular, starting with a rails-core, and including the ability to opt in or out of specific components. [...]
  • We will port all of our performance improvements into Rails. This includes architectural decisions that are big performance wins.[..]
  • As of Rails 3, Rails will have a defined public API with a test suite for that API. [..]
  • Rails will still ship with the “stack” version as the default (just as Merb does since 1.0), but the goal is to make it easy to do with Rails what people do with Merb today.
  • Rails will be modified to more easily support DataMapper or Sequel as first-class ORMs. [..]
  • Rails will continue their recent embrace of Rack, which [..] will improve the state of modular, sharable logic between applications.
  • In general, we will take a look at features in Merb that are not in Rails (the most obvious example is the more robust router) and find a way to bring them into Rails.

Personal perspective

I’m personally really excited about this opportunity. I had a hard time believing that we could work together but I was proved wrong. We have many challenges in front of us, but watching the two teams working together is very reassuring.

I’m also glad to see that we will have a Rails Evangelism team that I will be part of. I strongly believe that one of the many reasons why merb has been so successful is because we work and listen to our community. We have put a tremendous amount of energy trying to understand what you guys need and what you like and dislike. In return, we saw many people working hard on the wiki and the merb-book.

Can you imagine doing that with almost 4 Million Rails developers?

I’m also looking forward to working with a team and reaching to even more people.

Other news related to the merge:

If you have any questions, or if you want me to publicly answer questions on your blog please contact me. I’ll do my best to get back to everyone.

]]>
http://railsontherun.com/2008/12/24/rails-and-merb-core-team-working-together/feed/ 1
MerbCamp 2008 SanDiego http://railsontherun.com/2008/09/16/merbcamp-2008-sandiego/ http://railsontherun.com/2008/09/16/merbcamp-2008-sandiego/#comments Tue, 16 Sep 2008 06:27:00 +0000 Matt Aimonetti http://railsontherun.com/2008/09/16/merbcamp-2008-sandiego merbcamp

That’s now finally official, MerbCamp 2008 registration are open! What an exciting time!

History

To understand why I’m excited, we need to go back few months back. Merb was first released by Ezra has an alternative tool to handle file uploads. Merb came to reality because Ezra needed something fast, light and flexible to handle something that, let’s be frank about it, Rails had a hard time dealing with. Rails was king but was not as popular as now. Merb started as a simple Mongrel handler, in other words an alternative for small, light limited actions. Most people started using Merb simply to handle uploads. But as few cool kids started using Merb, they thought, hey, this thing is super fast, maybe I can use it to build small standalone apps. After all, hardcore developers don’t need “cool ajax helpers” and form builders to create a simple site. [Geoffrey Grosenback]() aka topfunky even proudly used Merb to reduild his site!
That was just enough to convince me to start using Merb back at version 0.3.4.

I was an active Rails user and contributor. Having to use a bare bone Ruby web framework was quite refreshing however the lack of testing framework was a real show stopper :( (Being hooked up on RSpec by Josh Knowles I ended up only writing a small portion of a Rails app with Merb 0.3.x (uploader backend).)

Quite quickly Merb’s philosophy changed and switched. The Mongrel handler framework started dreaming of becoming an alternative to Rails. Merb took the best from Rails but targeted another audience: the Ruby hackers living on the edge.
Merb prides itself in being less opinionated than Rails(that can be argued tho), ORM agnostic (supporting ActiveRecord, Sequel and DataMapper), Javascript framework agnostic and truly modular. People like Yehuda Katz, Michael Klishin aka antares got involved, as more contributors joined the effort, rules were enforced to make sure the framework would be as fast as possible and easy to extend without monkey patching. (ohh and fully tested using RSpec ;)

Engine Yard decided to support the development effort and helped with Merb’s major rewrite (0.9 versions). Today, Merb is divided in 3 repositories, merb-core, merb-more and merb-plugins. By letting developers only choose what they want to use and by following a principle of isolation with private/public APIs, I believe Merb is today the most flexible yet powerful Ruby framework available.
Furthermore, even though many people don’t understand the purpose of rewriting a “new Rails” from scratch, the reality is that many progress made by the merb team were ported back to Rails and inspired others (DataMapper for instance)!

Anyway, this is not a sale’s speech and I’m not trying to convince you to use Merb. My point is that Merb is finally coming to a point where the public API is stable and where one would find most tools he needs to build production ready applications.
And, that’s basically a long sum up explaining why I wanted to organize something special to celebrate the 1.0 release and to create more awareness around Merb’s awesomeness!

The Team behind MerbCamp

I started getting involved with the SDRuby community a couple of years ago. As I got to know more people I realized that many people lead by Patrick Crowley (leader of SDRuby and one of the organizers of SD BarCamp) had the desire to organize a local Ruby conf/camp.

At the same time, while I was working daily with Merb and contributing back to Merb’s code, many other SDRuby fellows were also getting really excited about Merb (Rob Kaufman, Ryan Felton to mention a few).

Seeing the opportunity to host the very first Merb event in San Diego (host of RubyConf 2005!) I chatted with yehuda and the rest of the merb team. All the merb people were really excited, Leah Sibler from Engine Yard even offered her expertise to organize such an event (she’s totally awesome at planning/running conferences).

However, setting up such an event isn’t something one can do on his own. Before promising anything, I checked that Rubyists from San Diego would be interested and would help. In no time, I got a lot of people offering to help.

The key thing for me was to get someone with a good experience in organizing conferences. A person with resources and contacts. The only person I knew in San Diego who would be good enough to do that was Patrick Crowley. We had a quick chat Patrick and I and it turned out that Patrick was very excited about organizing the very first MerbCamp in his town.
Patrick quickly got a team together who agreed on working on the project. We got back to the Merb team and sealed the deal.

Patrick even found the awesome venue that many other cities will envy us! He’s been running the show, running here and there, making phone calls to make sure registration would open on time, setup the website etc…. Thanks Patrick!

The Conf

MerbCamp will be an hybrid between a BarCamp, a conference and an unconference. When the organization team got together, we all agreed that what we like the most during conferences is networking. We certainly also enjoy some good talks and definitely enjoyed the hack-room during the last Rails Conf. We therefore decided to organize a conf *we* would love to go.

  • 1 scheduled track with “official talks” to make sure we have some serious content and to motivate people to signup ;)
  • BBQ at the beach, because we live in San Diego and we love that! (plus, big open meals are the best way to network)
  • BarCamp type impromptu talks
  • hack-rooms so people can work together
  • friendly and small conference (we limited the amount of participants to 200)

To conclude, I hope the “history” of MerbCamp 08 wasn’t too boring. People seem quite excited about this event, we even have guys in London who would get together to watch the talks via a webcam we are going to setup for them.
We hope to see you there, if not, we hope you’ll organize your own conference and we will come have fun with you.

]]>
http://railsontherun.com/2008/09/16/merbcamp-2008-sandiego/feed/ 0
db fixtures replacement solution step by step http://railsontherun.com/2008/09/07/db-fixtures-replacement-solution-step-by-step/ http://railsontherun.com/2008/09/07/db-fixtures-replacement-solution-step-by-step/#comments Sun, 07 Sep 2008 16:06:00 +0000 Matt Aimonetti http://railsontherun.com/2008/09/07/db-fixtures-replacement-solution-step-by-step Like most people who started with Rails a while back, I first loved Rails fixtures and ended up hating them (slow, a pain to maintain etc…).

I went through different experiments, trying different existing libs, writing my own solutions etc… I wasn’t quite satisfied until I found factory_girl from thoughtbot.

You might not feel the need for a decent fixtures solution if you do a lot of mocking/stubbing, but I recently came back from my “mock everything you can outside of models” approach and I’m getting closer to the mock roles, not objects approach. So, I’m loosing my model/controller testing separation but I’m gaining by not having to maintain “dumb mocks” which don’t always represent the real API behind. I mean, how many times did I change a Model, messing up my app but all my specs were still passing. Anyway, that’s a long discussion, which will be covered by wycats during merbcamp

So here is a simple example of how I use factory girl in a Merb + DataMapper app. (you can do the same in a Rails app, there is nothing specific to Merb in factory_girl).

  • I. create an empty app, set the ORM etc…
  • II. git pull and install factorygirl from http://github.com/thoughtbot/factorygirl/tree/master. Or install thoughtbot-factory_girl gem using GitHub gem server.
  • III. create a spec/factories.rb file. (You might prefer to create a folder called spec/factories and add a factory per model)
  • IV. modify spec_helper.rb and add the following
  require 'factory_girl'
  require File.dirname(__FILE__) + '/factories/index.html'
  • V. write some specs against a Client model

  • VI. Create the Model

  • VII. create a factory

  • IIX. run your specs

    failing specs

  • IX. fix the model (note that I set dependencies "dm-validations" in my init.rb)

  • X. run the specs

    passing specs

  • XI. add more specs

As you can see, Factory.build(:client) only creates a new instance of the Object, while Factory(:client) creates, saves and loads the instance.

  • XII. get them to pass

Factory Girl makes fixtures simple and clean. Here is another example for creating associations:

Factory Girl also supports sequencing, check out FG read me

In conclusion, Factory Girl is a mature and solid factory solution which will take you less than 15 minutes to get used to. It will offer you loads of flexibility and less frustration than good old yaml fixtures. You can also use it with existing fixtures if you want to start using it in an existing app.

]]>
http://railsontherun.com/2008/09/07/db-fixtures-replacement-solution-step-by-step/feed/ 4
Ruby developers don’t scale http://railsontherun.com/2008/08/27/ruby-developers-don-t-scale/ http://railsontherun.com/2008/08/27/ruby-developers-don-t-scale/#comments Wed, 27 Aug 2008 14:58:00 +0000 Matt Aimonetti http://railsontherun.com/2008/08/27/ruby-developers-don-t-scale Wow, it’s been a while since I blogged. With all the cool kids saying that spending time reading RSS feeds is overrated (see Defunkt’s keynote for instance) I even wonder if people will ever read this post!

Anyways, I have been quite busy preparing courses for classes I gave to a bunch a great Engineers at one of the Fortune 100 companies based in San Diego. I was also planning my big vacation trip to Europe and wrapping up few projects.

However, during my exile overseas, I came to the conclusion that Rubyists don’t scale. Since Twitter became stable again, we don’t hear many people ranting about Rails not scaling anymore. With one of my clients’ app handling around 7 million requests/day I can tell you Ruby/Merb do scale quite well! But ruby developers don’t seem to scale for some reason.

Maybe saying that we(Rubyists) don’t scale isn’t technically correct but that’s basically what one of my client told me.

Let’s go back in time a little bit and follow my client who we will call clientX.

  • ClientX has a great concept and wants to conquer the internet.
  • ClientX hears that Rails is the way to go.
  • ClientX hires a contractor/mercenary/freelancer/guns for hire/consultant (aka Me)
  • Me builds a killer app using Merb (killing framework)

  • ClientX raises loads of $$$

  • ClientX wants to hire a team because Me doesn’t want to become a FTE

  • ClientX and Me look for Rubyists wanting to relocate and get a decent salary
  • ClientX *can’t find someone they consider good enough and who would accept their package

  • Many JAVA guys are available on location and accept lower packages

  • Ruby app gets ported over to JAVA
  • Me sad :(

So is it really the Rubyists’ fault if we don’t want to relocate and only accept higher packages? Should I blame Obie for telling people to charge more and teaching how to hustle? Or should we just tell clients that it’s time to get used to working remotely?

Honestly, I don’t think any of the above explanations are valid. Ruby is the new/hot technology and very few people have the skills and experience to lead major projects. These people make a good living and enjoy their “freedom” and dream of building their own products. Most of them/us value their work environment, family and are reluctant to move.

scale

At the same time, companies do need people locally(at least a core team) and can’t always afford the cool kids.

ClientX, quite frustrated by the whole hiring process told me once: “you Ruby folks are too unavailable and difficult to work with! We need a committed team that actually cares about the company/product.”

That hurts when you worked hard on a project and just can’t satisfy the client by finding guys willing to relocate and work for them. It gets even more painful when your code gets entirely ported over to JAVA!

But at the same time I understand ClientX’s motivation, PHP guys are cheaper, JAVA guys are more available, why in the word did we go with Ruby and are now struggling finding people?

Once again, there is positive and negative side in everything, by choosing Ruby and a “great contractor” ClientX was able to catch up with the competition and even pass them in no time. They quickly raised good money and got everything they needed to become #1. I don’t believe it would have been possible to do the same thing so quickly with JAVA for instance. However choosing a cutting edge technology means you need to look harder for talented people.

It’s too bad the code gets rewritten in a different language but at the same time, I do my best to facilitate the process and to keep a good relation with my client. There was nothing personal in the decision, it’s just too bad we were not able to keep on using the latest/coolest/awesomess technology available :)

To finish on a positive note, here is the solution to scale your Ruby task force provided to you by the #caboose wisdom:

Based on my conversations with other #caboosers who hire other devs, the word in the street is that you just need to get one or two great ruby guys (who will probably cost you a lot) and find a bunch of smart people to train. You’ll end up with an awesome team of scalable rubyists ;)

]]>
http://railsontherun.com/2008/08/27/ruby-developers-don-t-scale/feed/ 26
Googlecharts featured on Github http://railsontherun.com/2008/06/27/googlecharts-featured-on-github/ http://railsontherun.com/2008/06/27/googlecharts-featured-on-github/#comments Fri, 27 Jun 2008 07:06:00 +0000 Matt Aimonetti http://railsontherun.com/2008/06/27/googlecharts-featured-on-github Github, probably the most famous social code hosting service just redesigned their homepage and are now featuring hosted projects.

I got a very good surprise when Takeo from Powerset & Stafftool hall of fame mentioned to me that Github picked one of my gems as the first featured project!

github

By the way, Takeo is also a Googlecharts contributor (+ a Merbist) and I had the honor to be the first one he ever forked!

Another Googlecharts user, Mokolabs from Graffletopia and iCal Share also decided to try Git and Github. In no time he had forked my project, made some modifications and sent me a pull request. w00t w00t!

To celebrate, we released version 1.3.4 with cleaner documentation, and enhanced features.

Documentation & Code Repo

Thanks to everyone involved in this project. And special kudos to the GitHub team for offering such an awesome service!

]]>
http://railsontherun.com/2008/06/27/googlecharts-featured-on-github/feed/ 0
About Metaprogramming speed http://railsontherun.com/2008/06/18/about-metaprogramming-speed/ http://railsontherun.com/2008/06/18/about-metaprogramming-speed/#comments Wed, 18 Jun 2008 07:03:00 +0000 Matt Aimonetti http://railsontherun.com/2008/06/18/about-metaprogramming-speed In a previous article I took an example of bad metaprogramming and I pushed people to think twice before using metaprogramming.

My main points were that:

  • you might make your code way slower if you don’t know what you are doing
  • readability might drop considerably
  • maintainability can become an issue

People left some very good comments about how to write the same module using metaprogramming and keep things fast.

Today Wycats pinged me about this post and told me that the issue was definemethod and that classeval is effectively the same as regular code, it gets evaluated in eval.c, just like regular Ruby code. On the other hand, defined_method has to marshall the proc.

I cleaned up my benchmarks using rbench, added some of the solutions provided to me and obtained the following results:

results

Here is the original/bad metaprogramming example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  module MetaTimeDSL

    {:second => 1, 
     :minute => 60, 
     :hour => 3600, 
     :day => [24,:hours], 
     :week => [7,:days], 
     :month => [30,:days], 
     :year => [364.25, :days]}.each do |meth, amount|
      define_method "#{meth}" do
        amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
        self * amount
      end
      alias_method "#{meth}s".intern, "#{meth}"
    end

  end
  Numeric.send :include, MetaTimeDSL

The no metaprogramming module is available there

Refactored:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

  module RefaMetaTimeDSL

  {:second => 1, 
   :minute => 60, 
   :hour => 3600, 
   :day => [24,:hours], 
   :week => [7,:days], 
   :month => [30,:days], 
   :year => [364.25, :days]}.each do |meth, amount|
    self.class_eval <<-RUBY
      def r_#{meth}
        #{amount.is_a?(Array) ? "#{amount[0]}.#{amount[1]}" : "#{amount}"}
      end
      alias_method :r_#{meth}s, :r_#{meth}
    RUBY
  end

end
Numeric.send :include, RefaMetaTimeDSL

the refactor 2 or eval based solution provided by Matt Jones which uses class_eval like the previous refactor.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  module EvalMetaTimeDSL
    def self.included(base)
      base.class_eval do
        [ [:e_second, 1], 
          [:e_minute, 60], 
          [:e_hour, 3600], 
          [:e_day, [24,:e_hours]], 
          [:e_week, [7,:e_days]], 
          [:e_month, [30,:e_days]], 
          [:e_year, [365.25, :e_days]]].each do |meth, amount|
            amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
            eval "def #{meth}; self*#{amount}; end"
            alias_method "#{meth}s", meth
          end
      end
    end
  end
  Numeric.send :include, EvalMetaTimeDSL

and finally, the “better metaprogramming” version:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

 module GoodMetaTimeDSL

  SECOND  = 1
  MINUTE  = SECOND * 60
  HOUR    = MINUTE * 60
  DAY     = HOUR * 24
  WEEK    = DAY * 7
  MONTH   = DAY * 30
  YEAR    = DAY * 364.25
  
  %w[SECOND MINUTE HOUR DAY WEEK MONTH YEAR].each do |const_name|
      meth = const_name.downcase
      class_eval <<-RUBY 
        def g_#{meth} 
          self * #{const_name} 
        end 
        alias g_#{meth}s g_#{meth} 
      RUBY
  end
end
Numeric.send :include, GoodMetaTimeDSL

Looking at the refactored version by Wycats, you can see he’s right and the major issue with the original version was definemethod. Using classeval does make things almost as fast and even faster than the no metaprogramming version.

Interesting enough, the benchmarks show that some methods from the meta modules are faster than the ones from the no meta module. Overall, an optimized metaprogramming can be more or else as fast as a non meta code. Of course, with the new VMs coming up, things might change a little bit depending on the language implementation.

In conclusion, metaprogramming can be as fast as no metaprogramming but that won’t help your code readability and maintainability, so make sure to only use this great trick when needed!

p.s: here is the benchmark file if you don’t believe me ;)

]]>
http://railsontherun.com/2008/06/18/about-metaprogramming-speed/feed/ 4
News update http://railsontherun.com/2008/06/18/news-update/ http://railsontherun.com/2008/06/18/news-update/#comments Wed, 18 Jun 2008 01:40:00 +0000 Matt Aimonetti http://railsontherun.com/2008/06/18/news-update I realized I haven’t updated this blog in a while. Here is a quick update on what’s happened and on things to come:

  • RailsConf 08. Great conference, probably my last Rails Conf though. I’ll be in Orlando for Ruby Conf 08 and I’ll focus on 1 or 2 local conferences (probably mountain west and another one).

  • MerbCamp 08 in San Diego this Fall organized by SD Ruby. Details are not finalized yet but Yehuda Katz announced it during his Merb talk at RailsConf.

  • Moved this blog to a new Joyent accelerator with git support and finally have the possibility to use Ambition! (planning on moving from Mephisto to Feather)

  • Launched a client’s Merb app and getting around 3 million hits/day. Merb is just awesome. (more info when the client’s app gets out of beta)

  • I’ll join Gregg Pollack from http://railsenvy.com/ during Qcon and take part in the Ruby for the Enterprise track. My talk will focus on Merb usage in real life.

  • Renamed my github username, new repo url: http://github.com/mattetti (sorry about that)

]]>
http://railsontherun.com/2008/06/18/news-update/feed/ 2
Avoid using metaprogramming (seriously!) http://railsontherun.com/2008/05/04/avoid-using-metaprogramming/ http://railsontherun.com/2008/05/04/avoid-using-metaprogramming/#comments Sun, 04 May 2008 08:29:00 +0000 Matt Aimonetti http://railsontherun.com/2008/05/04/avoid-using-metaprogramming Ruby is sexy, Ruby is cool and its metaprogramming potential offers some really cook features. However you might not realize that your cleverness is slowing down your code.

Today I was working on cleaning up merb_helper a Merb plugin that brings a lot of the stuff Rails developers are used to. In Merb we aim for speed and try to avoid magic.

merb_plugin didn’t receive a lot of love from the main contributors but few features were added by different contributors and the code became hard to maintain.

Looking at the code I quickly found this bad boy:

(Old Merb Time DSL using metaprogramming)

module MetaTimeDSL

    {:second => 1,
     :minute => 60,
     :hour => 3600,
     :day => [24,:hours],
     :week => [7,:days],
     :month => [30,:days],
     :year => [364.25, :days]}.each do |meth, amount|
      define_method "m_#{meth}" do
        amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
        self * amount
      end
      alias_method "m_#{meth}s".intern, "m_#{meth}"
    end

  end
  Numeric.send :include, MetaTimeDSL

The above code looks awful to me and I decided to rewrite it a way I thought would be more efficient:

 module TimeDSL

    def second
      self * 1
    end
    alias_method :seconds, :second

    def minute
      self * 60
    end
    alias_method :minutes, :minute

    def hour
      self * 3600
    end
    alias_method :hours, :hour

    def day
      self * 86400
    end
    alias_method :days, :day

    def week
      self * 604800
    end
    alias_method :weeks, :week

    def month
      self * 2592000
    end
    alias_method :months, :month

    def year
      self * 31471200
    end
    alias_method :years, :year

  end
  Numeric.send :include, TimeDSL

To make sure I was right, I run the following benchmarks:

require 'benchmark'
TIMES = (ARGV[0] || 100_000).to_i

Benchmark.bmbm do |x|

  x.report("metaprogramming 360.seconds") do
    TIMES.times do
      360.m_seconds
    end
  end

  x.report("no metaprogramming 360.hours") do
    TIMES.times do
      360.seconds
    end
  end

  x.report("metaprogramming 360.minutes") do
    TIMES.times do
      360.m_minutes
    end
  end

  x.report("no metaprogramming 360.minutes") do
    TIMES.times do
      360.minutes
    end
  end

  x.report("metaprogramming 360.hours") do
    TIMES.times do
      360.m_hours
    end
  end

  x.report("no metaprogramming 360.hours") do
    TIMES.times do
      360.hours
    end
  end

  x.report("metaprogramming 360.days") do
    TIMES.times do
      360.m_days
    end
  end

  x.report("no metaprogramming 360.days") do
    TIMES.times do
      360.days
    end
  end

  x.report("metaprogramming 360.weeks") do
    TIMES.times do
      360.m_weeks
    end
  end

  x.report("no metaprogramming 360.weeks") do
    TIMES.times do
      360.weeks
    end
  end

  x.report("metaprogramming 18.months") do
    TIMES.times do
      18.m_months
    end
  end

  x.report("no metaprogramming 18.months") do
    TIMES.times do
      18.months
    end
  end

  x.report("metaprogramming 7.years") do
    TIMES.times do
      7.m_years
    end
  end

  x.report("no metaprogramming 7.years") do
    TIMES.times do
      7.years
    end
  end

end

 Rehearsal ------------------------------------------------------------------
metaprogramming 360.seconds      0.130000   0.000000   0.130000 (  0.133164)
no metaprogramming 360.hours     0.050000   0.000000   0.050000 (  0.042655)
metaprogramming 360.minutes      0.130000   0.000000   0.130000 (  0.133327)
no metaprogramming 360.minutes   0.040000   0.000000   0.040000 (  0.042401)
metaprogramming 360.hours        0.140000   0.000000   0.140000 (  0.134312)
no metaprogramming 360.hours     0.040000   0.000000   0.040000 (  0.043125)
metaprogramming 360.days         0.130000   0.000000   0.130000 (  0.134949)
no metaprogramming 360.days      0.050000   0.000000   0.050000 (  0.043745)
metaprogramming 360.weeks        0.130000   0.000000   0.130000 (  0.135581)
no metaprogramming 360.weeks     0.050000   0.000000   0.050000 (  0.043544)
metaprogramming 18.months        0.130000   0.000000   0.130000 (  0.135234)
no metaprogramming 18.months     0.050000   0.000000   0.050000 (  0.044354)
metaprogramming 7.years          0.140000   0.000000   0.140000 (  0.144062)
no metaprogramming 7.years       0.050000   0.000000   0.050000 (  0.044392)
--------------------------------------------------------- total: 1.260000sec

                                     user     system      total        real
metaprogramming 360.seconds      0.130000   0.000000   0.130000 (  0.132567)
no metaprogramming 360.hours     0.040000   0.000000   0.040000 (  0.042777)
metaprogramming 360.minutes      0.140000   0.000000   0.140000 (  0.132554)
no metaprogramming 360.minutes   0.040000   0.000000   0.040000 (  0.043193)
metaprogramming 360.hours        0.130000   0.000000   0.130000 (  0.133027)
no metaprogramming 360.hours     0.050000   0.000000   0.050000 (  0.042613)
metaprogramming 360.days         0.130000   0.000000   0.130000 (  0.138637)
no metaprogramming 360.days      0.050000   0.000000   0.050000 (  0.043213)
metaprogramming 360.weeks        0.130000   0.000000   0.130000 (  0.134049)
no metaprogramming 360.weeks     0.040000   0.000000   0.040000 (  0.043713)
metaprogramming 18.months        0.140000   0.000000   0.140000 (  0.134941)
no metaprogramming 18.months     0.040000   0.000000   0.040000 (  0.043980)
metaprogramming 7.years          0.150000   0.000000   0.150000 (  0.143389)
no metaprogramming 7.years       0.040000   0.000000   0.040000 (  0.044585)
 0.136591)

The metaprogramming version of the same implementation is almost 3 times slower!

Moral of the story: be careful when using metaprogramming, you might end up slowing down your code considerably.

]]>
http://railsontherun.com/2008/05/04/avoid-using-metaprogramming/feed/ 13