I'm getting close to deploying an application built on Rails 3.1.x and started running some performance tests. After fiddling with ab for a bit, I'm seeing some very discouraging results yielding around 15 requests / second on Heroku.

When testing locally I see similar results that really demonstrates that it's an app issue more than anything.

I'm running Unicorn, which is about 40% faster than Thin on Celadon Cedar. Further, I'm using the PGSQL shared db.

I'm hopeful that someone could share a laundry list or essentially a launch checklist that I should move through when prepping an app for production and moving into the need for speed tuning. So far I've not found a real concise list of actionable items to move through that seems to make sense given my situation.

Or if you have solid practical experience moving through issues like this, any input would be appreciated!

There are some pretty low-hanging fruit that almost always yield pretty worthy performance gains:

1. Reduce the number of DB queries by using more efficient ActiveRecord statements. Be sure to use include and join where appropriate, and make sure you're using empty? over any? where possible to avoid SELECTs when you just need a COUNT.
2. Especially on heavier pages, cache views, even if only for a few minutes. You can often break larger or dynamic pieces into partials that can be cached without any negative effects, too.
3. Move any over-the-network activity to background jobs. This includes sending emails, fetching pages from anther website, and making API calls (even [especially?] to Heroku). There are a number of really good background job processing libraries in Ruby, DelayedJob is really popular because it works with any ActiveRecord database, but my favorite is Resque.

You need to be careful not to spend too much time optimizing Ruby routines. Unless you're doing something with a huge amount of data or processing (e.g. image resizing) you probably won't see very significant gains from optimizing loops or minimizing memory usage. And if you find certain pages are problematic, dig into your logs and see what is happening during those requests.

And if you're not already, autoscaling applications like HireFireApp are great for letting you handle loads of requests by scaling horizontally without the cost of running extraneous dynos during slow periods.

PS: There is a new Heroku Add-On called Blitz that lets you test a concurrent load of up to 5,000 users.