Why isn’t Unicorn the default webserver on Cedar if Cedar was explicitly designed with Unicorn-like webservers in mind?

One thought: dynos only have 512mb of RAM and so depending on your app’s memory footprint you might not even be able to run 2 Unicorn workers per dyno (let alone the 4 you would need to get reasonable throughput)

LEMON – Money Trees (Rap Genius Response to Heroku)

LEMON – Money Trees (Rap Genius Response to Heroku)

E.g., New Relic, which costs Rap Genius $8,000 / month, always reported 0ms spent in queue:

even though we were actually spending more time in queue than processing requests:

(we got these new measurements by installing our new gem, heroku-true-relic. See http://rapgenius.com/1506509 for more)

Rap Genius has released a gem called heroku-true-relic to patch New Relic to display the actual request queue times

These new accurate queue numbers confirm the results of our simulations: we are currently running 250 dynos (monthly bill: $27,000) with an average throughput of ~11000 requests per minute

A simulation with those numbers estimates that the average queue time should be around 290ms, which is very close to the 324ms average New Relic now reports. We don’t have a ton of data yet with the accurate request queueing, but that’s pretty close!

Heroku’s default webserver for Rails apps on both Bamboo and Cedar is thin, which is not concurrent.

This means that Heroku’s misstep affected every one of its Rails customers that didn’t change their webserver to Unicorn or Puma

(I’d be curious to know the actual percentage of Rails Cedar apps running on concurrent web servers)

Excerpts from Heroku’s docs:

The heroku.com stack only supports single threaded requests. Even if your application were to fork and support handling multiple requests at once, the routing mesh will never serve more than a single request to a dyno at a time.

See http://rapgenius.com/1501932

Heroku’s logs had an entry for the time a request spent in queue and that number was always 0. See http://rapgenius.com/1501395

Similarly, New Relic had stats on time spent in queue that were always 0:

I.e., when requests that are normally fast queue behind very slow requests at the dyno level:

This is hard to believe. When Tim Watson approached Heroku in February 2011 Heroku’s response was:

Tim,

You’re correct, the routing mesh does not behave in quite the way described by the docs. We’re working on evolving away from the global backlog concept in order to provide better support for different concurrency models, and the docs are no longer accurate. The current behavior is not ideal, but we’re on our way to a new model which we’ll document fully once it’s done.

In the meantime, you shouldn’t have any difficulties as long as you keep your web requests short (less than about 500ms), which is good practice anyway.

Sorry for any difficulty or confusion, and thanks for digging in and providing such a detailed analysis.

Adam