The Complete Guide to Rails Performance, Version 2
Today, the Complete Guide to Rails Performance has been updated to version 2.0. You can purchase it here.
All existing purchasers have had their copies updated on Gumroad. When I started this project, I always believed that a digital course should be better than a typical paperback programming book. That’s why I don’t include any DRM or proprietary video codecs. That’s why I think, like most software, updates should be free.
“Version 2.0” isn’t quite as drastic a change as a software v2.0, though. The world of Rails performance has actually changed very little since I wrote the course 2 years ago. The apps I consult on still have many of the same problems. The V2 update reflects this: I have revised the content for clarity, and updated a few places to reflect changes in Ruby 2.5 and Rails 5.2, but it is mostly still the same. I have also added four lessons: memory fragmentation, application server config, GC tuning, and PGBouncer config. These lessons were added based on new problems and thinking I’ve had since the course was released. Web-Scale Package purchasers will also get a new interview with Noah Gibbs of Appfolio next week.
So, what does it mean that not much has changed in the Rails performance world?
This tweet put me in an introspective mood this morning:
It is profoundly sad how Rails has institutionalized a "nobody cares" attitude toward performance. https://t.co/UhzvxyLjuz— Jeff Atwood (@codinghorror) June 1, 2018
To summarize, Jeff’s cofounder, Sam Saffron (who I interviewed for the CGRP), wrote a great, in-depth blog post about memory use in ActiveRecord. In short, Sam finds that ActiveRecord creates excessive amounts of objects, even when doing simple and supposedly “optimized” work. Sam posted a proof-of-concept patch which improves this quite a bit.
Jeff’s tweet diminishes the work of many Rails contributors. Aaron Patterson has spent the last two years working on Rails performance and a compacting garbage collector. Richard Schneeman has improved Sprockets’ performance a great deal. Sam Saffron himself has contributed over a dozen performance improvements to Rails, which, as far I can tell, have all been accepted. I know also that Andrew White, Eileen Uchitelle, and Rafael Franca are all Rails core members that care deeply about performance (probably because all of them have day-jobs running large Rails applications!). So any idea that Rails’ contributors or core members “don’t care” about performance is laughingly misguided, and is an opinion that can only really be held by someone outside the community. The way Jeff tried to turn it around in the replies into a “hot take” that people should “get angry” and “punk rock” about the “status quo” just made it more obvious.
It’s pretty easy to take potshots at a mature framework like Ruby on Rails. It has almost 13 years of history behind it. There’s going to be cruft, baggage, and outdated decisions baked in. That’s what happens. But there’s also tremendous productivity, something gained from the thousands of contributors who have all contributed their “lessons learned” back to the framework. But if you forget about that history, it’s easy to craft a benchmark to make it look like that history has overtaken it’s usefulness in the present.
This is the gap I’ve tried to bridge in my writing and in publishing The Complete Guide to Rails Performance. I believe that performance problems in Rails are pedagogical, not technical. It’s not because we don’t have enough people working on performance (though it helps!). It’s not because we don’t value it as a community (how many times do I have to cite all of the top 10,000 websites that run Rails at speed?). It’s because Rails (and Ruby) optimizes for programmer happiness, and that means we provide sharp tools which are easy to cut yourself on. Rather than throw the tools out, I think we need to teach people to use them safely.
ActiveRecord is probably the best example of what I’m talking about. It’s an extremely productive tool. It works very well for 80% of web-app use-cases. But every year, someone wants to throw it out and thinks that some other Rubygem or pattern (e.g. DataMapper) will save them. It’s so easy to craft a line of code with ActiveRecord that will slow your application to a crawl if you’re not thinking through through the consequences, as anyone who has written
User.all.each can tell you.
There is no One True Pattern or One True Framework. But there is a Thing Which Works For Most People. And if you end up being one of the 20% for whom it doesn’t work so well, or the tool’s productivity preference means that it’s easier to make performance mistakes, I don’t think that’s the tool or framework’s fault.
In this way, I think publishing the Complete Guide to Rails Performance was placing my faith in the developer community of Rails. If I didn’t think that people could make their Rails apps faster through knowledge and skills, and instead they had to wait until the framework or the language itself got faster, I would have gone to work at Github or Shopify and made a bunch of patches to Rails and Ruby. I might have started an alternative, “lightweight” framework or ORM that prioritized performance over usefulness and productivity. Instead, I think that teaching Rails developers how to find and fix performance problems will make a bigger dent in the average Rails app’s response time than improving the language or framework’s performance by even 2-3x, or by removing “dangerous” features.
As I think we’ve slowly discovered over the course of trying to make Ruby 3x faster, there is no “waste” or “bloat” that can be cut out of a framework or language without cost that suddenly makes the whole thing faster. It’s sort of like how politicians always promise to “cut waste in government spending”, but no-one can ever tell you exactly where or how which programs will be cut. Everything was implemented for a reason. There is no magic wand or amount of man-hours that can be waved at these problems. I’ve discovered this in my consulting and writing as well. I wish it was that easy. But it isn’t.
However, far from Jeff’s doomsday attitude, I believe that the macro picture for Ruby and Rails performance looks good, as it always had. Ruby 2.0 to 2.5 made a number of incremental performance improvements, particularly in garbage collection. I feel like the community has become more mature and performance-savvy over the last few years too. We’re waking up the mainstream Rails developer to things like
jemalloc and teaching them how to use ActiveRecord and avoid performance issues.
The technical future of Ruby looks strong, too. Ruby 2.6 will contain a JIT compiler. How cool is that? TruffleRuby has made great progress to becoming useable enough to run a Rails application. JRuby continues to truck along with more performance improvements and compatibility fixes all the time. The technical future of the language hardly looks dim - in fact, I think it’s much brighter than it was in 2011, when I got started in Ruby and Rails.
I’ll continue to do my part for the Rails performance community by publishing and writing, to improve the technical skills and capacity of the average Rails developer so that they can make their apps faster. Here’s to you, developers!
Want a faster website?
I'm Nate Berkopec (@nateberkopec). I write online about web performance from a full-stack developer's perspective. I primarily write about frontend performance and Ruby backends. If you liked this article and want to hear about the next one, click below. I don't spam - you'll receive about 1 email per week. It's all low-key, straight from me.
Products from Speedshop
The Complete Guide to Rails Performance is a full-stack performance book that gives you the tools to make Ruby on Rails applications faster, more scalable, and simpler to maintain.Learn more
The Rails Performance Workshop is the big brother to my book. Learn step-by-step how to make your Rails app as fast as possible through a comprehensive video and hands-on workshop. Available for individuals, groups and large teams.Learn more
I've written a new book, compiled from 4 years of my email newsletter.
MRI Ruby's Global VM Lock: frequently mislabeled, misunderstood and maligned. Does the GVL mean that Ruby has no concurrency story or CaN'T sCaLe? To understand completely, we have to dig through Ruby's Virtual Machine, queueing theory and Amdahl's Law. Sounds simple, right?
Programmers vaguely realize that 'premature optimization is bad'. But what is premature optimization? I'll argue that any optimization that does not come from observed measurement, usually in production, is premature, and that this fact stems from natural facts about our world. By applying an empirical mindset to performance, we can...
I've taught over 200 people at live workshops, worked with dozens of clients, and thousands of readers to make their Rails apps faster. What have I learned about performance work and Rails in the process? What makes apps slow? How do we make them faster?
Many Rails developers don't understand what causes ActiveRecord to actually execute a SQL query. Let's look at three common cases: misuse of the count method, using where to select subsets, and the present? predicate. You may be causing extra queries and N+1s through the abuse of these three methods.
NGINX Inc. has just released Ruby support for their new multi-language application server, NGINX Unit. What does this mean for Ruby web applications? Should you be paying attention to NGINX Unit?
Memory fragmentation is difficult to measure and diagnose, but it can also sometimes be very easy to fix. Let's look at one source of memory fragmentation in multi-threaded CRuby programs: malloc's per-thread memory arenas.
Application server configuration can make a major impact on the throughput and performance-per-dollar of your Ruby web application. Let's talk about the most important settings.
Did you miss Railsconf 2017? Or maybe you went, but wonder if you missed something on the performance front? Let me fill you in!
What happened at RubyConf 2016 this year? A heck of a lot of stuff related to Ruby performance, that's what.
WebFonts are awesome and here to stay. However, if used improperly, they can also impose a huge performance penalty. In this post, I explain how Rubygems.org painted 10x faster just by making a few changes to its WebFonts.
One of the most important parts of any webpage's performance is the content and organization of the head element. We'll take a deep dive on some easy optimizations that can be applied to any site.
New Relic is a great tool for getting the overview of the performance bottlenecks of a Ruby application. But it's pretty extensive - where do you start? What's the most important part to pay attention to?
Your website is slow, but the backend is fast. How do you diagnose performance issues on the frontend of your site? We'll discuss everything involved in constructing a webpage and how to profile it at sub-millisecond resolution with Chrome Timeline, Google's flamegraph-for-the-browser.
rack-mini-profiler is a powerful Swiss army knife for Rack app performance. Measure SQL queries, memory allocation and CPU time.
Most "scaling" resources for Ruby apps are written by companies with hundreds of requests per second. What about scaling for the rest of us?
Ruby apps in the memory-restrictive and randomly-routed Heroku environment don't have to be slow. Achieve <100ms server response times with the tips laid out below.