# Speedshop - Full Content > This file contains the full markdown content of all blog posts from Speedshop, a Ruby on Rails performance consultancy. --- ## Organization for Transformative Works Performance Audit URL: https://www.speedshop.co/blog/performance-lessons-from-ao3/ What follows is the complete text of an audit report I produced for the Organization for Transformative Works, a non-profit which was for many years run one of the largest fan-fiction archives on the Internet. This was produced as part of my [Ruby on Rails performance retainer service](https://www.speedshop.co/retainer.html). Since the app is open source, OTW has kindly agreed to let me publish the report online for all to see. The [Archive Of Our Own](https://github.com/otwcode/otwarchive), also known as "AO3", is a Rails application that is more than 16 years old. It was the focus of my engagement. Without any further ado, here's what I handed over to OTW and their crack team of volunteers. I think I am one of a very, very small number of people who has ever been _paid_ to work on AO3. The sheer amount of volunteer love and sweat in this project is truly astounding. ## The Report It's an honor to work on behalf of such an engaged community like the one surrounding AO3. It's also a tremendous honor to be one of the few people (maybe ever?) to get paid to work on this application. What you've accomplished with the volunteer time you have is truly incredible. For some context, AO3 is certainly one of the largest scale applications I've worked on in terms of requests per second. Certainly, in terms of just requests per dollar spent on the server rack, it's one of the most efficient I've ever worked on. I see three main areas for improvement: * Removing footguns which could potentially result in incidents in the future * Make the app feel faster for your users * Targeted reduction of technical debt To do this, I strongly suggest tracking and improving these two numbers: 1. The number of active, non-idle database connections to the primary. 2. Controller's p95 web transaction service time. > **AI Disclosure:** In compiling this report, I made extensive use of LLMs for research. I did not use LLMs to generate any of the text in this report. This document is organized at the top level by our desired Outcomes, which are my goals for your performance improvements over the next six months. Underneath that are specific Recommendations to achieve those outcomes. Each Recommendation has an associated cost and benefit, rated subjectively on a 5 point scale. Ratings are designed mostly to be relative to each other, i.e. a 5 is always harder or more valuable than a 4, etc. Even cost/benefit roughly means I think it's a toss-up whether or not you should do it, while a cost higher than the benefit rating means I think it's not worth doing in the near term future. I hope you enjoy this document and find it a useful guide for the next 6-12 months of performance work on your application. ## Outcome: Remove Footguns In the intro, I said you should focus on **reducing the number of active, non-idle database connections** to the primary. I see this as your primary scale bottleneck. If that number consistently stays at or below ~1/2 the number of CPUs on the DB (which, IIRC, was like 196 or something?) you'll be golden! Easy as that, right? This section is mostly about reducing opportunities for downtime (though not all directly relate to that DB metric). ### Recommendation: Remove puma_worker_killer. Cost: 1 Benefit: 2 You have `puma_worker_killer` in the `Gemfile`. There's no other config in the gem for it, but this gem is frequently configured with env variables. PWK's job is to `SIGTERM` processes which go over a particular memory threshold. This sounds like a nice-to-have "safety" feature, but it's a footgun in waiting. Several years ago I was consulting on a client project and the app was just really slow. I was looking at the response times and thinking, this app is about twice as slow as it probably should be. What's going on? We removed `puma_worker_killer` and watched response times drop in half instantly. It turned out that someone had installed and configured `puma_worker_killer` years ago and had forgotten about it. As the app grew over time, it continued to _work_, but the baseline memory usage of the app just kept increasing, leading to more and more frequent process restarts triggered by `puma_worker_killer`. In the end, it turned out Puma processes were processing about 20 requests on average before being restarted. Old processes are fast processes. We want processes to stick around for as long as possible. `puma_worker_killer` combines two things which are almost always _very difficult_ to observe as an SRE: 1. Configuration in ENV variables. ENV variables are almost always forgotten about. No one knows what they are and what they are set to, because they often contain secrets. Access to them is restricted as a result, and they're not easily checked or in the front of anyone's mind. 2. The "failure mode" is high process restarts, which is almost never properly observed. If PWK's thresholds are set too low, the only way you're going to find out is if you look at how often process restarts are happening and think: hm, that's a bit high (which, you may not even realize after looking at it). Your app could be restarting _after serving only a single request_ and you might not notice it, if load is low enough and you've provisioned enough puma processes. `puma_worker_killer` is sort of a relic from a Heroku-dominant era, where you needed to fit processes into teeny-tiny dynos of like 512MB. Nowadays, most people buy cloud hardware with 4GB of RAM per CPU, and an application which cannot fit within that constraint reliably is far more sick than something `puma_worker_killer` can even address. For that reason, I think `puma_worker_killer` should be removed from the setup. ### Recommendation: Change send_data to a background job powered workflow. Cost: 2 Benefit: 3 You have a couple places in the codebase where you're using `send_data` to send files of fairly significant size down to the client: 1. **DownloadsController#show** 2. **send_csv_data** helper, used by ChallengeSignupsController, TagWranglers controller, AdminUsers controller. There are two big issues with what's happening here. * **Memory usage**. You're loading ~several mb of stuff into memory at once to put some of these CSVs together. In a web process, that memory usage sticks around and is difficult to get rid of. In a background process, usually this kind of "dirtiness" in the heap is already occurring so you're not making things much worse, and it's in general easier to stop and restart background job processes. * **Transmitting large files to the client is slow** Ruby web servers aren't really designed for this. Puma does not buffer client responses and so you're essentially locking up the thread for as long as it takes to stream this response back to the client. It's worse for single-threaded situations (which you're in I think, for web today). Both of these I think could get out of hand and cause some minor incidents. For me, the best path forward would be to reorient the whole thing around a background job: 1. Controller kicks off a job to generate the CSV. 2. Controller returns to the web client a URL and says "your CSV will be here when it's done". 3. Background job does the CSV work and uploads to S3. 4. S3 serves the CSV. This would solve both problems for you at once. You could optionally try to work to stream the CSV upload as well to reduce memory usage in the worker, but honestly I think just getting it out of the web process is enough. ### Recommendation: Install some kind of database monitoring product Cost: 1 Benefit: 3 I think of observability stacks for Rails app as having three main components: 1. **Infrastructure**. Most of the time this is coupled to your provider, e.g. AWS Cloudwatch, but can include 3rd parties like Datadog. 2. **APM/RUM**. Options include Sentry, Datadog, New Relic, etc. 3. **Database**. Options include pganalyze, pghero, Datadog's Database Monitor. I feel like we're missing the last one, particularly for an app like this where performance and stability is so sensitive to DB conditions. I'm flying a bit in the dark here because I think there are really good options here for Postgres but less so for InnoDB/MySQL. It is, of course, possible to use _only parts_ of Datadog and not the whole hog, and I believe DD's database monitoring is like $70/80 a month. I _think_ the only open source option here is [Percona PMM](https://github.com/percona/pmm)? But I've never used it and can't say if it's any good, and it doesn't have an index advisor. A good database monitoring solution does the following: 1. Index suggestion. Based on actual query data, can do things like gauge how much impact a new index has on writes, how big it would be, what queries it would alleviate. 2. Capture and visualize EXPLAIN plans. 3. CPU, Memory, active connection count metrics. I'm not really sure what the best direction to go here is, but I was definitely feeling the absence of this tooling when preparing this report. ### Recommendation: Move load to readers with queues and GET requests. Cost: 3 Benefit: 3 I see a lot of people create replicas and then basically struggle to direct any meaningful usage to them. You're bottlenecked pretty much entirely by your SQL primary, so moving more usage to them would be helpful. I've only seen two strategies really work here. The first is to create **read only job queues**. These are job queues which are duplicates of an existing queue, like `high`, but only allow jobs in that queue to talk to the replica database using ActiveRecord's multi-db/role support. So, instead of just a `utilities` queue, you might also have a `utilities-read-only` queue. You'd have an `around_perform` hook in ApplicationJob that does something like: ```ruby def use_replica_if_read_only if read_only_queue? begin ActiveRecord::Base.connected_to(role: :reading) { yield } rescue ActiveRecord::ReadOnlyError self.class.set(queue: writable_queue_name).perform_later(*arguments) end else yield end end def read_only_queue? queue_name.include?("read-only") end def writable_queue_name queue_name.gsub(/[-_]?read-only[-_]?/, "") end ``` This pattern makes it easy to transition significant amounts of background jobs to only touch the reader. Second, you can make `GET` requests only talk to the reader. *In theory*, GET requests should not have any side effects or writes, so this should be possible. In addition, Rails makes it [easy to "read your own writes"](https://guides.rubyonrails.org/active_record_multiple_databases.html#activating-automatic-role-switching): > Rails guarantees "read your own write" and will send your GET or HEAD request to the writer if it's within the delay window. By default the delay is set to 2 seconds. You should change this based on your database infrastructure. Rails doesn't guarantee "read a recent write" for other users within the delay window and will send GET and HEAD requests to the replicas unless they wrote recently. I took a brief look around and don't see a lot of major issues with GET side effects. Lots of stuff writes to Redis but obviously that's not a problem. ### Recommendation: Improve the ES failure handling. Cost: 2 Benefit: 4 Currently, the app has a pretty hard failure path when Elastic goes down. If ES is unreachable, searches, connection refusal, or brownouts/timeouts end up in 500s, which will cascade and take down the app as the service degrades. It's my impression that this has happened in the past. I think it could be instructive to look at how [the Rubygems.org codebase tends to hold Elasticsearch much more at a distance](https://github.com/rubygems/rubygems.org), which means the service is a lot more resilient to ES outages. Here's a few things I think you could improve: 1. Add a query timeout in every query body. 2. Add a tighter client timeout - probably something like ~2s. I saw a few timeouts in the Sentry data that showed 30s timeouts in use. If you don't get a response in 2 seconds, you're not gonna get one in 30. 3. Add more layered error handling. When you catch a connection failure, timeout or other Elasticsearch error, return what you can and the rest can 200. The app current treats ES as implicitly available, which I think the past shows is not a great assumption. If ES was treated as a non-critical read-only feature, you could degrade the site into a no-search mode that I think would greatly improve your overall uptime. I would also install timeouts in queries and clients so that brownouts affect you far less - better not to tie up a Ruby process for that long. ### Recommendation: Move audits to a different database or truncate. Cost: 1 Benefit: 2 Like most people who have a "paper trail" feature in the database, I think you're just starting to feel some pain on this one, given the size of this table. It's unclear to me what your obligations are regarding how long these audits need to be kept. Obviously, the easiest thing to do is have a policy to only keep them around for short periods of time and dump the rest. Multi-db has a number of weird risks that just aren't that important for audits: transactional integrity, cascading deletes. The "best-effort" compromises you'll make by doing multi-db here I think are fine for the use-case. If you can't do that, I think audits make good sense to move to a separate database. You only read from them rarely, and the read performance isn't a big deal for them (since only admins can view) so any performance loss on "I can't JOIN natively on these tables anymore" is basically not an issue. Since user model changes and admin activity updates don't happen _that_ often, I kinda doubt there's much write load being added here. It would just be an improvement to the overall database size, making it easier to backup and possibly improving cache hitrates a little. ### Recommendation: Shard a "works" database (workers, chapters, kudos) by work_id, and "activity" database (readings, inbox_comments) by user_id. Cost: 4 Benefit: 5 You asked me in Slack about shards. I do see two pretty clear groupings of models that could shard: * **Work-owned objects**, like workers, chapters, kudos. * **Activity objects, owned by users**, like readings, inbox_comments. You could potentially split `comments` as well into two models, one for comments on Works and the other for comments on tags and admin posts somewhere else. The polymorphism means it wouldn't work to shard in my scheme today. There's still some important relations though that wouldn't survive this scheme: 1. `Reading.visible` joins directly between works and readings. 2. `Work.update_stat_counter` might need a rewrite if you don't also shard comments. It would also mean you'd need to probably do some kind of event/async system to get things to sync across databases. Comments created on a work would have to emit an event or job which then means inbox rows get created for users, etc. This is _kind of_ already how Reading creation works via `SADD`, so maybe this isn't that hard actually? And since you already cache so much, perhaps moving the source of those caches from 1 db to several just doesn't feel that painful in the view layer? Of course it introduces eventual consistency, but I don't think that's actually a huge concern, given what I've seen in the app's usage? It may be that audits are the easiest way to get your feet wet on ActiveRecord multi-db, and then readings and inbox_comments in a new (sharded) db are the second step. ## Outcome: Top controllers, by traffic, should all have a p95 of less than 1 second. I think the second most important thing I'd like to accomplish, beyond just keeping the website up more, would be to make it feel faster for your users. Currently, Sentry reports the overall p95 of all web transactions as 200ms, which is actually quite good. However, today, you have several transactions where the p95 is significantly higher: 1. **BookmarksController#index**, 2.58 seconds. 2. **AutocompleteController#tag**, 2.98 seconds. Most autocomplete controller actions have similar p95s. 3. **DownloadsController#show**, 7.72 seconds These three are what I would like to focus effort on, because the rest feel pretty good (or at least, additional effort here would not cause significant user-noticeable gains in performance). ### Recommendation: Swap Bullet for Prosopite, default to fail in tests. Cost: 2 Benefit: 3 I've never been all that high on Bullet, but recently I've had a lot of success with [prosopite](https://github.com/charkost/prosopite). It's highly accurate, with basically zero false positives. That means you can set up a workflow where Prosopite is configured to `raise` by default in the test environment. That means you can't ship new features with proven N+1s! That's an extremely powerful thing to be always checking for in the background without any additional effort. Usually what I do with clients is work along the following axis: 1. Install prosopite, log in dev and test. 2. Add prosopite as an RSpec context tag, like `:check_for_n_plus_ones`. Add it to all test classes which don't have N+1s in them today (add them everywhere they wouldn't fail a test today). 3. Gradually burn through the remaining bad examples/classes, adding the context one by one, until you have just a few left. At that point, you can... 4. Make prosopite `raise` by default in test and remove the remaining N+1s as global ignores in prosopite's config. It's extremely useful! ### Recommendation: Turn YJIT on. Cost: 1 Benefit: 2 You have YJIT disabled in `application.rb`. I recommend trying it. Pretty consistently, what I see is that it makes that app ~10-15% faster for 20-30% higher memory use. You'll have to look at your current server provisioning to decide if that's worth it for you. If you have any memory left, might as well use it and reduce load on your servers by ~10-15%. If you don't have any memory left, then I guess you're stuck with keeping it off. In my experience, Ruby 3.4+ does quite well with YJIT on. Of course it's also quite stable, since basically everyone has it turned on now. ### Recommendation: Refactor Rails.cache.fetch into view-layer fragment caches. Cost: 2 Benefit: 3 Looking through the traces, my overall picture is that you are over-using cache, peppering sometimes hundreds of cache calls all over the place. I can see how you got here. You're trying to remove load from the SQL database and put it somewhere else. But from the perspective of total latency of the entire transaction, you haven't improved things much (Redis isn't magically faster than a hot, in-the-buffers hit from a MySQL db). If you look at, for example, `_intro_module.html.erb`, there's several cache calls happening right in a row. I think this could be improved by focusing less on `Rails.cache` manual usage, but instead focusing on more cache (and cache-reuse) of entire HTML fragments at the view layer. You're caching _more_ activity (not just the query but also the HTML generation) and also across potentially _multiple_ queries and datasources. Another example: on the homepage, for each marked for later work, there are 5 cache round trips to render the blurb, even though the fragment itself is already cached. If the entire readings section was single fragment keyed per-user, you could turn 30 cache ops into 1. ### Recommendation: Use Turbo Drive or Turbo Frames. Cost: 4 Benefit: 5. I harp on this one on social media a lot but it's true. **The biggest performance impact you can have on a web app user is to turn a full page navigation into an SPA-style route change**. For "golden path" Rails apps like AO3, that means using Turbo. The reason why these kinds of requests are so much faster is because the CSSOM and Javascript VM are re-used. You don't need to completely relayout everything, recalculate the CSSOM and re-execute all your JS. You can just move on! It's truly the only thing that remove **seconds** from a user waiting on the page to do something, rather than milliseconds. For AO3, I think the most realistic transition would be to use Turbo Frames. It asks the least amount from the current setup in order to make the transition. I've done this migration myself on a legacy ActiveAdmin panel with lots of custom JQuery stuff, and I was able to make it happen pretty quickly. A more ambitious transition is Turbo Drive. I find that pretty much every jQuery plugin's assumption about how pageloads work will break, however. It would require an extensive inventory of _all_ your JavaScript and the behaviors you expect to work, and probably adding a lot more tests for that level of integration/browser behavior than what you have today. It's a lot of work, but there's really nothing that makes pageloads faster. If you can turn a full-page-nav into something else, people really feel it. ### Recommendation: Turn on Cloudflare Polish Cost: 1 Benefit: 3 Cloudflare Polish automatically serves `webp` and `avif` formatted images to compatible clients. You have Cloudflare on, but Polish is not. These image formats can save ~10-50% filesize, depending on the image. I love it, it's probably one of the easiest frontend optimizations you can make. Underneath, since it's working based on the Accept header, it literally can't break anything for anyone because the client _has to tell you_ they accept the format before Cloudflare decides to serve it to them! I recommend the `lossy` setting because in practice I haven't seen any visible degradation. ### Recommendation: Change CSRF policy to :header_or_legacy_token Cost: 2 Benefit: 3 You've got this somewhat-unusual `token_dispenser.json` route which you use to generate a CSRF token, which then gets provided to any form that needs it. You need to do this because you want to be able to cache logged-out pages with HTTP, which means of course you can't put a CSRF token in ``. Since Rails 8.2, you now have the option to change how CSRF authentication works. In Rails 8.2, you can configure Rails to check the [`Sec-Fetch-Site` header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Sec-Fetch-Site), which is sent by browsers on every request. From MDN: > this header tells a server whether a request for a resource is coming from the same origin, the same site, a different site, or is a "user initiated" request. The server can then use this information to decide if the request should be allowed. Possible values: cross-site The request initiator and the server hosting the resource have a different site (i.e., a request by "potentially-evil.com" for a resource at "example.com"). same-origin The request initiator and the server hosting the resource have the same origin (same scheme, host and port). same-site The request initiator and the server hosting the resource have the same site, including the scheme. none This request is a user-originated operation. For example: entering a URL into the address bar, opening a bookmark, or dragging-and-dropping a file into the browser window. So, Rails allows updates from `same-origin` or `same-site`, disallows all `cross-site` (unless in Rails' `trusted_origins`) or header missing (except GET of course). Since browsers themselves set Sec-Fetch-Site, it cannot be forged. With the header, the following all becomes unnecessary: * HomeController#token_dispenser * updatedCachedTokens() * loadedCSRF * csrf_meta_tags in layouts, authentication_token in forms Of course, this is not supported in every browser ever. It is supported in: - Chrome 76+ (2019) - Firefox 90+ (2021) - Safari 16.4+ (2023) - Edge 79+ (2020) I don't think AO3 has any official compatibility targets (e.g. [https://browsersl.ist/](https://browsersl.ist/)) (maybe it's time to officially come up with some?). You can test this out by switching to `header_or_legacy_token`, which will allow browsers which present the header to do so. The "feature check" though on the browser side is a bit weird. How do you know if you're sending that header or not? You don't! So, the approach (if you can't drop incompatible clients) would be to probably modify the token dispenser so that after the first request, it notices if the browser sent `Sec-Fetch-Site`, and if it did, it tells the frontend "hey, you don't have to come back for any more CSRF tokens. Just present sec-fetch-site." I think you'd probably want to store that in the session cookie or something. The primary benefit I see here would be then that form actions can happen immediately without waiting for CSRF tokens on a network roundtrip. ### Recommendation: Reduce cache fetches on Bookmarks#index Cost: 3 Benefit: 3 I took a look at a [random Bookmarks#index trace](https://organization-for-transformativ.sentry.io/insights/backend/summary/trace/913a2990d3074f349a9203042bbf62aa/?environment=staging&node=span-bd6b6c3a2376473f&project=4507539185991680&query=transaction.op%3Ahttp.server&referrer=insights-backend-overview&sort=-p95%28span.duration%29&source=performance_transaction_summary&statsPeriod=30d×tamp=1773123577&transaction=BookmarksController%23index) and saw the following: * 179 cache spans * 148 reads * 31 writes That's pretty high. Ideally we only go to the cache a handful of times. These were split between the `MemcacheStore` and `LocalStore`. Since the `MemCacheStore` is the only one that costs us $$$ to use and has network latency, I'll focus on that. - Reads: 114 - Hits: 84 - Misses: 30 - Hit rate: 73.7% That's a pretty poor hitrate. If we look in a bit more detail, here's how that breaks down by key shape: - ao3-v8.0:views/bookmark-blurb-bookmarks/...-v3 - 20 reads - 19 hits - 1 miss - 1 put - ao3-v8.0:.../blurb_css_classes - 20 reads - 20 hits - ao3-v8.0:.../bookmark_count - 40 reads total - **18 remote misses**! - 22 local hits - 18 puts ### Work metadata / fragments - ao3-v8.0:byline_data/... - 17 reads - 17 hits - ao3-v8.0:/v4/work_blurb_tag_cache_key/... - 7 reads - 7 hits - ao3-v8.0:views/works/...-showwarn-showfreeform-v11 - 7 reads - 7 hits - ao3-v8.0:views/works/.../stats-v4 - 7 reads - 2 hits - 5 misses - 5 puts - ao3-v8.0:works/.../count_visible_comments - 5 reads - 5 misses - 5 puts - ao3-v8.0:works/.../kudos_count-v2 - 5 reads - 5 hits - ao3-v8.0:/v1/public_bookmarks_count/... - 5 reads - 5 hits I basically see these as a kind of "fast" N+1 problem. Ideally I think we'd only be going to each keyspace one time. It's a sign of an "insufficiently Russian-dolled-cache". I think it also gets back to the observation I had earlier that you are in a lot of caches replacing single database queries with single cache lookups, which doesn't do much for you. I'd try to roll these up a single `bookmark_row` fragment, which should be able to wrap around ~8 of those keys (kudos, blurb, views, byline data, etc etc). Then you can `MGET` for all the bookmarks you want. It's also possible the cache keys like `byline_data` and `blurb_css_classes` are just too small to help to begin with and should be removed completely. I think this transition is difficult because it requires a deep domain understanding of the cache keys and ERD involved here, but it doesn't look impossible to me. ### Recommendation: Always preload current_user roles. Cost: 1 Benefit: 2 I noticed a lot of Sentry traces with `roles` lookups peppered all over the place. You already do the right thing in `permit_yo`: ```ruby def has_role?(role_name) return self.roles.any? { |role| role.name == role_name.to_s } if self.roles.loaded? role = Role.find_by(name: role_name) self.roles.include?(role) end ``` So that's good. The fact that these queries are happening then is a sign that the association wasn't `loaded`. I've encountered this permissions thing a ton of times. I think it's almost always worth it to just load all user roles upfront rather than try to be clever about which you load when. Since you're using Devise, you override `serialize_from_session`: ```ruby def self.serialize_from_session(key, salt) record = includes(:roles).where(primary_key => key).first record if record && record.authenticatable_salt == salt end ``` There are also a handful of places in views where you inspect `user.roles` directly, e.g. `@user.roles.any?`. I'd replace all these with `has_role?` and then make sure the association is loaded. ### Recommendation: Timeout Autocompletes, add a hard LIMIT, and consider LRU caches around common 3+ char words. Cost: 2 Benefit: 4 `AutocompleteSource.autocomplete_lookup` is a very performance sensitive method that powers all the various autocomplete controllers. It routinely has a p95 of 2sec+. There are a couple of things I'd do to improve it: **Add a hard LIMIT on `zrevrangebyscore` for 3+ character terms.**. Currently the ZREVRANGEBYSCORE can take 2 seconds or more when the search term is something really common. Adding a `limit: [0,50]` here would, I think, really help bring that P95 down a lot, because the score key of `autocomplete_tag_all_score_some_popular_word` could be thousands of tags, which you then just truncate down to 15 in the end. Ouch. **Get a small LRU cache together for popular 3+ character words**. After trying the previous fix, if I still wanted more "juice", I think I'd try to create a separate CacheStore with a given size limit (so that I can use LRU behavior to only cache "the most important" stuff) for 3+ character search parameters. I think this could even be process-local using MemoryStore and a size setting of like `10.megabytes`. **Add a timeout to these queries, particularly the zrevrangebyscore**. One thing I'm always thinking about with Redis is that it's single-threaded. It's really not designed to service long-running queries, because those queries may end up blocking other concurrent operations. Unlike a SQL database, which parallelizes quite well across CPU cores, your Redis DB is just locked to a single core and so a very slow query can end up affecting others. I would also consider a circuit breaker and/or open timeout to the autocomplete Redis. If it goes down, the site should be able to quickly recover and not go down completely. ### Recommendation: Use a test queue to make builds faster, more reliably fast Cost: 2 Benefit: 2 You currently have 13 Cucumber jobs working as explicit directory splits, and 3 RSpec jobs also statically configured. So, if you get unlucky and one of those Cucumber jobs takes 14 minutes, well, that's how long the build takes. There's no balancing happening there. I tell basically everyone these days to **balance tests between workers using a queue**. This allows the maximum amount of redistribution to occur, so that the minimum and maximum total execution time per worker (process on a single-machine approach, or per agent/runner/VM as well) to be as close together as possible. The "enterprisey" way is to pay for Knapsack Pro. However, there are a number of open source projects that can do similar stuff. I'm going to take as a constraint here that we have to use the default, free Github Actions Runner hardware. Those machines have 4 vCPU and 16GB of RAM per. 1. [test-queue](https://github.com/tmm1/test-queue) by the prolific tmm1, author of stackprof. I'm pretty sure the networked mode wouldn't work in Github Actions, but the fork-based model would. That would allow you to use the remaining 3 idle cores on each Github Actions worker. Has cucumber support though maybe it's not very heavily used. 2. [parallel_tests](https://github.com/grosser/parallel_tests) by grosser. Popular, I've seen this one in use a few times. Not networked, but supports Cucumber. 3. [spec-wrk](https://github.com/danielwestendorf/specwrk/blob/main/.github/workflows/specwrk-multi-node.yml) by Daniel Westendorf. No Cucumber support here but _does_ support networked runners in Github Actions. Could be cool if you added a Cucumber adapter! With some networked/parallel runners I think you could easily take builds down to 5 minutes or less. ### Recommendation: Remove resque inline in tests. Cost: 2 Benefit: 3 You have the following in your `resque` config: ```ruby Resque.inline = ENV["RAILS_ENV"] == "test" ``` And in `test.rb`: ```ruby config.active_job.queue_adapter = :inline ``` So in tests, all Resque enqueue calls execute synchronously. In my experience (which is more with the equivalent setting in Sidekiq), this ends up creating two big problems for you: 1. Tests now have a ton of extra state and state modification flying around that's hard to reason about because it's all implicit. This makes the tests harder to understand. 2. You will also be doing a lot of work which is not necessary to make the assertions pass. For both of those reasons, I prefer that any background queue draining be done **explicitly** instead of implictly. If you use the traditional arrange/act/assert framework, I think tests are much clearer if you include these queue drain/execute job calls in the **Act** or **Arrange** phase. It looks like you kind of already started to do this with `suspend_resque_workers` in `spec_helper.rb`. For ActiveJob, switching is easy. You just change the `queue_adapter` to `:test` and then drain only when necessary. Some specs already are written this way. Resque doesn't really have a Sidekiq fake mode built in. I think we basically could extend `suspend_resque_workers` to be the default. ## Outcome: Reduce technical debt These are just a couple of things I noticed as I was looking around. ### Recommendation: Complete the migration to Puma. Cost: 1 Benefit: 1 Of course _I'm_ gonna say this, as I'm the maintainer. But I really do think it's a great application server! It looks like you're about halfway to transitioning. Moving from single-threaded Unicorn to single-threaded Puma should just be a "straight swap". When you use the servers in this way they don't really differ that much in terms of behavior. I think probably this application never becomes multi-threaded. It's probably just more hassle than it's worth. I usually tell people: what changing your concurrency model gives you (from processes to threads, threads to fibers, etc) is allows you to run more open, idle connections with low switching cost and low memory use. If your workload doesn't have that need, process based concurrency probably works just fine for you. Almost every major Rails app, the real giga-shops like Shopify, Github, Intercom, Gusto: they're all running single-threaded. They decided the threading bugs aren't worth the ~30% memory savings. I suspect, given the limited volunteer resources and the age of this app, that the calculus is the same for you. ### Recommendation: load_admin_banner doesn't need caching. Cost: 1 Benefit: 1 This is just one of a couple of places I noticed unnecessary caching. From what I can see in the traces, this takes about 1ms whether or not it's hot or cold. Since it's on every page load via before_action, that stood out to me. ```ruby before_action :load_admin_banner def load_admin_banner if Rails.env.development? @admin_banner = AdminBanner.where(active: true).last else # http://stackoverflow.com/questions/12891790/will-returning-a-nil-value-from-a-block-passed-to-rails-cache-fetch-clear-it # Basically we need to store a nil separately. @admin_banner = Rails.cache.fetch("v1/admin_banner") do banner = AdminBanner.where(active: true).last banner.nil? ? "" : banner end @admin_banner = nil if @admin_banner == "" end end ``` Fetching _the exact same row every time_ in SQL is just not any slower than reading it out of Redis. Just a bit of unnecessary complexity. ### Recommendation: Move JQuery/JQueryUI to first party serving. Cost: 1 Benefit: 1 I noticed you're serving the following: ``` https://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js https://ajax.googleapis.com/ajax/libs/jqueryui/1.10.0/jquery-ui.min.js ``` I think in general, for a site with the kinds of concerns about privacy that AO3 has, you generally just want to "talk" to as few third parties as possible. So just for user privacy reasons, I think it's better to serve these first-party. And they're like 15 year old versions, so who knows, maybe one day Google axes them. It _used_ to be thought in the frontend communities that 3rd party CDNs were a performance "boost" because "oh, everyone will use the same CDN cached version of this library" but that became less true over the years as people realized this could be used to fingerprint clients and so browsers ended up changing how caching worked and breaking this behavior completely. And of course, not a lot of people use 13-year-old versions of jQuery anymore. ### Recommendation: Remove active_record_query_trace and replace with native Rails verbose query logs Cost: 1 Benefit: 1 You currently have three overlapping query source trace mechanisms. 1. `active_record_query_trace` gem. Appends a colorized multi-line call stack after each query in dev logs. 2. `config.active_record.verbose_query_logs` = true. Rails 5.2+ built-in, appends ↳ file:line after each query 3. `config/initializers/active_record_log_subscriber.rb` — a custom LogQuerySource module that also prepends ↳ file:line While I love this feature and of course recommend everyone use it, it is kinda baked into Rails now and I prefer to use the Rails mainline stuff where I can rather than bring in dependencies. Probably we only need one of these right? It is a little bit of a bummer but verbose_query_logs only displays a single line: ``` Before (with gem): User Load (0.4ms) SELECT "users".* FROM "users" WHERE ... app/models/concerns/is_active.rb:11:in `active?' app/models/user.rb:67:in `active?' app/controllers/users_controller.rb:42:in `index' After (without gem): User Load (0.4ms) SELECT "users".* FROM "users" WHERE ... ↳ app/controllers/users_controller.rb:42 ``` I think you can just keep verbose_query_logs and delete the other two. ### Recommendation: Move to SLO queues Cost: 3 Benefit: 5 SLO queues are a concept introduced in my book, Sidekiq in Practice. It was adopted successfully at Gusto and the community at large has really picked up and run with the idea. The basic idea is that all queues in your system are named after their queue time SLO - how long they promise that a job will be in the queue until it is a executed. For example, a common setup is: ``` within_0_seconds within_5_minutes within_15_minutes within_1_hour ``` You then launch any number of processes in Resque which listen to one of those queues (and one queue only). This allows you to intelligently autoscale: we don't need to start scaling up the within 1 hour queue until the latency of that queue is above ~45 minutes or so. Using many queues which are based on just the "domain" or "subject" of the queue (e.g. imports, exports, orders, etc) leads to an explosion of queue types, whose SLOs are not clear. You don't know when to page someone or wake them up because the queue time expectation is not clear. For a volunteer-run shop, I think getting completely crystal-clear about how long it's OK to wait in the queue until someone needs to be notified would be really helpful. It also makes it easy for app developers in the future to "sign up" for what level of queue time they need (it's right in the name!). --- ## Announcing the Rails Performance Apocrypha URL: https://www.speedshop.co/blog/announcing-apocrypha/ {% marginnote_lazy apocrypha_cover.jpg||true %} Hello Rubyists! Today, I'm launching a new product: The Ruby on Rails Performance Apocrypha. Over the last four years, I've written a lot of stuff to this newsletter. Until now, none of that stuff has been publicly accessible if you wanted to go back and read it again. If something useful was posted to the newsletter before you subscribed, you were just sort of screwed. So, I've compiled 4 years of writing to this newsletter into a book. It covers my usual topics: performance science and engineering, frontend performance, Ruby performance, and scaling. It's a fun ramble around all of these topics with a lot of tidbits and useful information scattered about. Each chapter is quite short, so it's easy to pick up and put down again. [It's available now on Gumroad for just $10.](https://gum.co/apocrypha) As always, it's DRM-free and available in PDF, e-reader and even HTML and plain-text formats. I called this book the “apocrypha”, because I consider my “main-line” of Rails performance instruction, the canonical “scripture”, to be my Rails Performance Workshop. By contrast, this book is a bit of an all-over-the-place ramble, and it covers some things that I didn't cover in great detail in my other instructional books and workshops, such as HTTP/2 resource prioritization, and a detailed how-to on how to use New Relic's Ruby VM information. Here's the chapter titles, in case you're wondering what's covered: * What I Value * Performance Science * Why Performance? * You Are Not a Compiler * What does 10% faster really mean? * Benchmarks for Rails Apps * Build-your-own APM * Reading Flamegraphs * DRM: Database, Ruby, Memory * Performance in the Design Space * Microservices and Trends * On Minitest * Corporate Support for Ruby * Why is Ruby Slow? * Popularity * Stinky Dependencies * Why Cache? * Software Quality at Startups * Frontend * Simple Frontend Config Changes * What is TTFB? * Always Use a CDN * Page Weights and Frontend Load Times * Lazy-loading * What’s Resource Prioritization? * HTML on the Wire * Exceptions: Silent, not Free * On Thread-Safety * What is the GVL? * Timeslicing the GVL * The GVL and C * Bloat * Minimum Viable Rails * The Weird Setting No One Used * Object Allocation * You Should Always Use a Production Profiler * Reproducing Issues Locally * Worker Killers * What’s QueryCache? * Reading New Relic’s Ruby VM Tab * Test Setup * What is Time Consumed? * Request Queue Times * Amdahl’s Law * Threads * CPU-bound or IO-bound? * What is Swap? * Database Pools * Single-thread Performance * Read Replicas * Why Lambda? * Never use Perf-M * Daily Restarts [Check it out on Gumroad.](https://gum.co/apocrypha) --- ## We Made Puma Faster With Sleep Sort URL: https://www.speedshop.co/blog/we-made-puma-faster-with-sleep-sort/ Puma 5 (codename Spoony Bard{% sidenote 1 "When Puma gets a new 'supercontributor' that submits lots of important work to the project, we let them name the next release. This release features a lot of code from Will Jordan, who named this release 'Spoony Bard'. Will said: 'Final Fantasy IV is especially nostalgic for me, the first big open-source project I ever worked on was a fan re-translation of the game back in the late 90s.'" %}) was released today (my birthday!). There's a lot going on in this release, so I wanted to talk about the different features and changes to give Puma users confidence in upgrading. ## Experimental Performance Features For Cluster Mode on MRI This is probably the headline of the release - two features for reducing memory usage, and one for reducing latency. Puma 5 contains 3 new experimental performance features: * `wait_for_less_busy_worker` config. This may reduce latency on MRI through inserting a small delay (sleep sort!) before re-listening on the socket if worker is busy. Intended result: If enabled, should reduce latency in high-load (>50% utilization) Puma clusters. * `fork_worker` option and `refork` command for reduced memory usage by forking from a worker process instead of the master process. Intended result: If enabled, should reduce memory usage. * Added `nakayoshi_fork` config option. Reduce memory usage in preloaded cluster-mode apps by GCing before fork and compacting, where available. Intended result: If enabled, should reduce memory usage. All of these experiments are only for **cluster mode** Puma configs running on **MRI**. We're calling them _experimental_ because we're not sure if they'll actually have any benefit. We're pretty sure they're stable and won't break anything, but we're not sure they're actually going to have big benefits in the real world. People's workloads are often not what we anticipate, and synthetic benchmarks are usually not of any help in figuring out if a change will be beneficial or not. We do not believe any of the new features will have a negative effect or impact the stability of your application. This is either a "it works" or "it does nothing" experiment. If any of the features turn out to be particularly beneficial, we may make them defaults in future versions of Puma. **If you upgrade and try any of the 3 new features, please post before and after results or screenshots to [this Github issue](https://github.com/puma/puma/issues/2258).** "It didn't do anything" is still a useful report in this case. Posting ~24 hours of "before" and ~24 hours of "after" data would be most helpful. ### wait_for_less_busy_worker: sleep sort for faster apps?! This feature was contributed to Puma by Gitlab. Turn it on by adding `wait_for_less_busy_worker` to your Puma config. When a request comes in to a Puma cluster, the operating system randomly selects a listening, free Puma worker process to pick up the request. "Listening" and "free" being the key words - a Puma process will only listen to the socket (and pick up more requests) if it has nothing else to do. However, when running Puma with multiple threads, Puma will also listen on the socket when all of its busy threads are waiting on I/O or have otherwise released [the Global VM Lock](/blog/the-ruby-gvl-and-scaling/). When Gitlab investigated switching from Unicorn to Puma, they encountered an issue with this behavior. Under high load with moderate thread settings (a max pool size of 5 in their case), average request latency increased. Why? Remember, I said that the operating system _randomly_ assigns a request to a _listening_ worker process. So, it will never send a request to a worker process that's busy doing other things, but what about a worker process that's got 4 threads that are processing other requests, but all 4 of those threads happen to be waiting on I/O right now? Imagine a Puma cluster with 3 workers: * Worker 1: 0/5 threads busy. * Worker 2: 1/5 threads busy. * Worker 3: 4/5 threads busy. If Worker 3's 4 active threads happen to all have released the GVL, allowing that worker to listen to the socket, and a new request comes in - which worker process should we assign the request to, ideally? Worker 1, right? Unfortunately, most operating systems will assign the request to Worker 3 33% of the time. So, what do we do? We want the operating system to prefer less-loaded workers. It would be really cool if we could sort the list of workers listening on the socket so that the operating system would give requests to the least-loaded worker. Well, we can't really do that easily, but we can do something else. `wait_for_less_busy_worker` causes a worker to _wait_ to re-listen on the socket if it's thread pool isn't completely empty. This means that in high-load scenarios, the operating system will assign requests to less-loaded workers. **This is basically sleep-sorting our workers**. We're kind of doing doing this: ``` [].tap { |a| workers.map { |e| Thread.new{ sleep worker_busyness.to_f/1000; a << e} }.each{|t| t.join} } ``` ... and hiding "more loaded" workers from the operating system by letting less-loaded workers listen first! Originally the proposal was for a more complicated sort - processes slept longer if they had more busy threads - but that was removed when it was found that a simpler on/off sleep was just as effective. The net effect is that in high-load scenarios, request latency decreases. This is because workers with more busy threads are slower than workers with no busy threads. We're assuring that requests get assigned to the faster workers. Prior to this patch, Gitlab saw an increase in latency using Puma compared to Unicorn - after this patch, latency was the same (they also were able to reduce their fleet size by almost 30% thanks to Puma's memory-saving multithreaded design). There may be even more efficient ways for us to implement this behavior in the future. There's some magic you can do with `libev`, I'm pretty sure, or we can just implement a different sleep/wait strategy. ### fork_worker Adding `fork_worker` to your puma.rb config file (or `--fork-worker` from the CLI) turns on this feature. This mode causes Puma to fork additional workers from worker 0, instead of directly from the master process: ``` 10000 \_ puma 5.0.0 (tcp://0.0.0.0:9292) [puma] 10001 \_ puma: cluster worker 0: 10000 [puma] 10002 \_ puma: cluster worker 1: 10000 [puma] 10003 \_ puma: cluster worker 2: 10000 [puma] 10004 \_ puma: cluster worker 3: 10000 [puma] ``` Similar to the `preload_app!` option, the `fork_worker` option allows your application to be initialized only once for copy-on-write memory savings, and it has two additional advantages: 1. **Compatible with phased restart.** Because the master process itself doesn't preload the application, this mode works with phased restart (`SIGUSR1` or `pumactl phased-restart`), unlike `preload_app!`. When worker 0 reloads as part of a phased restart, it initializes a new copy of your application first, then the other workers reload by forking from this new worker already containing the new preloaded application. This allows a phased restart to complete as quickly as a hot restart (`SIGUSR2` or `pumactl restart`), while still minimizing downtime by staggering the restart across cluster workers. 2. **'Refork' for additional copy-on-write improvements in running applications.** Fork-worker mode introduces a new `refork` command that re-loads all nonzero workers by re-forking them from worker 0. This command can potentially improve memory utilization in large or complex applications that don't fully pre-initialize on startup, because the re-forked workers can share copy-on-write memory with a worker that has been running for a while and serving requests. You can trigger a refork by sending the cluster the `SIGURG` signal or running the `pumactl refork` command at any time. A refork will also automatically trigger once, after a certain number of requests have been processed by worker 0 (default 1000). To configure the number of requests before the auto-refork, pass a positive integer argument to `fork_worker` (e.g., `fork_worker 1000`), or `0` to disable. ### nakayoshi_fork Add `nakayoshi_fork` to your puma.rb config to try this option. Nakayoshi means "friendly", so this is a "friendly fork". The concept was [originally implemented by MRI supercontributor Koichi Sasada](https://github.com/ko1/nakayoshi_fork) in a gem, but we wanted to see if we could bring a simpler version into Puma. Basically, we just do the following before forking a worker: ```ruby 4.times { GC.start } GC.compact # if available ``` The concept here is that we're trying to get as clean of a Ruby heap as possible before forking to maximize [copy-on-write](https://en.wikipedia.org/wiki/Copy-on-write) benefits. That should, in turn, lead to reduced memory usage. ## Other New Features A few more things in the grab-bag: * You can now compile Puma on machines where OpenSSL is not installed. * There is now a `thread-backtraces` command in pumactl to print all active threads backtraces. This has been available via SIGINFO on Darwin, but now it works on Linux via this new command. * `Puma.stats` now has a `requests_count` counter. * `lowlevel_error_handler` got some enhancements - we also pass the status code to it now. * Phased restarts and worker timeouts should be faster. * `Puma.stats_hash` provides Puma statistics as a hash, rather than as JSON. ## Loads of Bugfixes The number of bugfixes in this release is pretty huge. Here's the most important ones: * Shutdowns should be more reliable. * Issues surrounding socket closing on shutdown have been resolved. * Fixed some concurrency bugs in the Reactor. * `out_of_band` should be much more reliable now. * Fixed an issue users were seeing with ActionCable and not being able to start a server. * Many stability improvements to `prune_bundler`. ## Nicer Internals and Tests This release has seen a massive improvement to our test coverage. We've pretty much doubled the size of the test suite since 4.0, and it's way more stable and reproducible now too. A number of breaking changes come with this major release. [For the complete list, see the HISTORY file.](https://github.com/puma/puma/blob/master/History.md) ## Thanks to Our Contributors! This release is our first major or minor release with new maintainer MSP-Greg on the team. Greg has been doing tons of work on the test suite to make it more reliable, as well as a lot of work on our SSL features to bring them up-to-date and more extendable. Greg is also our main Windows expert. The following people contributed more than 10 commits to this release: * [Tim Morgan](https://github.com/seven1m) * [Vyacheslav Alexeev](https://github.com/alexeevit) * [Will Jordan](https://github.com/wjordan) * [Jeff Levin](https://github.com/jalevin) * [Patrik Ragnarsson](https://github.com/dentarg), who's also been very helpful in our Issues tracker. If you've like to make a contribution to Puma, please see our [Contributors Guide](https://github.com/puma/puma/blob/master/CONTRIBUTING.md). We're always looking for more help and try to make it as easy as possible to contribute. Enjoy Puma 5! --- ## The Practical Effects of the GVL on Scaling in Ruby URL: https://www.speedshop.co/blog/the-ruby-gvl-and-scaling/ The Global Virtual Machine Lock confuses many Rubyists. Most Rubyists I've met have a vague sense that the GVL is somehow bad, and has something to do concurrency or parallelism. {% sidenote "'CRuby' refers to the mainline Ruby implementation, written in C. Sometimes people call this 'MRI'." %} The GVL (formerly known as GIL, as you're about to learn) is a unique feature to CRuby, and doesn't exist in JRuby or TruffleRuby. JavaScript's popular V8 virtual machine also has a VM lock. CPython also has a _global_ VM lock. That's three of the most popular dynamic languages in the world! VM locks in dynamic languages are very common. {% sidenote "Instead of removing the GVL, Ruby core has signaled that it will take an approach similar to V8 Isolates with inspiration from the Actor concurrency model (discussed at the end)." %} Understanding CRuby's Global VM Lock is important when thinking about scaling Ruby applications. It will probably never be removed from CRuby completely, and its behavior changes how we scale Ruby apps efficiently. Understanding what the GVL is and why the current GVL is "global" will help you to answer questions like these: * What should I set my Sidekiq concurrency to? * How many threads should I use with Puma? * Should I switch to Puma or Sidekiq from Unicorn, Resque, or DelayedJob? * What are the advantages of event-driven concurrency models, like Node? * What are the advantages of a global-lock-less language VM, like Erlang's BEAM or Java's JVM? * How will Ruby's concurrency story change in Ruby 3? We'll deal with these questions and more in this article. ## What we're locking: the language virtual machine {% sidenote "Most descriptions of the GVL immediately dive into concepts like atomicity and thread-safety. This description will start from a more basic premise and work up to that." %} {% sidenote "YARV was essentially [Koichi Sasada's graduate thesis.](https://en.wikipedia.org/wiki/YARV)" %} Wait: isn't it the GIL? What's the GVL? GIL stands for Global Interpreter Lock, and it's something that was removed from Ruby (or just mutated, depending on how you look at it) in Ruby 1.9, when Koichi Sasada introduced YARV (Yet Another Ruby VM) to Ruby. YARV changed CRuby's internal structure so that the lock existed around the Ruby virtual machine, not an interpreter. The correct terminology for over a decade now has been GVL, not GIL. {% sidenote "You can interact with instruction sequences [via the InstructionSequence class](https://ruby-doc.org/core-2.5.1/RubyVM/InstructionSequence.html). Everything is an object in Ruby!" %} How does an interpreter differ from a virtual machine? A virtual machine is a little like a CPU-within-a-CPU. Virtual machines are computer programs that usually take simple instructions, and those instructions manipulate some internal state. A [Turing machine](https://en.wikipedia.org/wiki/Turing_machine), if it was implemented in software, would be a kind of virtual machine. We call them virtual machines and not machines because they're implemented in software, rather than in hardware, like a CPU is. Before Ruby 1.9, Ruby didn't really have a separate virtual machine step - it just had an interpreter. As your Ruby program ran, it actually interpreted each line of Ruby as it went. Now, we just interpret the code once, turn it into a series of VM instructions, and then execute those instructions. This is much faster than interpreting Ruby constantly. {% marginnote_lazy turingmachine.gif|A Turing machine, implemented in software, would be a kind of virtual machine. [Wikimedia Commons by RosarioVanTuple](https://commons.wikimedia.org/wiki/File:TuringBeispielDiskretAnimatedGIF_uk.gif)|true %} The Ruby Virtual Machine understands a simple instruction set. Those instructions are generated from the Ruby code you write by the interpreter, and then the virtual machine instructions are fed into the Ruby VM. Let's watch this in action. First, in case you didn't know, you can execute Ruby from the command line using the -e option: ``` $ ruby -e "puts 1 + 1" 2 ``` {% marginnote_lazy escanor_stack_meme_opt.jpeg||true %} Now, you can then dump the instructions for this simple program by calling `--dump=insns`: ``` $ ruby --dump=insns -e "puts 1 + 1" == disasm: #@-e:1 (1,0)-(1,10)> (catch: FALSE) 0000 putself ( 1)[Li] 0001 putobject_INT2FIX_1_ 0002 putobject_INT2FIX_1_ 0003 opt_plus , 0006 opt_send_without_block , 0009 leave ``` Ruby is a "stack-based" VM. You can see how this works by looking at the generated instructions here - we add the integer 1 to the stack two times, than call `plus`. When `plus` is called, there are two integers on the stack. Those two integers are replaced by the result, 2, which is then on the stack. So, what does the Ruby VM have to do with threading, concurrency and parallelism? ## Concurrency and Paralellism {% marginnote_lazy checkout_counter.jpg||true %} You may be aware that there's a difference between concurrency and parallelism. Imagine a grocery store. At this grocery store, we have a line and some checkout clerks working to pull customers from the line and get their groceries checked out. Each of our grocery store checkout clerks works in parallel. They don't need to talk to each other to do their job, and what one clerk is doing doesn't affect the other in any way. They're working 100% in parallel. {% marginnote_lazy concurrent_checkout.jpg||true %} Now, a clerk _can_ work on multiple customers _concurrently_. This would look like a clerk grabbing multiple customers from the line, working on one customer's groceries for a moment, then switching to another customer's groceries, and so on. This would be working concurrently. Let's take a more concrete example. Compare three grocery store clerks working in parallel with a single one working concurrently. To check out a customer, we must perform two operations: scanning their groceries, and then bagging them. Imagine each customer's groceries take the exact same amount of time to scan and bag. Three customers arrive. Let's say scanning takes time `A` and bagging takes time `B`. Our three parallel clerks will process these three customers in time `(A + B)`. {% marginnote_lazy parallel.jpg|The parallel case.|true %} What about our concurrent clerk? All three of her customers arrive at the same time. The clerk scans each customer's groceries, then bags each customer's groceries. Each customer is worked on concurrently, but never in parallel. {% marginnote_lazy concurrency.jpg|The concurrent case.|true %} In the concurrent case, our first customer experiences a total service time of `(3A + B)`. They had to wait for everyone else's groceries to be checked out for their own groceries to get bagged. The second customer will exerience a total service time of `(3A + 2B)`, and the final customer will experience a service time of `(3A + 3B)`. Notice how the customers who got the concurrent checkout clerk experienced a longer total service time than the customers who used our three parallel clerks. In short: **concurrency is interesting, but parallelism is what speeds up systems and allows them to handle increased load**. Performing two operations concurrently means that the start and end times of those operations overlapped at some point. For example, you and I sit down to a sign a contract. However, there is only one pen. I sign where I'm supposed to, hand the pen to you, and then you sign. Then, you hand the pen back to me and I initial a few lines. You might say that we signed the contract concurrently, but never in parallel - there was only one pen, so we couldn't sign the contract at the exact same time. Peforming operations in parallel means that we are doing those operations *at the exact same instant*. In my contract example, a parallel contract signing would involve two pens (and probably two copies of the contract, otherwise it would get a little crowded). ## Concurrency and Paralellism on a Computer On a modern operating system, programs are run with a combination of processes and threads. Processes have at least one thread, and can have up to thousands. To extend the grocery store analogy, processes are like the checkout counters that our clerks use. They contain tools and common resources, like the point-of-sale terminal and the barcode scanner, but they don't actually _do_ anything. A process usually contains a memory allocation (the heap), file descriptors (sockets, files, etc), and other such computer resources. Threads actually run our code. Each process has at least one thread. In our analogy, they're like the store clerks. They also hold a small amount of information. For example, if we're adding two local variables in a Rails application, our thread contains information about those two variables (_thread-local storage_) and also what line of code we're currently running (the _stack_). {% marginnote_lazy pentium.jpg||true %} Threads run the code when they are scheduled to by the operating system's kernel. The Ruby runtime itself doesn't actually manage when threads are executed - the operating system decides that. When Ruby was written in the 90s, all processes had just one thread. This started to change in the early 2000s, necessitating the rewrite of the language VM in Ruby 1.9 (YARV), which is what gave us the GVL as we know it today. ## What the GVL actually does As mentioned earlier, the Ruby Virtual Machine is what actually turns Ruby virtual machine instructions (generated by the interpreter) into CPU instructions. {% marginnote_lazy vm_lock_bernie.jpg||true %} The Ruby Virtual Machine is not internally thread-safe. If two threads try to access the Ruby VM at the same time, really Bad Things would happen. This is a bit like the point of sale terminal at our grocery store checkout counters. If two checkout clerks tried to use the same POS terminal, they would interrupt each other and probably keep losing their work or corrupting each other's work. You would end up paying for someone else's groceries! So, because it isn't safe for multiple threads to access the Ruby Virtual Machine at the same moment, instead we use a global lock around it so that only one thread can access it in parallel. {% sidenote "One caveat of the Javascript GVL is that it isn't actually global: you can create additional Isolates. Koichi Sasada's proposal for Ractors (formerly Guilds) would be similar." %} It is extremely common for dynamic language VMs to not be thread-safe. As mentioned, CPython and V8 are the most prominent examples. Java is probably the best example of a semi-dynamic language that _does_ have a threadsafe VM. It's also why so many languages are written on top of the JVM: writing your own threadsafe VM is really hard. {% marginnote_lazy realize.gif|TFW you realize that there's always going to be locks, the only difference is what level they're implemented at and who implements them|true %} There's a few good reasons that having a GVL is so popular: * It's faster. Single-threaded performance improves because you don't have to constantly lock and unlock internals. * Integrating with extensions, such as C extensions, is easier. * It's easier to write a lockless VM than one with a lot of locks. Each Ruby process has its own Global VM Lock, so it might be more accurate to say that it's a "process-wide VM lock". Its "global" in the same sense that a "global variable" is global. Only one thread in any Ruby process can hold the global VM lock at any given time. Since a thread needs access to the Ruby Virtual Machine to actually run any Ruby code, effectively only one thread can run Ruby code at any given time. {% marginnote_lazy songofmyppl.jpg|Let me play you the song of my people: "_GGVVVVLLLLLLLLLLLLLLLL_"|true %} Think of the GVL like the conch shell in the Lord of the Flies - if you have it, you get to speak (or execute Ruby code in this case). If the GVL is already locked by a different thread, other threads must wait for the GVL to be released before they can hold the GVL. ## Amdahl's Law: Why 1 Sidekiq Process Can Be 2x as Efficient as DelayedJob or Resque Your programs actually do many things that don't need to access the Ruby Virtual Machine. The most important is waiting on I/O, such as database and network calls. These actions are executed in C, and the GVL is explicitly released by the thread waiting on that I/O to return. When the I/O returns, the thread attempts to reacquire the GVL and continue to do whatever the program says. This has enormous real-world performance impacts. Imagine you have a stack of satellite image data you have to process (with Ruby). You have written a Sidekiq job, called `SatelliteDataProcessorJob`, and each job works on a small fraction of all of the satellite data. ```ruby class SatelliteDataProcessorJob include Sidekiq::Worker def perform(some_satellite_data) process(some_satellite_data) touch_external_service(some_satellite_data) add_data_to_database(some_satellite_data) end end ``` Let's imagine that `process` is a 100% Ruby method, which does not call C extensions or external services. Further, let's imagine that `touch_external_service` and `add_data_to_database` are effectively 100% I/O methods that spend all of their time waiting on the network. First, an easy question: if each run of `SatelliteDataProcessorJob` takes 1 second, and you have 100 enqueued jobs and just 1 Sidekiq process with 1 thread, how long will it take to process all the jobs? Assume infinite CPU and memory resources. 100 seconds. How about if you two processes? 50 seconds. And 25 seconds for 4 processes and so on. That's parallelism. Now, let's say you have 1 Sidekiq process with 10 threads. How long will it take to process all of those jobs? The answer is _it depends_. If you're on JRuby or TruffleRuby, it will take about 10 seconds, because each thread is fully parallel with all the other threads. But on MRI, we have the GVL. Does adding threads increase concurrency? {% marginnote_lazy AmdahlsLaw.svg|From [Daniels 220 @ Wikipedia](https://commons.wikimedia.org/wiki/File:AmdahlsLaw.svg)|true %} It turns out, this exact problem interested computer scientist Gene Amdahl back in 1967. He proposed something called Amdahl's Law, which gives the theoretical speedup in latency for the execution of tasks with fixed workloads when resources increase. Amdahl figured out that the speedup you got from additional parallelism was related to the proportion of execution time that could be done in parallel. Sound familiar? Amdahl's Law is simply `1 / (1 - p + p/s)`, where `p` is the percentage of the task that could be done in parallel, and `s` is the speedup factor from the part of the task that gained improved resources (the parallel part). So, in our example, let's say that half of `SatelliteDataProcessorJob` is GVL-bound and half is IO-bound. In this case, `p` is `0.5` and `s` is 10, because we can wait for IO in parallel and there are 10 threads. **In this case, Amdahl's Law shows that a Sidekiq process would go through our jobs up to 1.81x faster than a single-threaded Resque or DelayedJob process.** Many background jobs in Ruby spend at least 50% of their time waiting on IO. For those jobs, Sidekiq can lead to a 2x decrease in resource usage, because 1 Sidekiq process can do the work of what used to take 2 single-threaded processes. So, even with a GVL, adding threads to applications increases throughput-per-process, which in turn lowers memory consumption. ## Threads, Puma and GVL-caused Latency This also means that "how many threads does my Sidekiq or Puma process need" is a question answered by "how much time does that thread spend in non-GVL execution?" or "how much time does my program spend waiting on I/O?" Workloads with high percentages of time spent in I/O (75%+ or more) often benefit from 16 threads or even more, but more typical workloads see benefit from just 3 to 5 threads. {% marginnote_lazy paralellizable.jpg||true %} It's possible to configure your thread pools to be _too large_. Setting Puma or Sidekiq to thread settings higher than 5 can lead to contention for the GVL if the work is not parallelizable enough. This increases service latency. While total time to process all of the units of work remains the same, the latency experienced by each individual unit of work increases. Imagine a grocery store where a checkout clerk grabbed 16 people off of the checkout queue and checked those 16 people's groceries concurrently, scanning one item per person before scanning one item from the next person's cart. Rather than experiencing checkout time as `(A + B)`, they experience a checkout time of `16(A+B)`. {% sidenote "This effect is generally present in a concurrent-but-not-100%-parallel system where overall utilization is not extremely high. [We're mitigating this effect slightly in Puma 5](https://github.com/puma/puma/pull/2079) by having Puma workers with more than one thread delay listening to the socket, so less-loaded workers pick up requests first." %} Some people misidentify this additional latency as "context switching" costs. However, latency experienced by the individual unit of work is increasing _without additional switching cost_. In any case, context switching on modern machines and operating systems is pretty cheap relative to the time it takes to service a typical web app request or background job. It does not add hundreds of milliseconds to response times - but oversaturating the GVL can. If adding threads to a CRuby process can increase latency, why is it still useful? {% sidenote "Shouldn't adding an additonal thread only increase memory usage by 8MB, which is the size of the thread's stack allocation? Ah, if only memory usage was that simple. [Learn more about the complexities of RSS and thread-induced fragmentation here.](/blog/malloc-doubles-ruby-memory/)" %} **Adding more threads to a Ruby process helps us to improve CPU utilization at less memory cost than an entire additional process.** Adding 1 process might use 512MB of memory, but adding 1 thread will probably cause less than 64MB of additional memory usage. With 2 threads instead of 1, when the first thread releases the GVL and listens on I/O, our 2nd thread can either pick up new work to do, increasing throughput and utilization of our server. GitLab switched from Unicorn (single-thread model) to Puma (multi-thread model) and [saw a 30% decrease in memory usage across their fleet.](https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/7455#note_239070865) If you're memory-constrained on your host, this allows you to run 30% more throughput for the same money. That's awesome. ## The Future For a decade now, bystanders have declared that Ruby is dead because it "doesn't have a proper concurrency story". I think we've shown that there is a concurrency story in Ruby. First, we have process-based concurrency. We multiply GVLs by multiplying processes. This works perfectly fine, if you have enough memory. If you're out of memory, you can use Sidekiq or Puma, which provides a threaded container for our apps, and then let pre-emptive threading do its thing. Ruby has proven that process-based concurrency (which is really what the GVL forces us to do) scales well. It's not much more expensive than other models, especially these days when memory is so cheap on cloud providers. Think critically about what an Actor-style approach or an Erlang Process-style approach would _actually change_ about your deployment at the end of the day: you would use less memory per CPU. But on large deployments, most web applications are already CPU-bottlenecked, not memory! #### Ractor Koichi Sasada, author of YARV, is proposing a new concurrency abstraction for Ruby 3 called Ractors. It's a proposal based on the Actor concurrency model (hence Ruby Actor -> Ractor). Basically, Actors are boxes for objects to go into, and each actor can only touch its own objects, but can send and receive objects to/from other Actors. Here's an example written by Koichi Sasada: ```ruby r = Ractor.current rs = (1..10).map{|i| r = Ractor.new r, i do |r, i| r.send Ractor.recv + "r#{i}" end } r.send "r0" p Ractor.recv #=> "r0r10r9r8r7r6r5r4r3r2r1" ``` Eventually (not yet in the current implementation), each Ractor will get their own VM lock. That means the example code above will execute in parallel. This is made possible because Ractors don't share mutable state. Instead, they only share immutable objects, and can send mutable objects between each other. This should mean that we don't need a VM lock inside of a Ractor. [Koichi Sasada's Ractor proposal is now public](https://github.com/ruby/ruby/compare/master...ko1:ractor), though as of this writing the docs are mostly in Japanese, and "each Ractor gets its own VM lock" has not yet been implemented. Ractors will essentially allow us to "multiply" GVLs in a process, which would make the GVL no longer "global", although the lock will still exist in each Ractor. The Global VM Lock will become a Ractor VM Lock. ## TL:DR; Thanks for listening to me whinge. Here's what you need to remember: * If you are memory bottlenecked on Ruby, you need to **saturate the GVL** by adding more threads, which will allow you to get _more CPU work done with less memory use_. * The GVL means that parallelism is limited to I/O in Ruby, so **switch to a multithreaded background job processor before you switch to a multithreaded web server**. Also, you'll probably use much higher threadpool sizes with your background jobs than with your web server. * Ruby 3 **might make the GVL no longer global** by allowing you to multiply VMs using Ractors. Application servers and background job processors will probably change their backend to take advantage of this, you won't really have to change much of your code at all, but you will no longer have to worry about thread safety (yay). * Process based concurrency scales very well, and while it might lose a few microseconds to other concurrency models, these **concurrency switching costs generally don't matter for the typical Rails application**. Instead, the important thing is saturating CPU, which is the most scarce resource in today's computing environments. --- ## The World Follows Power Laws: Why Premature Optimization is Bad URL: https://www.speedshop.co/blog/why-premature-optimization-is-bad/ {% sidenote "This post is a sample of the content available in the [Complete Guide to Rails Performance](https://railsspeed.com). It's actually the first lesson - there are 30+ more lessons and 18 hours of video in the course itself." %} I want to tell you about a physicist from Schenectady, a Harvard linguist, and an Italian economist. {% marginnote_lazy pareto.jpg|Pareto's poltiical views are bit suspect, unfortunately, because he chose to see the way things *are* (unequal) as the way they *ought to be*.|true %} The Italian economist you may already have heard of - Vilifredo Pareto. He became famous for something called **The Pareto Principle**, the idea that for most things, 80% of the effect comes from just 20% of the causes. The Pareto Principle is fundamental to performance work because it reminds us why premature optimization is so inefficient and useless. While you've probably *heard* of the Pareto Principle, I want you to *understand why* it actually works. And to do that, we're going to have to talk about probability distributions. ## Benford - the physicist Frank Benford was an American electrical engineer and physicist who worked for General Electric. It was the early 20th century, when you had a job for life rather than a startup gig for 18 months, so he worked there from the day he graduated from the University of Michigan until his death 38 years later in 1948. {% marginnote_lazy logtables.jpg|A page from Henry Briggs' first table of common logarithms, Logarithmorum Chilias Prima, from 1617. [Wikipedia](https://commons.wikimedia.org/wiki/File:Logarithmorum_Chilias_Prima_page_0-67.jpg)|true %} Back in that time, before calculators, if you wanted to know the logarithm of a number - say, 12 - you looked it up in a book. The books were usually organized by the leading digit. If you wanted to know the logarithm of 330, you first went to the section for 3, then looked for 330. Benford noticed that the first pages of the book were far more worn out than the last pages. Benford realized this meant that the numbers looked up in the table began more often with 1 than with 9. Most people would have noticed that and thought nothing of it. But Benford pooled 20,000 numbers from widely divergent sources (he used the numbers in newspaper stories) and found that the leading digit of all those numbers followed a power law too. This became known as [Benford's Law](https://en.wikipedia.org/wiki/Benford%27s_law). Here are some other sets of numbers that conform to this power law: {% marginnote_lazy physicalconstants.png||false %} * Physical constants of the universe (pi, the molar constant, etc.) * Surface areas of rivers * Fibonacci numbers * Powers of 2 * Death rates * Population censuses That the _physical constants of the universe_ follow this distribution is probably the most mind-blowing revelation of Benford's Law, for me, anyway. Benford's Law is so airtight that it's been admitted in US courts as evidence of accounting fraud (someone used RAND in their Excel sheet!). It's been used to identify other types of fraud too - elections, scientific and even macroeconomic data. What would cause numbers that have (seemingly) little relationship with each other to conform so perfectly to this non-random distribution? ## Zipf - the linguist {% marginnote_lazy zipf_wiki.png|A plot of the rank versus frequency for the first 10 million words in 30 different languages of Wikipedia. Note the logarithmic scales. [Licensed CC-BY-SA by SergioJimenez.](https://commons.wikimedia.org/wiki/File:Zipf_30wiki_en_labels.png) |false %} At almost exactly the same time as Benford was looking at first leading digits, George Kingsley Zipf was studying languages at Harvard. Uniquely, George was applying the techniques of a new and interesting field - statistics - to the study of language. This landed him an astonishing insight: in nearly every language, some words are used a lot, but most (nearly all) words are used hardly at all. Only a few words account for most of our use of language. The Brown Corpus is a collection of literature used by linguistics researchers. It consists of 500 samples of English-language text comprising 1 million words. Just 135 unique words are needed to account for 50% of those million words. That's insane. Zipf's probability distribution is *discrete*. Discrete distributions are comprised of whole integers. Continuous distributions can take on any value. If you take Zipf's distribution and make it continuous instead of discrete, you get the Pareto distribution. ## Pareto - the economist Pareto initially noticed a curious distribution when he was thinking about wealth in society - he noticed that 80% of the wealth and income came from 20% of the people in it. {% marginnote_lazy pareto.png||true %} The Pareto distribution, pictured, has been found to hold for a scary number of completely different and unrelated fields in the sciences. For example, here are some natural phenomena that exhibit a Pareto (power law) distribution: * Wealth inequality * Sizes of rocks on a beach * Hard disk drive error rates (!) * File size distribution of Internet traffic (!!!) We tend to think of the natural world as random or chaotic. In schools, we're taught the bell curve/normal distribution. **But reality isn't normally distributed.** It's log-normal. Many probability distributions, in the wild, support the Pareto Principle: > 80% of the output will come from 20% of the input {% marginnote_lazy Normal_Distribution_PDF.svg|Normal distributions are taught in schools because they're quite easy to talk about mathematically, not because they're particularly good descriptions of the natural world.|true %} While you may have heard this before, what I'm trying to get across to you is that it isn't made up. The Pareto distribution is used in hundreds of otherwise completely unrelated scientific fields - and we can use its ubiquity to our advantage. It doesn't matter what area you're working in - if you're applying equal effort to all areas, you *are wasting your time*. What the Pareto distribution shows us is that most of the time, our efforts would be better spent *finding* and *identifying* the crucial 20% that accounts for 80% of the output. Allow me to reformulate and apply this to web application performance: > 80% of an application's work occurs in 20% of its code. There are other applications in our performance realm too: > 80% of an application's traffic will come from 20% of its features. > 80% of an application's memory usage will come from 20% of its allocated objects. The ratio isn't always 80/20. Actually, usually it's way more severe - 90/10, 95/5, 99/1. Sometimes it's less severe. So long as it isn't 50/50 we're talking about a non-normal distribution. This is why premature optimization is so bad and why performance monitoring, profiling and benchmarking are so important. The world is full of power-law distributions, not normal distributions. Spreading your effort evenly across a power-law distribution is a massive waste of effort. What the Pareto Principle reveals to us is that optimizing any random line of code in our application is in fact unlikely to speed up our application at all! 80% of the "slowness" in any given app will be hidden away in a minority of the code. {% marginnote_lazy haskell.png||true %} So instead of optimizing blindly, applying principles at random that we read from blog posts, or engaging in Hacker-News-Driven-Development by using the latest and "most performant" web technologies, we need to measure where the bottlenecks and problem areas are in our application. ## An Optimization Story - Measurement, Profiling and Benchmarking There's only one skill in performance work that you need to understand completely and deeply - how to *measure* your application's performance. Once you have that skill mastered, knowing every possible thing about performance might be a waste of time. Your problems are not other's problems. There are going to be lessons to learn that solve problems you don't have (or don't comprise that crucial 20% of the causes of slowness in your application). On the flip side, you should realize that the Pareto Principle is extremely liberating. You *don't* need to fix every performance issue in your application. You don't need to go line-by-line to look for problems under every rock. You need to *measure* the actual performance of your application, and focus on the 20% of your code that is the worst performance offender. {% marginnote_lazy minitest_knows.jpg|My first conference talk ever, actually.|true %} I once gave [a conference talk that was a guided read-through of Minitest](https://www.youtube.com/watch?v=ojd1G4gOMdk), the Ruby testing framework. Minitest is a great read if you've got a spare hour or two - it's fairly short at just 1,500 lines. As I was reading Minitest's code, I came across this funny line: ```ruby def self.runnable_methods methods = methods_matching(/^test_/) case self.test_order when :random, :parallel then max = methods.size methods.sort.sort_by { rand max } when :alpha, :sorted then methods.sort else raise "Unknown test_order: #{self.test_order.inspect}" end end ``` This code is extremely readable as to what's going on; we determine which methods on a class are runnable with a regex ("starts with test_"), and then sort them depending upon this test class's `test_order`. Minitest uses the return value to execute all of the `runnable_methods` on all the test classes you give it. Usually this is a randomized array of method names, because the default test order is `:random`. What I was honing in on was this line, which is run when `:test_order` is `:random` or `:parallel`: ```ruby max = methods.size methods.sort.sort_by { rand max } ``` This seemed like a really roundabout way to do `methods.shuffle` to me. Maybe Ryan (Minitest's author) was doing some weird thing to ensure deterministic execution given a seed. Minitest runs your tests in the same order given the same seed to the random number generator. It turns out methods.shuffle is deterministic, though, just like the code as written. So, I decided to benchmark it, mostly out of curiosity. Whenever I need to write a micro benchmark of Ruby code, I reach for [`benchmark/ips`](https://github.com/evanphx/benchmark-ips).{% sidenote 1 "The reason I use benchmark/ips rather than the stdlib benchmark is because the stdlib version requires you to run a certain line of code X number of times and tells you how long that took. The problem with that is that I don't usually know how fast the code is to begin with, so I have no idea how to set X. Usually I run the code a few times, guess at a number of X that will make the benchmark take 10 seconds to run, and then move on. benchmark/ips does that work for me by running my benchmark for 10 seconds and calculating iterations-per-second." %} `ips` stands for iterations-per-second. The gem is an extension of the `Benchmark` module, something we get in the Ruby stdlib. Here's that benchmark: ```ruby require "benchmark/ips" class TestBench def methods @methods ||= ("a".."z").to_a end def fast methods.shuffle end def slow max = methods.size methods.sort.sort_by { rand max } end end test = TestBench.new Benchmark.ips do |x| x.report("faster alternative") { test.fast } x.report("current minitest code") { test.slow } x.compare! end ``` This benchmark suggested that `shuffle` was 12x faster than `sort.sort_by { rand methods.size }`. This makes sense - `shuffle` randomizes the array with C, which will always be faster than randomizing it with pure Ruby. In addition, Ryan was actually sorting the array twice - once in alphabetical order, followed by a random shuffle based on the output of `rand`. {% marginnote_lazy ryans_talk.jpg|[Ryan's conference talks](https://www.youtube.com/watch?v=5KVcsV_jseQ) are pretty good, too.|true %} I asked Ryan Davis, `minitest` author, what was up with this. He gave me a great reply: "you benchmarked it, but did you profile it?" What did he mean by this? Well, first, you have to know the difference between **benchmarking and profiling - the two fundamental performance measurement tools.** There are a lot of different ways to define this difference. Here's my attempt: ### Benchmarking A benchmark is a test of one or many different pieces of code that measures how fast they execute or how many resources they consume. When we benchmark, we take two competing pieces of code and compare them. It could be as simple as a one liner, like in my story, or as complex as an entire web framework. Then, we put them up against each other (usually comparing them in terms of iterations/second) using a simple, contrived task. At the end of the task, we come up with a single metric - a score. We use the score to compare the two competing options. In my example above, it was just how fast each line could shuffle an array. If you were benchmarking web frameworks, you might test how fast a framework can return a simple "Hello World" response. Benchmarks put the competing alternatives on exactly equal footing by coming up with a contrived, simple, non-real-world example. {% marginnote_lazy rails-sucks.png|[I wrote a v v long post once about why this benchmark doesn't mean much for Rails](/blog/is-ruby-too-slow-for-web-scale/)|true %} It's usually too difficult to benchmark real-world code because the alternatives aren't doing *exactly* the same thing. For example, comparing Rails against Sinatra isn't entirely fair because Rails has many features that Sinatra does not - even for a simple Hello World response, the Rails application is, for example, performing many security checks that the Sinatra app doesn't. Comparing these frameworks in a 1-to-1 benchmark will always be slightly misleading for that reason. ### Profiling Profiles are a accounting of all the sub-steps required to run a given piece of code. When we profile, we're usually examining the performance characteristics of an entire, real-world application. For example, this might be a web application or a test suite. Because profiling works with real-world code, we can't really use it to compare competing alternatives, because the alternative usually doesn't exactly match what we're profiling. Profiling doesn't usually produce a comparable "score" at the end with which to measure these alternatives, either. But that's not to say profiling is useless - it can tell us a lot of valuable things, like what percentage of CPU time was used where, where memory was allocated, and what lines of code are important and which ones aren't. What Ryan was asking me was - "Yeah, that way is faster on this one line, but does it really matter in the grand scheme of Minitest"? How much time does a Minitest test run actually spend shuffling the methods? 1%? 10%? 0.001%? Profiling can tells us that. {% marginnote_lazy thatwasalie.jpg|You said that this one-line change would speed up minitest. A higher-level benchmark determined *that* was a lie.|true %} Is this one line really part of Pareto's "20%"? We can assume, based on the Principle, that 80% of Minitest's execution time will come from just 20% of its code. Was this line part of that 20%? I've already shown you how to benchmark on the micro scale. But before we get to profiling, I'm going to do a quick macro-benchmark to test my assumption that using `shuffle` instead of `sort.sort_by` will speed up Minitest. Minitest is used to run tests, so we're going to benchmark a whole test suite. [Rubygems.org](https://github.com/rubygems/rubygems.org/), an open-source Rails application with a Minitest suite, will make a good example test suite. When micro-benchmarking, I reach for `benchmark-ips`. When macro-benchmarking (and especially in this case, with a test suite), I usually reach first for the simplest tool available: the unix utility `time`! We're going to run the tests 10 times, and then divide the total time by 10. ``` $ time for i in {1..10}; do bundle exec rake; done ... real 15m59.384s user 11m39.100s sys 1m15.767s ``` When using `time`, we're usually only going to pay attention the `user` statistic. `real` gives the actual total time (as if you had used a stopwatch), `sys` gives the time spent in the kernel (in a test run, this would be things like shelling out to I/O), and `user` will be the closest approximation to time actually spent running Ruby. You'll notice that `user` and `sys` don't add up to `real` - the difference is time spent waiting on the CPU while other operations (like running my web browser, etc) block. With stock `minitest`, the whole thing takes 11 minutes and 39 seconds, for an average of 69.9 seconds per run. Now, let's alter the Gemfile to point to a modified version (with `shuffle` on the line in question) of `minitest` on my local machine: ```ruby gem 'minitest', require: false, path: '../minitest' ``` To make sure the test is 100% fair, I only make the change to my local version after I check out `minitest` to the same version that Rubygems.org is running (5.8.1). {% marginnote_lazy computers.gif|Even so-called performance experts mess this shit up sometimes.|true %} The result? 11 minutes 56 seconds. Longer than the original test! We know my code is faster in micro, but the macro benchmark told me that it actually takes longer. A lot of things can cause this (the most likely being other stuff running on my machine), but what's clear is this - my little patch doesn't seem to be making a big difference to the big picture of someone's test suite. While making this change *would*, in *theory*, speed up someone's suite, in reality, the impact is so minuscule that it didn't really matter. So, while a benchmark told me one thing - X is 10x faster than Y! - a higher-level benchmark told me another (make your change and this thing didn't really matter.) Not only does this show the value of profiling (which would have told me before that the sorting didn't take much of the total time) but also how microbenchmarks and relative comparisons can mislead. Performance measurement is a critical skill. Anywhere along the way, I could have been mislead by a single number or a rogue measurement. By applying a scientific, empirical approach, I was able to put my benchmark in context of a larger program. Premature optimization is ignoring these lessons and optimizing "when we feel like it", or optimizing constantly all the time. Hopefully I've convinced you: it's a guaranteed waste of time. **Repeat after me: I will not optimize anything in my application until my metrics tell me so.** --- ## Why Your Rails App is Slow: Lessons Learned from 3000+ Hours of Teaching URL: https://www.speedshop.co/blog/what-i-learned-teaching-rails-performance/ {% marginnote_lazy setofskills.jpg|"What I do have is a particular set of skills, a set of skills which makes me a nightmare for slow Rails applications like you."|true %} For the last 4 years, I've been working on making Rails applications faster and more scalable. I teach workshops, I [sell a course](https://www.railsspeed.com/), and I [consult](https://www.speedshop.co/retainer.html). If you do anything for a long period of time, you start to see patterns. I've noticed four different factors that prevent software organizations from improving the performance of their Rails applications, and I'd like to share them here. ## Performance becomes a luxury good, especially when no one is watching Often times at my workshops, I discover that an attendee simply has no visiblity into what their application is doing in production - they either don't understand their dashboards, they don't have them, or they're not allowed to even access them ("DevOps team only"). Performance metrics are often just not tracked. No one is aware if the app is over or underscaled, no one knows if the app is "slow" or "fast". Is it any wonder, then, that no one spends any time working on it? **Performance is rarely the first priority of any organization, and often gets "trickled down" hours and resources**. {% marginnote_lazy workcleanfast.png||true %} Some of this is actually a good thing. There's a reason that the classic programming mantra of "make it work, make it clean, make it fast" is in that order and not the opposite way around. People pay for software that does stuff. If it does that stuff quickly and in a pleasantly performant way, then that's great, but it's not always required (especially if the organization is first to market in their space and customers have no other options). Often, my consulting clients are at a point in their organization where they're no longer scraping by on ramen and cheeto dust, but have a solid business that's expanding (slowly or quickly). They've finally gotten their heads above water and they're ready to start thriving, not just surviving. People don't come to me when they're still trying to achieve product market fit unless things have become untenable, and then that's more of a rescue job. {% marginnote_lazy travoltawallet.gif|When the time comes to set the budget on performance instead of feature velocity.|true %} This is a natural and correct progression. It also means organizations accumulate performance debt during that initial period of building and obtaining product-market fit. I think there may be some that believe that all kinds of technical debt are some kind of sinful black stain on any organization, and that if you just Coded The Right Way or were a Software Craftsperson™ this would not have happened. I think that's probably wrong. Before achieving ramen profitability, businesses must take out technical debt as a kind of financing of their own product development runway. This will happen regardless of one's coding techniques or knowledge level. {% marginnote_lazy pmburn.jpg||true %} However, performance is not a luxury good. It isn't something that can simply be ignored until one's organization has a spare four or five figures in the couch cushions. Like technical debt, there is a point when feature work grinds to a halt because the organization is too busy maintaining the performance debt that has accrued. Requests are timing out. Customers are complaining about slow the app feels and switching to competitors. You're scared to check the AWS bill. Ideally, organizations monitor and sensibly take out performance debt when required, and understand the full extent of the work that must be done in the future. To do this sort of "sensible debt accrual", **you need performance monitoring/metrics and you need to understand how to present numbers to management**. I find that while most people know subscribe to a performance monitoring service, such as New Relic, Skylight or Scout, they often have no idea how to read it and extract useful insights from it, making it a very expensive average latency monitor. Being able to actually use your APM is a critical performance skill that I cover in great detail in my workshops and course. {% marginnote_lazy scoutexample.png|If you can't draw insights from this, you're just throwing cash out the door.|true %} Monitoring these metrics allows you to assess where you're at and to figure out what parts of the application have accrued performance debt. It also helps you to make decisions on the "cost/benefit" of future work. It also means you need to be able to "speak manager" or "speak business". The business case for adding more features is obvious to the non-technical side of your organization. There is a great business case for performance, fortunately, both [from the side of the customer](https://wpostats.com/) and from the cost side as well - reducing average latency by 50% means you can spend 50% less on your application's servers thanks to queueing theory and something called [Little's Law](https://en.wikipedia.org/wiki/Little's_law). At my workshops, I spend a lot of time simply discussing terminology, like request queueing, latency, throughput, tracing and profiling. Giving people the vocabulary they need to understand the tools out there seems to be half the battle of getting everyone comfortable reading their own metrics. ## Complex apps and complex problems, with little training This leads me to the second cause of performance problems in software - a simple lack of knowledge. We can't optimize what we don't understand and we can't fix what we can't see. I wrote the [Complete Guide to Rails Performance](https://railsspeed.com) simply because there was so much information about this topic that had simply never been compiled before into one place. {% marginnote_lazy confusedscaleman.jpg|"What's request queueing?"|true %} This shows itself most when scaling for throughput. Most organizations simply aren't tracking critical scaling metrics or even know what they are, often because they believe the platform-as-a-service that they're using should "take care of this" for them. By the time I've been called in, they're spending thousands of dollars a month more than they need to, and could have fixed this months or even years ago with some simple autoscaling policies and a bit of organizational knowledge around scaling. Or, the flipside is happening and they're massive under-scaled, with 25-50% of their total request latency being just time spent queueing for resources. Performance work is not rocket science. However, unlike a lot of other areas in software{% sidenote 1 "The only other area in software that requires an even wider base of knowledge is security. Consider [Rowhammer](https://en.wikipedia.org/wiki/Row_hammer) - basically an electrical engineering exploit in very particular configurations of DRAM." %}, it can require an extremely broad base of knowledge. When your customer says the site "feels slow", the problem can quite reasonably be almost anywhere between the pixels on the user's screen (say, an issue with the customer's client machine) and the electrons running through the silicon on your cloud service provider (for example, a mitigation for a recent Intel security issue puts your servers above capacity). Feature work and even to a large extent refactoring work generally only requires knowledge of the language and frameworks in use. Performance work often needs esoteric knowledge from other fields (such as queueing theory) in addition to highly in-depth knowledge in your frameworks and language.{% sidenote 2 "I wrote a [3000+ word blog](/blog/three-activerecord-mistakes/) about the critical performance differences between English-language synonyms ".present?" and ".exists?" in Rails, for example, but my [Rails performance course](https://www.railsspeed.com) spends the majority of the time talking about things which are not Ruby-specific." %} {% marginnote_lazy debuggingrails.jpg|Looking at a flamegraph of a Rails app for the first time often leads to this reaction.|true %} This depth of knowledge simply isn't present in many organizations, especially those who place sprint velocity before the development of engineering capacity and skills in the organization. The workshops I've been doing have really allowed me to go in deep on complex problems and help people deal with the "wrinkles" introduced by their application. Getting to look over people's shoulders while they experience an error or something I hadn't anticipated has been very rewarding, both for them and for me as an educator. Also, during those workshops, I don't emphasize "pre-baked" problem/solutions, but instead have the attendees bring their real world applications, and we immediately try to apply what we've learned on their actual apps right then and there. I don't want anyone to go home and run into a problem caused by the complexity of their app - rather, I'd like that to happen while we're both in the same room! ## Boiling frogs - even when tracked, performance slips without fix In the Slack channel for the Complete Guide to Rails Performance, we've had a few conversations about managing performance work in the software organization. {% marginnote_lazy elmoflames.gif|Walking into the office on Monday like|true %} An organizational culture that always places completeness over quality inevitably runs into issues. Often when I get new clients, they're experiencing not just performance issues but have problems with all the various dimensions of software quality: low correctness (an excess of bugs and lack of test coverage), high complexity ("technical debt", spaghetti organization), and a poor deployment pipeline (broken builds, janky deploys). These aspects of software quality tend to either all be good or all be bad. Project management can (and often should) sacrifice quality for a period of time to prioritize completeness and features, but when it's done pathologically, it inevitably leads to ruin. I find that the lack of software quality culture often arises because no one is measuring it{% sidenote 3 "I actually really don't vibe with the 'software craftsperson' aesthetic that people like Uncle Bob try to push. Quality is great but it isn't everything. It's possible to turn this into navelgazing, and building ivory towers." %}. Feature velocity is measured, or at least vaguely tracked, with things like pull request counts, sprint points, or user stories. We shipped 5 stories last week, so management expects us to ship 5 this week. Fortunately, many software quality measures are actually very easy to track. How many bugs were reported or experienced by customers last week? How much downtime did we have? How many deploys were there? Are these numbers rising or falling? In terms of performance, most organizations would benefit from setting simple thresholds that, if exceeded, move performance work into the "bug fixing" pipeline that the organization employs. For example, an organization can commit to a maximum 95th percentile latency of 1 second. If a transaction{% sidenote 4 "In New Relic parlance - a single controller action is a 'transaction'." %} exceeds that threshold, a new bug is recorded. For organizations that want to improve the customer's experience and perceived performance of the application, other budgets may be necessary. For example, a first-page-load time of 5 seconds. This page load target has implications that flow down throughout the stack, as one simply cannot ship 10 megabytes of JavaScript and also have a page load in 5 seconds{% sidenote 5 "In fact, I would estimate that to keep page load times below 5 seconds on the average connection and hardware, you can probably ship only a few hundred KB" %}. Software engineers are often poor communicators, and they very often fail to communicate to other parts of the organization that prioritizing feature velocity at all costs is not sustainable. Think of it this way: how do you think the project managers in your organization would answer the following questions? {% marginnote_lazy bezos.gif|Bezos showering in your AWS bill|true %} 1. Is your tolerance for the slowness of our application infinite? (i.e. can the app just beachball for all customers all the time?) 2. Do you have infinite money to spend on our EC2 instance bills? If the answer to either of those questions is "no", then it is **your job as a software developer** to find and make explicit those tolerances. They will be different for every organization. These performance requirements can be easily translated into automated alerts and thresholds. You just have to have the conversation beforehand. The difference between "Hey boss - we've been shipping 12 points a week for the last 8 weeks and now we can't ship anything for 6 weeks because we need to write tests and make the homepage load time somewhat bearable" and "we exceeded the limit for page load time that we all agreed upon 6 months ago, and we'll need to reduce velocity for a while to compensate" is miles apart. As a result of seeing this pattern often enough, I've changed how I phrase my consulting deliverables, as I now realize I need to provide ammunition for the engineers when bringing back my recommendations to the "business side". ## It's not Ruby, and it isn't (really) Rails {% marginnote_lazy leavematzalone.jpg|LEAVE MATZ ALONE|true %} And, finally, here's what isn't the reason why your web application is slow: your framework or language choice. Once 90th percentile latency is lower than 500 milliseconds and median latency is below 100 milliseconds, most web application backends are no longer the bottleneck in their customer's experience (if they ever were to begin with, which, in the age of 10 megabyte JavaScript bundles, they are usually not). It's 2017 and web applications don't return flat HTML files anymore{% sidenote 5 "CNN.com took 5MB of resources and 112 requests to render for me, today. R.I.P. the old light web." %}. Websites are gargantuan, with JavaScript bundles stretching into the size of megabytes and stylesheets that couldn't fit in ten Apollo Guidance Computers. So how much of a difference does a web application which responds in 1 millisecond or less make in this environment? Vanishingly little. Nowadays, the average webpage takes 5 seconds to render. Some JavaScript single-page-applications can take 12 seconds or more on initial render. Server response times simply make up a minority part of the actual user experience of loading and interacting with a webpage - cutting 99 milliseconds off the server response time just doesn't make a difference. Not to mention: if Ruby on Rails, frequently maligned "as too slow" or "can't scale", can run several of the top 1000 websites in the world by traffic, including that little fly-by-night outfit called GitHub, then it's a fine choice for whatever your application is. Rails is just an example here - there are many comparable frameworks in comparable languages that you could substitute like Python and Django. There are some web applications for which 100 milliseconds of latency is an unacceptable eon (advertising is the most common case), but for the vast majority of us delivering HTML or JSON to a client, that's zippy quick. ## Whither Rails Today? One of the questions I ask in my post-workshop survey is "How do you feel writing Ruby on Rails? Would you like to keep doing it?". The answers I get back are always astoundingly positive. For all the FUD on the web at large, people writing Ruby are incredibly happy doing it. And that's what keeps me writing and teaching: as long as people feel that performance concerns are keeping them from enjoying Ruby or choosing it as their tech stack, I'll keep doing what I do. --- ## 3 ActiveRecord Mistakes That Slow Down Rails Apps: Count, Where and Present URL: https://www.speedshop.co/blog/three-activerecord-mistakes/ {% marginnote_lazy weirdriddles.gif|"When does ActiveRecord execute queries? No one knows!"|true %} ActiveRecord is great. Really, it is. But it's an abstraction, intended to insulate you from the actual SQL queries being run on your database. And, if you don't understand how ActiveRecord works, you may be causing SQL queries to run that you didn't intend to. Unfortunately, the performance costs of many features of ActiveRecord means we can't afford to ignore unnecessary usage or treat our ORM as just an implementation detail. We need to understand exactly what queries are being run on our performance-sensitive endpoints. Freedom isn't free, and neither is ActiveRecord. One particular case of ActiveRecord misuse that I find is common amongst my clients is that ActiveRecord is executing SQL queries that aren't really necessary. Most of my clients are completely unaware that this is even happening. {% marginnote_lazy dirtythree.jpg||true %} Unnecessary SQL is a common cause of overly slow controller actions, especially when the unnecessary query appears in a partial which is rendered for every element in a collection. This is common in search actions or index actions. This is one of the most common problems I encounter in my performance consulting. It's a problem in nearly every app I've ever worked on. One way to eliminate unnecessary queries is to poke our heads into ActiveRecord and understand its internals, and know exactly how certain methods are implemented. **Today, we're going to look at the implementation and usage of three methods which cause lots of unnecessary queries in Rails applications: `count`, `where` and `present?`**. ## How Do I Know if a Query is Unnecessary? I have a rule of thumb to judge whether or not any particular SQL query is unnecessary. Ideally, a Rails controller action should execute **one SQL query per table**. If you're seeing more than one SQL query per table, you can usually find a way to reduce that to one or two queries. If you've got more than a half-dozen or so queries on a single table, you almost definitely have unnecessary queries. {% sidenote 1 "Please don't email or tweet with me with 'Well ackshually...' on this one. It's a guideline, not a rule, and I understand there are circumstances where more than one query per table is a good idea." %} The number of SQL queries per table can be easily seen on NewRelic, for example, if you have that installed. {% marginnote_lazy washeyes.jpg|I keep an eyewash station next to my desk for really bad N+1s|true %} Another rule of thumb is that **most queries should execute during the first half of a controller action's response, and almost never during partials**. Queries executed during partials are usually unintentional, and are often N+1s. These are easy to spot during a controller's execution if you just read the logs in development mode. For example, if you see this: ``` User Load (0.6ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 2]] Rendered posts/_post.html.erb (23.2ms) User Load (0.3ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 3]] Rendered posts/_post.html.erb (15.1ms) ``` ... you have an N+1 in this partial. Usually, when a query is executed halfway through a controller action (somewhere deep in a partial, for example) it means that you haven't [`preload`ed](https://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#method-i-preload) the data that you needed. So, let's look specifically at the `count`, `where` and `present?` methods, and why they cause unnecessary SQL queries. ## .count executes a COUNT every time I see this one at almost every company I contract for. It seems to be little-known that calling `count` on an ActiveRecord relation will *always* try to execute a SQL query, every time. This is inappropriate in most scenarios, but, in general, **only use `count` if you want to always execute a SQL COUNT _right now_**. {% marginnote_lazy count.gif|"How many queries do we want per table?"|true %} The most common cause of unnecessary `count` queries is when you `count` an association you will use later in the view (or have already used): ``` # _messages.html.erb # Assume @messages = user.messages.unread, or something like that

Unread Messages: <%= @messages.count %>

<% @messages.each do |message| %> blah blah blah <% end %> ``` This executes 2 queries, a `COUNT` and a `SELECT`. The COUNT is executed by `@messages.count`, and `@messages.each` executes a SELECT to load all the messages. Changing the order of the code in the partial and changing `count` to `size` eliminates the `COUNT` query completely and keeps the `SELECT`: ``` <% @messages.each do |message| %> blah blah blah <% end %>

Unread Messages: <%= @messages.size %>

``` Why is this the case? We need not look any further than [the actual method definition of `size` on ActiveRecord::Relation:](https://github.com/rails/rails/blob/94b5cd3a20edadd6f6b8cf0bdf1a4d4919df86cb/activerecord/lib/active_record/relation.rb#L210) ```ruby # File activerecord/lib/active_record/relation.rb, line 210 def size loaded? ? @records.length : count(:all) end ``` {% marginnote_lazy triggeredcount.jpg||true %} If the relation is loaded (that is, the query that the relation describes has been executed and we have stored the result), we call `length` on the already loaded record array. [That's just a simple Ruby method on Array](https://ruby-doc.org/core-2.5.0/Array.html#method-i-length). If the ActiveRecord::Relation *isn't* loaded, we trigger a `COUNT` query. On the other hand, [here's how `count` is implemented](https://github.com/rails/rails/blob/94b5cd3a20edadd6f6b8cf0bdf1a4d4919df86cb/activerecord/lib/active_record/relation/calculations.rb#L41) (in ActiveRecord::Calculations): ```ruby def count(column_name = nil) if block_given? # ... return super() end calculate(:count, column_name) end ``` And, of course, [the implementation of `calculate`](https://github.com/rails/rails/blob/94b5cd3a20edadd6f6b8cf0bdf1a4d4919df86cb/activerecord/lib/active_record/relation/calculations.rb#L131) doesn't memoize or cache anything, and executes a SQL calculation every time it is called. Simply changing `count` to `size` in our original example would have still triggered a `COUNT`. The record's wouldn't be `loaded?` when `size` was called, so ActiveRecord will still attempt a `COUNT`. Moving the method *after* the records are loaded eliminates the query. Now, moving our header to the end of the partial doesn't really make any logical sense. Instead, we can use the `load` method. ```

Unread Messages: <%= @messages.load.size %>

<% @messages.each do |message| %> blah blah blah <% end %> ``` `load` just causes all of the records described by `@messages` to load immediately, rather than lazily. [It returns the ActiveRecord::Relation, not the records.](https://api.rubyonrails.org/classes/ActiveRecord/Relation.html#method-i-load) So, when `size` is called, the records are `loaded?` and a query is avoided. Voilà. What if, in that example, we used `messages.load.count`? We'd still trigger a COUNT query! When *doesn't* `count` trigger a query? Only if the result has been cached by `ActiveRecord::QueryCache`.{% sidenote 2 "I have some Opinions on the use of QueryCache, but that's a post for another day." %} This could occur by trying to run the same SQL query twice: ```

Unread Messages: <%= @messages.count %>

... lots of other view code, then later:

Unread Messages: <%= @messages.count %>

``` {% marginnote_lazy pissed.gif|Every time you use count when you could have used size|true %} **In my opinion, most Rails developers should be using `size` in most of the places that they use `count`.** I'm not sure why everyone seems to write `count` instead of `size`. `size` uses `count` where it is appropriate, and it doesn't when the records are already loaded. I think it's because when you're writing an ActiveRecord relation, you're in the "SQL" mindset. You think: "This is SQL, I should write count because I want a COUNT!" So, when do you actually want to use `count`? Use it when you won't actually *ever* be loading the full association that you're `count`ing. For example, take this view on Rubygems.org, which displays a single gem: In the "versions" list, the view does a `count` to get the total number of releases (versions) of this gem. [Here's the actual code:](https://github.com/rubygems/rubygems.org/blob/d8a48488d29cbfc83efd2e936c74290c54041288/app/views/rubygems/show.html.erb#L36) ``` <% if show_all_versions_link?(@rubygem) %> <%= link_to t('.show_all_versions', :count => @rubygem.versions.count), rubygem_versions_url(@rubygem), :class => "gem__see-all-versions t-link--gray t-link--has-arrow" %> <% end %> ``` The thing is, this view *never* loads *all* of the Rubygem's versions. It only loads five of the most recent ones, in order to show that versions list. So, a `count` makes perfect sense here. Even though `size` would be logically equivalent (it would just execute a COUNT as well because `@versions` is not `loaded?`), it states the intent of the code in a clear way. My advice is to grep through your `app/views` directory for `count` calls and make sure that they actually make sense. If you're not 100% sure that you really need a real SQL `COUNT` right then and there, switch it to `size`. Worst case, ActiveRecord will still execute a `COUNT` if the association isn't loaded. If you're going to use the association later in the view, change it to `load.size`. ## .where means filtering is done by the database What's the problem with this code (let's say its `_post.html.erb`) ``` <% @posts.each do |post| %> <%= post.content %> <%= render partial: :comment, collection: post.active_comments %> <% end %> ``` and in Post.rb: ```ruby class Post < ActiveRecord::Base def active_comments comments.where(soft_deleted: false) end end ``` {% marginnote_lazy whoaguy.gif||true %} If you said, "this causes a SQL query to be executed on every rendering of the post partial", you're correct! `where` always causes a query. I didn't even bother to write out the controller code, because *it doesn't matter*. You can't use `includes` or other preloading methods to stop this query. `where` will always try to execute a query! This also happens when you call scopes on associations. Imagine instead our Comment model looked like this: ```ruby class Comment < ActiveRecord::Base belongs_to :post scope :active, -> { where(soft_deleted: false) } end ``` Allow me to sum this up with two rules: **Don't call scopes on associations when you're rendering collections** and **don't put query methods, like `where`, in instance methods of an ActiveRecord::Base class**. Calling scopes on associations means we cannot preload the result. In the example above, we can preload the comments on a post, but we can't preload the *active* comments on a post, so we have to go back to the database and execute new queries for every element in the collection. This isn't a problem when you only do it once, and not on every element of a collection (like every post, as above). Feel free to use scopes galore in those situations - for example, if this was a PostsController#show action that only displayed one post and its associated comments. But in collections, scopes on associations cause N+1s, every time. The best way I've found to fix this particular problem is to **create a new association**. [Justin Weiss](https://www.justinweiss.com/), of "Practicing Rails", taught me this in [this blog post about preloading Rails scopes](https://www.justinweiss.com/articles/how-to-preload-rails-scopes/). The idea is that you create a new association, which you *can* preload: ```ruby class Post has_many :comments has_many :active_comments, -> { active }, class_name: "Comment" end class Comment belongs_to :post scope :active, -> { where(soft_deleted: false) } end class PostsController def index @posts = Post.includes(:active_comments) end end ``` The view is unchanged, but now executes just 2 SQL queries, one on the Posts table and one on the Comments table. Nice! ``` <% @posts.each do |post| %> <%= post.content %> <%= render partial: :comment, collection: post.active_comments %> <% end %> ``` The second rule of thumb I mentioned, **don't put query methods, like where, in instance methods of an ActiveRecord::Base class**, may seem less obvious. Here's an example: ```ruby class Post < ActiveRecord::Base belongs_to :post def latest_comment comments.order('published_at desc').first end ``` What happens if the view looks like this? ``` <% @posts.each do |post| %> <%= post.content %> <%= render post.latest_comment %> <% end %> ``` {% marginnote_lazy rules.gif||true %} That's a SQL query on every post, regardless of what you preloaded. In my experience, **every instance method on an ActiveRecord::Base class will eventually get called inside a collection**. Someone adds a new feature and isn't paying attention. Maybe it's by a different developer than the one who wrote the method originally, and they didn't fully read the implementation. Ta-da, now you've got an N+1. The example I gave could be rewritten as an association, like I described earlier. That can still cause an N+1, but at least it can be fixed easily with the correct preloading. Which ActiveRecord methods should we *avoid* inside of our ActiveRecord model instance methods? Generally, it's pretty much everything in the [`QueryMethods`](https://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html), [`FinderMethods`](https://api.rubyonrails.org/classes/ActiveRecord/FinderMethods.html), and [`Calculations`](https://api.rubyonrails.org/classes/ActiveRecord/Calculations.html). Any of these methods will usually *try* to run a SQL query, and are resistant to preloading. `where` is the most frequent offender, however. ## any?, exists? and present? Rails programmers have been struck by a major affliction - they're adding a particular predicate method to just about every variable in their applications. `present?` has spread across Rails codebases faster than the plague in 13th century Europe. The vast majority of the time, the predicate adds nothing but verbosity, and really, all the author needed was a truthy/falsey check, which they could have done by just writing the variable name. [Here's an example](https://github.com/codetriage/codetriage/blob/b92e347e0f4714b4646be930e341be5a44761b95/app/models/doc_comment.rb#L9) from [CodeTriage](https://www.codetriage.com/), a free and open-source Rails application written by my friend [Richard Schneeman](https://schneems.com/): ```ruby class DocComment < ActiveRecord::Base belongs_to :doc_method, counter_cache: true # ... things removed for clarity... def doc_method? doc_method_id.present? end end ``` What is `present?` doing here? One, it transforms the value of doc_method_id from either `nil` or an `Integer` into `true` or `false`. Some people have Strong Opinions about whether predicates should return true/false or can return truthy/falsey. I don't. But adding `present?` also does something else, and we have to [look at the implementation](https://github.com/rails/rails/blob/94b5cd3a20edadd6f6b8cf0bdf1a4d4919df86cb/activesupport/lib/active_support/core_ext/object/blank.rb#L26) to figure out what: ```ruby class Object def present? !blank? end end ``` `blank?` is a more complicated question than "is this object truthy or falsey". Empty arrays and hashes are truthy, but `blank`, and empty strings are also `blank?`. In the example above from CodeTriage, however, the only things that `doc_method_id` will *ever* be is `nil` or `Integer`, meaning `present?` is logically equivalent to `!!`: ```ruby def doc_method? !!doc_method_id # same as doc_method_id.present? end ``` {% marginnote_lazy oldmanyellscloud.jpg||true %} Using `present?` in cases like this is the wrong tool for the job. If you don't care about "emptiness" in the value you're calling the predicate on (i.e. the value cannot be `[]` or `{}`), use the simpler (and much faster) language features available to you. I sometimes see people even do this on values *which are already boolean*, which means you're just adding verbosity and making me wonder if there's some weird edge cases I'm not seeing. Alright, that's my style gripe. I understand that you may not agree. `present?` makes more sense when dealing with strings, which can frequently be empty (`""`). **Where people get into trouble is calling predicates, such as `present?`, on ActiveRecord::Relation objects.** Let's say you need to know if an ActiveRecord::Relation has any records. You can use the English-language synonyms any?/present?/exists? or their negations none?/blank?/empty?. Surely it doesn't matter which method you choose, right? Just pick the one that sounds the most natural when read aloud? Nope. What SQL queries do you think the following code will execute? Assume `@comments` is an ActiveRecord::Relation. ``` - if @comments.any? h2 Comments on this Post - @comments.each do |comment| ``` The answer is *two*. One will be an existence check, triggered by `@comments.any?` (`SELECT 1 AS one FROM ... LIMIT 1`), then the `@comments.each` line will trigger a loading of the entire relation (`SELECT "comments".* FROM "comments" WHERE ...`). What about this? ``` - unless @comments.load.empty? h2 Comments on this Post - @comments.each do |comment| ``` This one only executes one query - `@comments.load` loads the entire relation right away with `SELECT "comments".* FROM "comments" WHERE ...`. And this one? ``` - if @comments.exists? This post has = @comments.size comments - if @comments.exists? h2 Comments on this Post - @comments.each do |comment| ``` Four! `exists?` doesn't memoize itself and it doesn't load the relation. `exists?` here triggers a `SELECT 1 ...`, `.size` triggers a `COUNT` because the relation hasn't been loaded yet, and then the next `exists?` triggers ANOTHER `SELECT 1 ...` and finally `@comments` loads the entire relation! Yay! Isn't this fun? You could reduce this down to just 1 query with the following: ``` - if @comments.load.any? This post has = @comments.size comments - if @comments.any? h2 Comments on this Post - @comments.each do |comment| ``` And it just gets better - this behavior changes depending if you're on Rails 4.2, Rails 5.0 or Rails 5.1+. Here's how it works in Rails 5.1+: | method | SQL generated | memoized? | implementation | Runs query if `loaded?` | |----------|--------------------------------------|--------------------|----------------------------|-------------------------| | present? | SELECT "users".* FROM "users" | yes (`load`) | Object (!blank?) | no | | blank? | SELECT "users".* FROM "users" | yes (`load`) | `load`; `blank?` | no | | any? | SELECT 1 AS one FROM "users" LIMIT 1 | no unless `loaded` | `!empty?` | no | | empty? | SELECT 1 AS one FROM "users" LIMIT 1 | no unless `loaded` | `exists?` if !`loaded?` | no | | none? | SELECT 1 AS one FROM "users" LIMIT 1 | no unless `loaded` | `empty?` | no | | exists? | SELECT 1 AS one FROM "users" LIMIT 1 | no | ActiveRecord::Calculations | yes | Here's how it works in Rails 5.0: | method | SQL generated | memoized? | implementation | Runs query if `loaded?` | |--------|------------------------------|--------------------|-----------------|-------------------------| | present? | SELECT "users".* FROM "users" | yes (`load`) | Object (!blank?) | no | | blank? | SELECT "users".* FROM "users" | yes (`load`) | `load`; `blank?` | no | | any? | SELECT COUNT(*) FROM "users" | no unless `loaded` | `!empty?` | no | | empty? | SELECT COUNT(*) FROM "users" | no unless `loaded` | count(:all) > 0 | no | | none? | SELECT COUNT(*) FROM "users" | no unless `loaded` | `empty?` | no | | exists? | SELECT 1 AS one FROM "users" LIMIT 1 | no | ActiveRecord::Calculations | yes | Here's how it works in Rails 4.2: | method | SQL generated | memoized? | implementation | Runs query if `loaded?` | |----------|-------------------------------|---------------------|-----------------|-------------------------| | present? | SELECT "users".* FROM "users" | yes | Object (!blank?)| no | | blank? | SELECT "users".* FROM "users" | yes | to_a.blank? | no | | any? | SELECT COUNT(*) FROM "users" | no unless `loaded` | `!empty?` | no | | empty? | SELECT COUNT(*) FROM "users" | no unless `loaded` | count(:all) > 0 | no | | none? | SELECT "users".* FROM "users" | yes (`load` called) | Array | no | | exists? | SELECT 1 AS one FROM "users" LIMIT 1 | no | ActiveRecord::Calculations | yes | `any?`, `empty?` and `none?` remind me of the implementation of `size` - if the records are `loaded?` do a simple method call on a basic Array, if they're not loaded, *always run a SQL query*. `exists?` has no caching or memoization built in, just like other ActiveRecord::Calculations. This means that `exists?`, which is another method people like to write in these circumstances, is actually much worse than `present?` in some cases! **These six predicate methods, which are English-language synonyms all asking the same question, have completely different implementations and performance implications, and these consequences depend on which version of Rails you are using.** So, let me distill all of the above into some concrete advice: * `present?` and `blank?` should not be used if the ActiveRecord::Relation will never be used in its entirety after you call `present?` or `blank?`. For example, `@my_relation.present?; @my_relation.first(3).each`. * `any?`, `none?` and `empty?` should probably be replaced with `present?` or `blank?` unless you will only take a section of the ActiveRecord::Relation using `first` or `last`. They will generate an extra existence SQL check if you're just going to use the entire relation if it exists. In essence, change `@users.any?; @users.each...` to `@users.present?; @users.each...` or `@users.load.any?; @users.each...`, but `@users.any?; @users.first(3).each` is fine. * `exists?` is a lot like `count` - it is never memoized, and always executes a SQL query. Most people probably do not actually want this behavior, and would be better off using `present?` or `blank?` ## Conclusion {% marginnote_lazy doless.gif||true %} As your app grows in size and complexity, unnecessary SQL can become a real drag on your application's performance. Each SQL query involves a round-trip back to the database, which entails, usually, at *least* a millisecond, and sometimes much more for complex `WHERE` clauses. Even if one extra `exists?` check isn't a big deal, if it suddenly happens in every row of a table or a partial in a collection, you've got a big problem! ActiveRecord is a powerful abstraction, but since database access will never be "free", we need to be aware of how ActiveRecord works internally so that we can avoid database access in unnecessary cases. ## App Checklist * Look for uses of `present?`, `none?`, `any?`, `blank?` and `empty?` on objects which may be ActiveRecord::Relations. Are you just going to load the entire array later if the relation is present? If so, add `load` to the call (e.g. `@my_relation.load.any?`) * Be careful with your use of `exists?` - it ALWAYS executes a SQL query. Only use it in cases where that is appropriate - otherwise use `present?` or any other the other methods which use `empty?` * Be extremely careful using `where` in instance methods on ActiveRecord objects - they break preloading and often cause N+1s when used in rendering collections. * `count` always executes a SQL query - audit its use in your codebase, and determine if a `size` check would be more appropriate. --- ## The Complete Guide to Rails Performance, Version 2 URL: https://www.speedshop.co/blog/rails-performance-version-two/ Today, the Complete Guide to Rails Performance has been updated to version 2.0. [You can purchase it here](https://www.railsspeed.com). All existing purchasers have had their copies updated on Gumroad. When I started this project, I always believed that a digital course should be *better* than a typical paperback programming book. That's why I don't include any DRM or proprietary video codecs. That's why I think, like most software, updates should be free. "Version 2.0" isn't quite as drastic a change as a software v2.0, though. The world of Rails performance has actually changed very little since I wrote the course 2 years ago. The apps I consult on still have many of the same problems. The V2 update reflects this: I have revised the content for clarity, and updated a few places to reflect changes in Ruby 2.5 and Rails 5.2, but it is mostly still the same. I have also added four lessons: memory fragmentation, application server config, GC tuning, and PGBouncer config. These lessons were added based on new problems and thinking I've had since the course was released. Web-Scale Package purchasers will also get a new interview with Noah Gibbs of Appfolio next week. So, what does it mean that not much has changed in the Rails performance world? This tweet put me in an introspective mood this morning:

It is profoundly sad how Rails has institutionalized a "nobody cares" attitude toward performance. https://t.co/UhzvxyLjuz

— Jeff Atwood (@codinghorror), June 1, 2018

To summarize, Jeff's cofounder, Sam Saffron (who I interviewed for the CGRP), wrote a great, in-depth blog post about memory use in ActiveRecord. In short, Sam finds that ActiveRecord creates excessive amounts of objects, even when doing simple and supposedly "optimized" work. Sam posted a proof-of-concept patch which improves this quite a bit. Jeff's tweet diminishes the work of many Rails contributors. Aaron Patterson has spent the last two years working on Rails performance and a compacting garbage collector. Richard Schneeman has improved Sprockets' performance a great deal. Sam Saffron himself has contributed over a dozen performance improvements to Rails, which, as far I can tell, have all been accepted. I know also that Andrew White, Eileen Uchitelle, and Rafael Franca are all Rails core members that care deeply about performance (probably because all of them have day-jobs running large Rails applications!). So any idea that Rails' contributors or core members "don't care" about performance is laughingly misguided, and is an opinion that can only really be held by someone outside the community. The way Jeff tried to turn it around in the replies into a "hot take" that people should "get angry" and "punk rock" about the "status quo" just made it more obvious. It's pretty easy to take potshots at a mature framework like Ruby on Rails. It has almost 13 years of history behind it. There's going to be cruft, baggage, and outdated decisions baked in. That's what happens. But there's also tremendous productivity, something gained from the thousands of contributors who have all contributed their "lessons learned" back to the framework. But if you forget about that history, it's easy to craft a benchmark to make it look like that history has overtaken it's usefulness in the present. This is the gap I've tried to bridge in my writing and in publishing The Complete Guide to Rails Performance. **I believe that performance problems in Rails are pedagogical, not technical**. It's not because we don't have enough people working on performance (though it helps!). It's not because we don't value it as a community (how many times do I have to cite all of the top 10,000 websites that run Rails at speed?). **It's because Rails (and Ruby) optimizes for programmer happiness, and that means we provide sharp tools which are easy to cut yourself on.** Rather than throw the tools out, I think we need to teach people to use them safely. ActiveRecord is probably the best example of what I'm talking about. It's an extremely productive tool. It works very well for 80% of web-app use-cases. But every year, someone wants to throw it out and thinks that some other Rubygem or pattern (e.g. DataMapper) will save them. It's so easy to craft a line of code with ActiveRecord that will slow your application to a crawl if you're not thinking through through the consequences, as anyone who has written `User.all.each` can tell you. There is no One True Pattern or One True Framework. But there is a Thing Which Works For Most People. And if you end up being one of the 20% for whom it doesn't work so well, or the tool's productivity preference means that it's easier to make performance mistakes, I don't think that's the tool or framework's fault. In this way, I think publishing the Complete Guide to Rails Performance was placing my faith in the developer community of Rails. If I didn't think that people could make their Rails apps faster through knowledge and skills, and instead they had to wait until the framework or the language itself got faster, I would have gone to work at Github or Shopify and made a bunch of patches to Rails and Ruby. I might have started an alternative, "lightweight" framework or ORM that prioritized performance over usefulness and productivity. Instead, I think that **teaching Rails developers how to find and fix performance problems** will make a bigger dent in the average Rails app's response time than improving the language or framework's performance by even 2-3x, or by removing "dangerous" features. As I think we've slowly discovered over the course of trying to make Ruby 3x faster, there is no "waste" or "bloat" that can be cut out of a framework or language without cost that suddenly makes the whole thing faster. It's sort of like how politicians always promise to "cut waste in government spending", but no-one can ever tell you exactly where or how which programs will be cut. Everything was implemented for a reason. There is no magic wand or amount of man-hours that can be waved at these problems. I've discovered this in my consulting and writing as well. I wish it was that easy. But it isn't. However, far from Jeff's doomsday attitude, I believe that the macro picture for Ruby and Rails performance looks good, as it always had. Ruby 2.0 to 2.5 made a number of incremental performance improvements, particularly in garbage collection. I feel like the community has become more mature and performance-savvy over the last few years too. We're waking up the mainstream Rails developer to things like `jemalloc` and teaching them how to use ActiveRecord and avoid performance issues. The technical future of Ruby looks strong, too. Ruby 2.6 will contain a JIT compiler. How cool is that? TruffleRuby has made great progress to becoming useable enough to run a Rails application. JRuby continues to truck along with more performance improvements and compatibility fixes all the time. The technical future of the language hardly looks dim - in fact, I think it's much brighter than it was in 2011, when I got started in Ruby and Rails. I'll continue to do my part for the Rails performance community by publishing and writing, to improve the technical skills and capacity of the average Rails developer so that they can make their apps faster. Here's to you, developers! --- ## A New Ruby Application Server: NGINX Unit URL: https://www.speedshop.co/blog/nginx-unit-for-ruby/ There's a new application server on the block for Rubyists - NGINX Unit. As you could probably guess by the name, it's a project of [NGINX Inc.](https://www.nginx.com/company/), the for-profit open-source company that owns the NGINX web server. In fall of 2017, they announced the [NGINX Unit](https://unit.nginx.org/) project. It's essentially an application server designed to replace all of the various application servers used with NGINX. In Ruby's case, that's Puma, Unicorn, and Passenger.{% sidenote 1 "For a far more in-depth comparison of these application servers, read [my article about configuring Puma, Passenger and Unicorn](/blog/appserver/)" %} NGINX Unit also runs Python, Go, PHP and Perl. The overarching idea seems to be to make microservice administration a lot easier. One NGINX Unit process can run any number of applications running any number of languages - for example, one NGINX Unit server can manage a half-dozen different Ruby applications, each running a different version of the Ruby runtime. Or you can run a Ruby application and a Python application side-by-side. The combinations are only limited by your system resources. {% marginnote_lazy bullshit-meter.gif||true %} Unfortunately, the "microservice" space is quite prone to buzzword-laden marketing pages.{% sidenote 2 "I really don't like when software projects advertise themselves as \"modern\". It's like \"subtweeting\" all pre-existing software projects in this problem space and saying they're all old and busted, and this is the New Way To Do Things. Why it's better than the \"old busted ways\" is never explicitly stated. This kind of marketing preys on software developer's fear of becoming obsolete in their skillset, rather than making any substantive point." %} Words like "dynamic", "modular", "lightweight" are mixed in with "service mesh", "seamless" and "graceful". This article is going to be about cutting through the marketing and getting into what NGINX Unit means for those of us running production Ruby applications. Before I move on to more about NGINX Unit's architecture and what makes it unique, let's make sure we all understand the difference between an application server and a web server. A **web server** connects to clients over HTTP, and usually serves static files or **proxies** to other HTTP-enabled servers, and acts as a middleman. An **application server** is the thing which actually starts and runs the language runtime. In Ruby, these functions are sometimes combined. For example, all of the major Ruby application servers *also* are web servers. However, many web servers, such as Nginx and Apache, are *not* also application servers. Nginx UNIT is both a web *and* application server. {% marginnote_lazy nginx-unit.png||false %} NGINX Unit runs four different types of processes: main, router, controller, and application. Application processes are the self-explanatory ones - this would just be the Ruby runtime running your Rails application. The router and controller processes, and how they interact with each other and the application processes, is what defines how NGINX Unit works. The main process creates the router and application processes. That's really all it does. Application processes in NGINX Unit are dynamic, however - the number of processes running can be changed at any time, Ruby versions can be changed, or even entire new applications can be added while the server is running. The thing that tells the main process what application processes to run is the controller process. The controller process (like the main process, there's only one) has two jobs: expose a JSON configuration API over HTTP, and configure the router and main processes. This is probably the most novel and interesting part of NGINX Unit for Rubyists. Rather than working with configuration files, you POST JSON objects to the controller process to tell it what to do. For example, with this json file: ``` { "listeners": { "*:3000": { "application": "rails" } }, "applications": { "rails": { "type": "ruby", "processes": 5, "script": "/www/railsapp/config.ru" } } } ``` ... we can PUT it to an NGINX Unit controller process with this (assuming our NGINX Unit server is listening on port 8443): ``` curl -d "myappconfig.json" -X PUT '127.0.0.1:8443' ``` ... and create a new Ruby application. NGINX Unit's JSON configuration object is divided into *listeners* and *applications*. Applications are the actual apps you want to run. Listeners are where those apps are exposed to the world (i.e. what port they're on). Changes in application and listener configuration are supposed to be seamless. For example, a "hot deploy" of a new version of your application would be accomplished by adding a new application to the configuration: ``` { "rails-new": { "type": "ruby", "processes": 5, "script": "/www/rails-new-app/config.ru" } } curl -d "mynewappconfig.json" -X PUT ``` and then switching the listener to the new application: ``` curl -X PUT -d '"rails-new"' '127.0.0.1:8443/listeners/*:3000/application` ``` This transition is (supposedly) seamless, and clients won't notice. This is similar to a Puma "phased restart". In a phased restart in Puma, each worker process is restarted one at a time, which means that the other works processes are up and available to take requests. Puma accomplishes this using a control server (managed by the `pumactl` utility). However, unlike Puma, NGINX Unit "hot restarts" will not have two versions of the application taking requests at the same time. In a [Puma phased restart](https://github.com/puma/puma/blob/master/docs/restart.md), say your application has six workers. Halfway through the phased restart, 3 works will be running the old code, and half will be running new code. This can cause some problems with database schema changes, for example. NGINX Unit restarts happen "all at once", so while two versions of the code will be running at once, only one version will be taking requests at any point in time. This functionality seems quite useful to those who are running their own Ruby applications on a service such as AWS, where you have to manage your own deployment. However, Heroku users won't find any of this useful, as you've already had this sort of "hot deploy" functionality using [Heroku's preboot system](https://devcenter.heroku.com/articles/preboot). However, these two features aren't doing exactly the same thing. Heroku creates an entirely new virtual server and hot-swaps the whole thing, whereas NGINX Unit is just changing processes on a single machine, but they're completely the same from a client perspective. {% marginnote_lazy nginx-unit-router.png||false %} The router process is pretty much what it sounds like - the thing which turns HTTP connections from clients into requests to the web application processes. NGINX claims a single Unit router can handle thousands of simultaneous connections. The router works a lot like an NGINX web server, and has a number of worker threads to accept, buffer and parse incoming connections. To me, this is one of the most exciting parts of NGINX Unit for Rubyists. It is very difficult for Ruby application servers to deal with HTTP connections without some kind of reverse proxy in front of the app server. Unicorn, for example, is recommended for use only behind a reverse proxy because it cannot buffer requests. That is, if a client sends one byte of their request and then stops (due to network conditions, a bad cellphone connection perhaps), then the Unicorn process just stops all work and cannot continue until that request has finished buffering. Using NGINX, for example, in front of Unicorn allows NGINX to buffer that request before it reaches Unicorn. Since NGINX is written in highly optimized C and it's *not* restricted by Ruby's GVL, it can buffer hundreds of connections for Unicorn. Passenger solves this problem by basically just being an addon for NGINX or Apache{% sidenote 3 "Now you know why it's called *Passenger*!" %} (`mod_ruby`!) and offloading all of the connection-related work to the webserver. In this way, NGINX Unit is more similar to Passenger than it is to Unicorn. The application configuration has a `processes` key. This key can have a minimum number and maximum number of processes: ``` { "rails-new": { "type": "ruby", "processes": { "spare": 5 "max": 10 }, "script": "/www/rails-new-app/config.ru" } } ``` For some reason, the "minimum" number of processes is called "spare". The config above will start 5 processes immediately, and will scale to 10 if the load requires it. No word yet on if any settings like Puma's `preload_app!` and similar settings in Passenger and Unicorn are available so you will be able to start up processes before they are needed *and* take advantage of copy-on-write memory. This leaves the application processes. The interesting and novel thing here is that the router does not communicate with the application processes via HTTP - it uses Unix sockets and shared memory. This looks like an optimization aimed at microservice architectures, as communicating between services on the same machine will be considerably faster without any HTTP in between. I have yet to see any Ruby code examples of how this could work, however. It is unclear to me in the long-term if it is intended for you to run NGINX in front of NGINX Unit, or if NGINX Unit can run on it's own without anything in front of it. As of right now (Q1 2018), you should probably be running NGINX *in front* of NGINX Unit as a reverse proxy, because NGINX Unit lacks static file serving, HTTPS (TLS), and HTTP/2. Obviously, the [integration is pretty seamless](http://unit.nginx.org/integration/). NGINX Unit is approaching a stable 1.0 release. You can't really run it in production right now for Ruby applications: As I write this sentence, the Ruby module is literally 5 days old. It's still under very active development right now - minor versions are released every few weeks. TLS and HTTP-related features seem like the next "big features" to come down the pipe, with static file serving being next. There is *some* discussion about support for Java, which could probably be turned into support for JRuby and TruffleRuby as well. There is no Windows support, and I don't think I would hold my breath for any in the future. NGINX Unit only supports Ruby 2.0 and above. I will not be benchmarking NGINX Unit in this post. It's Ruby module is extremely new and probably not ready for any kind of benchmarking. However, the real reason I won't be benchmarking NGINX Unit against Puma, Unicorn or Passenger is because application server choice in Ruby is not a matter of speed (techincally, latency) but throughput. Application servers tend to differ in *how many requests* they can serve in parallel, rather than *how quickly they do it*. Application servers impose very little latency overhead on the applications they serve, probably on the order of a couple of milliseconds. The most important Ruby application server setting which affects throughput is *threading*. The reason is that it is the only application server setting which can increase the number of requests served *concurrently*. A multithreaded Ruby application server can make greater and more efficient use of the available CPU and memory resources and serve more requests-per-minute than a single-threaded Ruby application process. Currently, the only *free* application server which runs Ruby web applications in multiple threads is Puma. Passenger Enterprise will do it, but you must pay for a license. NGINX Unit plans support for multiple threads in Python applications, so it is not inconceivable that it will support Ruby applications in multiple threads sometime in the future. So, how does NGINX Unit currently "shake out" in comparison to Unicorn, Passenger and Puma? I think that the traditional Rails application setup: one monolithic application, run on a Plaform-as-a-Service provider like Heroku will probably not see any benefit at all from NGINX Unit's current features and planned roadmap. Puma already serves these users very well. NGINX Unit may be interesting for Unicorn users who want to stop using a reverse proxy. Once NGINX Unit's HTTP features are fleshed out, it could replace a Unicorn/NGINX setup with just a single NGINX Unit server. NGINX Unit is probably most *directly* comparable to Phusion Passenger, which also recently went into the "microservice" realm by supporting Javascript and Python as well as Ruby applications. NGINX Unit currently supports more languages and will probably support even more in the future, so those that need greater language support will probably switch. However, Phusion is a Ruby-first company, so I expect Passenger to always "support" Ruby in a better, more complete way than NGINX Unit ever will. And, as mentioned above, Phusion Passenger Enterprise supports multithreaded execution *today*. So, what is the ideal NGINX Unit app? If you're running your own cloud (that is, not on a service which manages the routing for you, like Heroku) and you have many Ruby applications running on different Ruby versions or many services in many different languages *and* those services/apps need to talk to each other, quickly, it looks like NGINX Unit was designed for you. If you don't fit that profile, though, it's probably best to stick to the existing top three options (Puma, Passenger, and Unicorn). --- ## Malloc Can Double Multi-threaded Ruby Program Memory Usage URL: https://www.speedshop.co/blog/malloc-doubles-ruby-memory/ {% marginnote_lazy easy-button.jpg|Sometimes, it really is that simple.|true %} It's not every day that a simple configuration change can completely solve a problem. I had a client whose Sidekiq processes were using a lot of memory - about a gigabyte each. They would start at about 300MB each, then slowly grow over the course of several hours to almost a gigabyte, where they would start to level off. I asked him to change a single environment variable: `MALLOC_ARENA_MAX`. "Please set it to `2`." His processes restarted, and immediately the slow growth was eliminated. Processes settled at about half the memory usage they had before - around 512MB each. {% marginnote_lazy ilied.gif|Actually, it's not that simple. There are no free lunches. Though this one might be close to free. Like a ten cent lunch.|true %} Now, before you go copy-pasting this "magical" environment variable into all of your application environments, know this: there are drawbacks. You may not be suffering the problem it solves. There are no silver bullets. Ruby is not known for being a language that's light on memory use. Many Rails applications suffer from up to a gigabyte of memory use *per process*. That's approaching Java levels. [Sidekiq](https://github.com/mperham/sidekiq), the popular Ruby background job processor, has processes which can get just as large or even larger. The reasons are many, but one reason in particular is extremely difficult to diagnose and debug: fragmentation. {% marginnote_lazy log.jpeg|Typical Ruby memory growth looks logarithmic.|false %} The problem manifests itself as a slow, creeping memory growth in Ruby processes. It is often mistaken for a memory leak. However, unlike a memory leak, memory growth due to fragmentation is logarithmic, while memory leaks are linear. A memory leak in a Ruby program is usually caused by a C-extension bug. For example, if your Markdown parser leaks 10kb every time you call it, your memory growth will continue forever *at a linear rate*, since you tend to call the markdown parser at a regular frequency. Memory fragmentation causes logarithmic growth in memory. It looks like a long curve, approaching some unseen limit. All Ruby processes experience *some* memory fragmentation. It's an inevitable consequence of how Ruby manages memory. In particular, Ruby cannot *move* objects in memory. Doing so would potentially break any C language extensions which are holding raw pointers to a Ruby object. If we can't move objects in memory, fragmentation is an inevitable result. It's a fairly common issue in C programs, not just Ruby. {% marginnote_lazy malloc-arena-max.png|Actual client graph. This is what fragmentation looks like. Note the enormous drop after MALLOC_ARENA_MAX changed to 2.|false %} **However, fragmentation can sometimes cause Ruby programs to *twice* as much memory as they would otherwise, sometimes as much as four times more!** Ruby programmers aren't used to thinking about memory, especially not at the level of `malloc`. And that's OK: the entire language is designed to abstract memory away from the programmer. It's right in the manpage. But while Ruby can guarantee memory *safety*, it cannot provide perfect memory *abstraction*. One cannot be completely ignorant of memory. Because Ruby programmers are often inexperienced with how computer memory works, when problems occur, they often have no idea where to even start with debugging it, and may dismiss it as an intrinsic feature of a dynamic, interpreted language like Ruby. {% marginnote_lazy princess.jpg|"And underneath 4 layers of memory abstraction, she noticed some fragmentation!"|true %} What makes it worse is that memory is abstracted away from Rubyists through *four separate layers*. First is the Ruby virtual machine itself, which has its own internal organization and memory tracking features (sometimes called the [ObjectSpace](http://ruby-doc.org/core-2.4.0/ObjectSpace.html)). Second is the allocator, which differs *greatly* in behavior depending on the particular implementation you're using. Third is the operating system, which abstracts actual physical memory addresses away into virtual memory addresses. The way it does this varies significantly depending on the kernel - Mach does this much differently than Linux, for example. Finally, there's the actual hardware itself, which uses several strategies to keep frequently-accessed data in "hot" locations where it can be more quickly accessed. There are even special parts of the CPU involved here, such as the [translation lookaside buffer](https://en.wikipedia.org/wiki/Translation_lookaside_buffer). This is what makes memory fragmentation so difficult for Rubyists to deal with. It's a problem that generally happens at the level of the virtual machine and the allocator, parts of the Ruby language that 95% of Rubyists are probably unfamiliar with. Some fragmentation is inevitable, but it can also get so bad that it doubles the memory usage of your Ruby processes. How can you know if you're suffering the latter rather than the former? What causes critical levels of memory fragmentation? Well, I have one thesis about a cause of memory fragmentation which affects multithreaded Ruby applications, like webapps running on Puma or Passenger Enterprise, and multithreaded job processors such as Sidekiq or Sucker Punch. ## Per-Thread Memory Arenas in glibc Malloc It all boils down to a particular feature of the standard `glibc` malloc implementation called "per-thread memory arenas". To understand why, I need to explain how garbage collection works in CRuby *really quickly*. {% marginnote_lazy heapfrag.gif|ObjectSpace visualization by Aaron Patterson. Each pixel is an RVALUE. Green is "new", red is "old". See [heapfrag](https://github.com/tenderlove/heapfrag).|false %} All objects have a entry in the `ObjectSpace`. The `ObjectSpace` is a big list which contains an entry for *every* Ruby object currently alive in the process. The list entries take the form of `RVALUE`s, which are 40-byte C `struct`s that contain some basic data about the object. The exact contents of these structs varies depending on the class of the object. As an example, if it is a very short String like "hello", the actual bits that contain the character data are embedded directly in the `RVALUE`. However, we only have 40 bytes - if the string is 23 bytes or longer, the `RVALUE` contains only a raw pointer to where the object data *actually* lies in memory, outside the `RVALUE`. `RVALUE`s are further organized in the `ObjectSpace` into 16KB "pages". Each page contains about 408 `RVALUE`s. These numbers can be confirmed by looking at the `GC::INTERNAL_CONSTANTS` constant in any Ruby process: ```ruby GC::INTERNAL_CONSTANTS => { :RVALUE_SIZE=>40, :HEAP_PAGE_OBJ_LIMIT=>408, # ... } ``` Creating a long string (let's say it's a 1000-character HTTP response for example) looks like this: 1. Add an `RVALUE` to the `ObjectSpace` list. If we are out of free slots in the `ObjectSpace`, we lengthen the list by 1 heap page, calling `malloc(16384)`. 2. Call `malloc(1000)` and receive a address to a 1000-byte memory location.{% sidenote 1 "Actually, Ruby will request an area slightly larger than it needs in case the string is added to or resized." %} This is where we'll put our HTTP response. The malloc calls here are what I want to bring your attention to. All we're doing is asking for a memory location of a particular size, *somewhere*. **Actually, `malloc`'s contiguity is *undefined***, that is, it makes no guarantees about *where* that memory location will actually be. This means that, from the perspective of the Ruby VM, fragmentation (which is fundamentally a problem about *where* memory is) is a problem of the allocator.{% sidenote 2 "However, allocation patterns and sizes can definitely make things harder for the allocator." %} Ruby can, in a way, measure the fragmentation of its own `ObjectSpace`. A method in the `GC` module, `GC.stat`, provides a wealth of information about the current memory and GC state. It's a little overwhelming and is under-documented, but the output is a hash that looks like this: ```ruby GC.stat => { :count=>12, :heap_allocated_pages=>91, :heap_sorted_length=>91, # ... way more keys ... } ``` There are two keys in this hash that I want to point your attention to: `GC.stat[:heap_live_slots]` and `GC.stat[:heap_eden_pages]`. `:heap_live_slots` refers to the number of slots in the `ObjectSpace` currently occupied by live (not marked for freeing) `RVALUE` structs. This is roughly the same as "currently live Ruby objects". {% marginnote_lazy eden.jpg|The Eden heap|true %} `:heap_eden_pages` is the number of `ObjectSpace` pages which currently contain *at least one* live slot. `ObjectSpace` pages which have at least one live slot are called eden pages. `ObjectSpace` pages which contain no live objects are called tomb pages. This distinction is important from the GC's perspective, because tomb pages can be returned back to the operating system. Also, the GC will put new objects into eden pages first, and then tomb pages after all the eden pages have filled up. This reduces fragmentation. If you divide the number of live slots by the number of slots in all eden pages, you get a measure of the current fragmentation of the ObjectSpace. As an example, here's what I get in a fresh `irb` process: ```ruby 5.times { GC.start } GC.stat[:heap_live_slots] # 24508 GC.stat[:heap_eden_pages] # 83 GC::INTERNAL_CONSTANTS[:HEAP_PAGE_OBJ_LIMIT] # 408 # live_slots / (eden_pages * slots_per_page) # 24508 / (83 * 408) = 72.3% ``` About 28% of my eden page slots are currently unoccupied. A high percentage of free slots indicates that the ObjectSpace's RVALUEs are spread across many more heap pages than they would be if we could move them around. This is a kind of internal memory fragmentation. Another measure of internal fragmentation in the Ruby VM comes from `GC.stat[:heap_sorted_length]`. This key is the "length" of the heap. If we have three ObjectSpace pages, and I `free` the 2nd one (the one in the middle), I only have two heap pages remaining. However, I cannot move heap pages around in memory, so the "length" of the heap (essentially the highest index of the heap pages) is still 3. {% marginnote_lazy swisscheese.jpg|Yes, this heap is fragmented, but it looks *really tasty*.|true %} Dividing `GC.stat[:heap_eden_pages]` by `GC.stat[:heap_sorted_length]` gives a measure of internal fragmentation at the level of ObjectSpace pages - a low percentage here would indicate a lot of heap-page-sized "holes" in the ObjectSpace list. While these measures are interesting, most memory fragmentation (and most allocation) doesn't happen in the `ObjectSpace` - it happens in the process of allocating space for objects which don't fit inside a single `RVALUE`. It turns out that's most of them, according to experiments performed by Aaron Patterson and Sam Saffron. A typical Rails app's memory usage will be 50%-80% in these `malloc` calls to get space for objects larger than a few bytes. When Aaron says "managed by the GC" here, he means "inside the `ObjectSpace` list". Ok, so let's talk about where per-thread memory arenas come in. The per-thread memory arena was an optimization introduced in `glibc` 2.10, [and lives today in `arena.c`](https://github.molgen.mpg.de/git-mirror/glibc/blob/master/malloc/arena.c). It's designed to decrease contention between threads when accessing memory. In a naive, basic allocator design, the allocator makes sure only one thread can request a memory chunk from the main arena at a time. This ensures that two threads don't accidentally get the same chunk of memory. If they did, that would cause some pretty nasty multi-threading bugs. However, for programs with a lot of threads, this can be slow, since there's a lot of contention for the lock. *All* memory access for *all* threads is gated through this lock, so you can see how this could be a bottleneck. Removing this lock has been an area of major effort in allocator design because of its performance impact. There are even a few lockless allocators out there. The per-thread memory arena implementation alleviates lock contention with the following process (paraphrased from [this article by Siddhesh Poyarekar](https://siddhesh.in/posts/malloc-per-thread-arenas-in-glibc.html)): 1. We call `malloc` in a thread. The thread attempts to obtain the lock for the memory arena it accessed previously (or the main arena, if no other arenas have been created). 2. If that arena is not available, try the next memory arena (if there are any other memory arenas). 3. If none of the memory arenas are available, create a new arena and use that. This new arena is linked to to the last arena in a linked list. In this way, the main arena is basically extended into a linked list of arenas/heaps. The number of arenas is limited by `mallopt`, specifically the `M_ARENA_MAX` parameter (documented [here](http://man7.org/linux/man-pages/man3/mallopt.3.html), note the "environment variables" section). By default, the limit on the number of per-thread memory arenas that can be created is 8 times the number of available cores. Most Ruby web applications run about 5 threads per core, and Sidekiq clusters can often run far more than that. In practice, this means that many, many per-thread memory arenas can get created by a Ruby application. Let's take a look at exactly how this would play out in a multithreaded Ruby application. 1. You are running a Sidekiq process with the default setting of 25 threads. 2. Sidekiq begins running 5 new jobs. Their job is to communicate with an external credit card processor - so they POST a request via HTTPS and receive a response ~3 seconds later. 3. Each job (which is running a separate thread in Rubyland) sends an HTTP request and waits for a response using the `IO` module. Generally, almost all IO in CRuby releases the Global VM lock, which means that these threads are working *in parallel* and may contend for the main memory arena lock, causing the creation of new memory arenas. If multiple CRuby threads are running but *not* doing I/O, it is pretty much impossible for them to contend for the main memory arena because the Global VM Lock prevents two Ruby threads from executing Ruby code at the same time. Thus, per-thread-memory arenas only affect CRuby applications which are both multithreaded and performing I/O. How does this lead to memory fragmentation? {% marginnote_lazy tetris.jpg|Bin-packing can be fun, too!|true %} Memory fragmentation is essentially a [bin packing problem](https://en.wikipedia.org/wiki/Bin_packing_problem) - how can we efficiently distribute oddly-sized items between multiple bins so that they take up the least amount of space? Bin-packing is made much more difficult for the allocator because a) Ruby never moves memory locations around (once we allocate a location, the object/data stays there until it is freed) b) per-thread memory arenas essentially create a *lot* of different bins, which cannot be combined or "packed" together. Bin-packing is already NP-hard, and these constraints just make it even more difficult to achieve an optimal solution. Per-thread memory arenas leading to large amounts of RSS use over time is something of a [known issue on the glibc malloc tracker](https://sourceware.org/bugzilla/show_bug.cgi?id=11261). In fact, the [MallocInternals wiki](https://sourceware.org/glibc/wiki/MallocInternals) says specifically: > As pressure from thread collisions increases, additional arenas are created via mmap to relieve the pressure. The number of arenas is capped at eight times the number of CPUs in the system (unless the user specifies otherwise, see mallopt), which means a heavily threaded application will still see some contention, but the trade-off is that there will be less fragmentation. There you have it - lowering the number of available memory arenas reduces fragmentation. There's an explicit tradeoff here: fewer arenas decreases memory use, but may slow the program down by increasing lock contention. Heroku discovered this side-effect of per-thread memory arenas when they created the Cedar-14 stack, which upgraded glibc to version 2.19. [Heroku customers reported greater memory consumption of their applications when upgrading their apps to the new stack.](https://devcenter.heroku.com/articles/tuning-glibc-memory-behavior) Testing by Terrence Hone of Heroku produced some interesting results: | Configuration | Memory Use | | -------- | -------- | | Base (unlimited arenas) | 1.73x | | Base (before arenas introduced) | 1x | | MALLOC_ARENA_MAX=1 | 0.86 | | MALLOC_ARENA_MAX=2 | 0.87 | Basically, the default memory arena behavior in libc 2.19 reduced execution time by 10%, but increased memory use by 75%! Reducing the maximum number of memory arenas to 2 essentially eliminated the speed gains, but reduced memory usage over the old Cedar-10 stack by 10% (and reduced memory usage by about 2X over the default memory arena behavior!). | Configuration | Response Times | | -------- | -------- | | Base (unlimited arenas) | 0.9x | | Base (before arenas introduced) | 1x | | MALLOC_ARENA_MAX=1 | 1.15x | | MALLOC_ARENA_MAX=2 | 1.03x | For almost *all* Ruby applications, a 75% memory gain for 10% speed gain is *not* an appropriate tradeoff. But let's get some more real-world results in here. ## A Replicating Program {% marginnote_lazy 2arenas.jpg||false %} I wrote [a demo application](https://github.com/speedshop/sidekiqdemo), which is a Sidekiq job which generates some random data and writes the response to a database. After switching `MALLOC_ARENA_MAX` to 2, memory usage was 15% lower after 24 hours. I've noticed that real-world workloads magnify this effect greatly, which means I don't fully understand the allocation pattern which can cause this fragmentation yet. I've seen plenty of memory graphs on the [Complete Guide to Rails Performance](https://www.railsspeed.com/) Slack channel that show 2-3x memory savings in production with `MALLOC_ARENA_MAX=2`. ## Fixing the Problem There are two main solutions for this problem, along with one possible solution for the future. ### Fix 1: Reduce Memory Arenas One fairly obvious fix would be to reduce the maximum number of memory arenas available. We can do this by changing the `MALLOC_ARENA_MAX` environment variable. As mentioned before, this increases lock contention in the allocator and *will* have a negative impact on the performance of your application across the board. It's impossible to recommend a generic setting here, but it seems like 2 to 4 arenas is appropriate for most Ruby applications. Setting `MALLOC_ARENA_MAX` to 1 seems to have a high negative impact on performance with only a very marginal improvement to memory usage (1-2%). Experiment with these settings and *measure the results* both in memory use reduction and performance reduction until you've made a tradeoff appropriate for your app. ### Fix 2: Use `jemalloc` Another possible solution is to simply use a different allocator. `jemalloc` also implements per-thread arenas, but their design seems to avoid the fragmentation issues present in `malloc`. The above tweet was from when I removed jemalloc from [CodeTriage](https://www.codetriage.com/)'s background job processes. As you can see, the effect was pretty drastic. I also experimented with using `malloc` with `MALLOC_ARENA_MAX=2`, but memory usage was still almost *4 times* greater than memory usage with `jemalloc`. **If you can switch to jemalloc with Ruby, do it.** It seems to have the same or better performance than `malloc` with far less memory use. This isn't a `jemalloc` blog post, but some finer points on using `jemalloc` with Ruby: * [You can use it on Heroku with this buildpack.](https://github.com/mojodna/heroku-buildpack-jemalloc) * Do not use `jemalloc` 4.x with Ruby. It has a bad interaction with Transparent Huge Pages that reduces the memory savings you'll see. Instead, use `jemalloc` 3.6. 5.0's performance with Ruby is currently unknown. * You do not need to compile Ruby with jemalloc (though you can). [You can dynamically load it with LD_PRELOAD.](https://github.com/jemalloc/jemalloc/wiki/Getting-Started) ### Fix 3: Compacting GC Fragmentation can generally be reduced if one can *move* locations in memory around. We can't do that in CRuby because C-extensions may use raw pointers to refer to Ruby's memory - moving that location would cause a segfault or incorrect data to be read. [Aaron Patterson has been working on a compacting garbage collector for a while now.](https://www.youtube.com/watch?v=8Q7M513vewk) The work looks promising, but perhaps a ways off in the future. ## TL;DR: Multithreaded Ruby programs may be consuming 2 to 4 times the amount of memory that they really need, due to fragmentation caused by per-thread memory arenas in `malloc`. To fix this, you can reduce the maximum number of arenas by setting the `MALLOC_ARENA_MAX` environment variable or by switching to an allocator with better performance, such as `jemalloc`. The potential memory savings here are so great and the penalties so minor that **I would recommend that if you are using Ruby and Puma or Sidekiq in production, you should always use `jemalloc`**. While this effect is most pronounced in CRuby, [it may also affect the JVM and JRuby.](https://github.com/cloudfoundry/java-buildpack/issues/320) --- ## Configuring Puma, Unicorn and Passenger for Maximum Efficiency URL: https://www.speedshop.co/blog/appserver/ {% marginnote_lazy unicorn_car.jpg||true %} In Ruby, web application servers are like gasoline in a car: the fancy stuff won't make your car go any faster, but the nasty stuff will bring you grinding to a halt. Application servers can't actually make your app significantly *faster* - no, they're all pretty much the same and changing from one to the other won't improve your throughput or response times by much. But it *is* easy to shoot yourself in the foot with a bad setting or misconfigured server. It's one of the most common problems I see on client applications. This post will be about optimizing resource usage (memory and CPU) and maximizing throughput (that is, requests-per-second) from the three major Ruby application servers: Puma, Unicorn and Passenger. I'm going to use the terms "server" and "container" interchangeably, because nothing here is specific to a virtualized environment. I can cover all three of the popular application servers in a single guide because they all use, fundamentally, the same design. With the `fork` system call, these application servers create several child processes, which then do the job of serving requests. {% sidenote 1 "In all three app servers, the 'master' process that creates the child processes does not actually answer any requests. Passenger will actually shut down the 'master' preload process after a while if you haven't forked recently." %} Most of the differences between these servers lie in the finer details (which I'll also cover here, where important for maximum performance). Throughout this guide, we're going to try to maximize our throughput-per-server-dollar. We want to serve the most number of requests per second for the lowest amount of server resources (and therefore, cash). ## The most important configuration settings for performance {% marginnote_lazy dyno.jpg|Timeouts are fairly important too, but they're not really throughput-related. I'll leave them for another day.|true %} There are 4 fundamental settings on your application server that determine its performance and resource consumption: * Number of child processes. * Number of threads. * Copy-on-write. * Container size. Let's go through each in turn. ### Child process count Unicorn, Puma and Passenger all use a `fork`ing design.{% sidenote 2 "JRuby people can probably skip to the next section." %} This means that they create one application process and call `fork` a number of times to create copies of that application process. We call these copies child processes. The number of child processes we have on each server is probably the most important setting for maximizing throughput-per-server-dollar. {% sidenote 3 "This is because of CRuby's Global VM Lock. Only one thread can execute Ruby code at a time, so the only way to achieve parallel Ruby work is to run multiple processes." %} We want to run *as many processes per server as possible* without exceeding the resources of the server. **I recommend that all Ruby webapps run at least 3 processes per server or container**. This maximizes routing performance. Puma and Unicorn both use a design where the child processes listen directly on a single socket, and then let the operating system balance load between the processes. Passenger uses a reverse proxy (nginx or Apache) to route requests to a child process.{% sidenote 5 "Passenger's [least-busy-process-first](https://www.phusionpassenger.com/library/indepth/ruby/request_load_balancing.html) routing is actually one of my favorite features of theirs." %} Both approaches are pretty efficient and mean that a request will be quickly routed to a worker that is idle. Routing at higher layers (that is, at the load balancer or Heroku's HTTP mesh) is far more difficult to do so efficiently, because the load balancer usually has no idea whether or not the servers its routing to are busy or not.{% sidenote 6 "For one client I had, moving from 30 servers with 2 processes each to 3 servers with 20 processes each almost *completely* eliminated the timeout errors they were having (which were being caused by fast requests piling up behind slow ones)." %} Consider a setup with 3 servers, each running 1 processes (so a total of 3 processes). How does the load balancer optimally route a request to one of the three servers? It could do so randomly or in a round-robin fashion, but this does not guarantee that the request will be routed to a server with an idle, waiting process. For example, with a round-robin strategy, let's say Request A is routed to server #1. Request B is then routed to server #2, and Request C to server #3. {% marginnote_lazy unicornhead.jpg|My face when you give me a request but all my children are busy.|true %} Now here comes a fourth request, Request D. What happens if Request B and C have already been successfully served and those servers (2 and 3) are idle, but Request A was somebody's CSV export and will take 20 seconds to complete? The load balancer will continue to give requests to server #1 even though its busy and won't process them until it's done with Request A. All load balancers have ways of knowing if a server is *completely* dead, but most of these methods have a long lag time (i.e. 30 seconds or more of delay). Running higher numbers of processes per server insulates us from the risk of long-lived requests "hogging" the majority of a server's child processes, because at the *server* level, requests will *never* be given to an already-busy process. Instead, they'll back up at the socket level or the reverse proxy until a worker is free. From experience, I find that 3 processes per server is a good minimum to achieve this. If you can't run at least 3 processes per server due to resource constraints, get a bigger server (more on that later). So, we should run at least 3 child processes per container. But what's the maximum? That's constrained by our memory and CPU resources. Let's start with memory. Each child process uses a certain amount of memory. Obviously, we shouldn't add more child processes than our server's RAM can support! {% marginnote_lazy log.jpeg|Actual memory usage of Ruby processes is logarithmic. Due to memory fragmentation, memory usually doesnt level off, but only approaches a limit.|true %} Measuring the actual memory usage of a single Ruby application process can be tricky, however. It's not enough to just start up a process on your computer or production environment and check the number right away. {% marginnote_lazy puma_bloat.png|After a while, Puma workers can get rather... large.|true %} For a number of reasons, **Ruby web application processes increase in memory usage over time**, even as much as doubling or tripling their memory usage from when they are spawned. To get an accurate measurement of how much memory your Ruby application processes are using, *disable all process restarts* (worker killers) and wait 12-24 hours to take a measurement with `ps`. If you're on Heroku, you can use the new [Heroku Exec](https://devcenter.heroku.com/articles/exec) to use `ps` on a running dyno, or simply divide Heroku's memory usage metric by the number of processes you are running per dyno. Most Ruby applications will use between 200 and 400 MB per process, but some can use as much as 1GB. {% marginnote_lazy david_meme.jpg|1 upvote = 1 prayer|true %} Be sure to give yourself some headroom on the memory number - if you want an equation, set your child process count to something like (`TOTAL_RAM` / (`RAM_PER_PROCESS` * 1.2)) Exceeding the available memory capacity of a server/container can cause major slowdowns as memory is overcommitted and swapping starts to occur. This is why you want your application's memory usage to be predictable and consistent with no sudden spikes. Sudden increases in memory usage are a condition I call *memory bloat*. Solving this is a topic for another day or post, but the topic is covered in [The Complete Guide to Rails Performance](http://www.railsspeed.com) Second, we don't want to exceed the available CPU capacity of our server. Ideally, we don't spend more than 5% of our total deployed time at 100% CPU usage - more than that means that we're being bottlenecked by the available CPU capacity. Most Ruby and Rails applications tend to be memory-bottlenecked on most cloud providers, but sometimes CPU can be the bottlenecking resource too. How do you know? Just use your favorite server monitoring tool - AWS's built in tools are probably good enough for figuring out if CPU usage is frequently maxing out. {% marginnote_lazy thatwasalie.jpg|You said that OS context switching was expensive. Actual production use determined that was a lie.|true %} It's frequently said that you shouldn't have more child processes per server than CPUs. This is only *partly* true. It's a good starting point, but actual CPU usage is the metric you should watch and optimize. In practice, most applications will probably settle at a process count that is 1.25-1.5x the number of available hyperthreads. On Heroku, use [log-runtime-metrics](https://devcenter.heroku.com/articles/log-runtime-metrics) to get a CPU load metric written to your logs. I would look at the 5 and 15 minute load averages - if they are consistently close to or higher than 1, you are maxing out CPU and need to reduce child process counts. Setting child process counts is pretty easy in every application server: ```ruby # Puma $ puma -w 3 # Command-line option workers 3 # in your config/puma.rb # Unicorn worker_processes 3 # config/unicorn.rb # Passenger (nginx/Standalone) # Passenger can automatically scale workers up and down - I don't find this # super useful. Instead, just run a constant number by setting the max and min: passenger_max_pool_size 3; passenger_min_instances 3; ``` Instead of setting this to a hard number, you may want to set it to an environment variable such as `WEB_CONCURRENCY`: ```ruby workers Integer(ENV["WEB_CONCURRENCY"] || 3) ``` In summary, most applications will want to use 3-8 processes per server, depending on available resources. Highly memory-constrained applications or apps which have high 95th percentile times (5-10 seconds or more) may want to run higher numbers, up to 4x the available hyperthread count. Most app's child process counts should not exceed 1.5x the amount of available hyperthreads. ### Thread count Puma and Passenger Enterprise support multi-threading your application, so this discussion is aimed at those servers. Threads can be a resource-light way of improving your application's concurrency (and, therefore, throughput). Rails is already threadsafe, and most applications aren't doing weird things like creating their own threads or using globals to access shared resource, like database connections (looking at you, `$redis`!) So, *most* Ruby web-applications are thread-safe. The only *real* way to know is to actually give it a shot. Ruby applications tend to surface threading bugs in loud, exception-raising ways, so it's easy to give it a shot and see the results. {% marginnote_lazy amdahl.png||false %} So how many threads should we use? The speedup you can gain from additional parallelism depends on the *portion of your program's execution which can be done in parallel*. [This is known as Amdahl's Law](https://en.wikipedia.org/wiki/Amdahl%27s_law). In MRI/C Ruby, we can only parallelize waiting on IO (waiting on a database result, for example). For *most* web applications, this is probably 10-25% of their total time. You can check for your own application by looking at the amount of time you spend "in the database" per request. Unfortuantely, what Amdahl's law reveals is that for programs that have small parallel portions (less than 50%), there is little to no benefit past a handful of threads. This matches my own experience: on client applications, thread settings of more than 5 have no effect. [Noah Gibbs also tested this against the Discourse homepage benchmark](https://appfolio-engineering.squarespace.com/appfolio-engineering/2017/1/31/the-benchmark-and-the-rails) and settled on a thread count of 6. {% marginnote_lazy setit.jpg||true %} Unlike process count, where I advise you to constantly check the metrics against your settings and tune appropriately, with threads, it's usually OK to just "set it and forget it" to 5 threads per application server process. In MRI/C Ruby, threads can have a surprisingly large memory impact. This is due to a host of complicated reasons (which I'll probably get into in a future post). Be sure to check memory consumption before and after adding threads to the application. Do *not* expect that each thread will only consume an additional 8MB of stack space, they will often increase total memory usage by *far* more than that. Here's how to set thread counts: ```ruby # Puma. Again, I don't really use the "automatic" spin-up/spin-down features, so # I set the max and min to the same number. $ puma -t 5:5 # Command-line option threads 5, 5 # in your config/puma.rb # Passenger (nginx/Standalone) passenger_concurrency_model thread; passenger_thread_count 5; ``` For JRuby people - threads are fully parallelizable, so you can take the full benefit of Amdahl's law here. Setting thread counts for you will be more like setting process counts under MRI (described above) - increase them until you run out of memory or CPU resources. ### Copy-on-write behavior All Unix-based operating systems implement copy-on-write memory behavior. It's pretty simple: when a process `fork`s and creates a child, that child process' memory is *shared*, completely, with the parent process. All memory reads from the child process will simply read from the parent's memory. However, modifying a memory location creates a copy, solely for the private use of the child process. It's extremely useful for reducing the memory usage of forking webservers, since child processes should, in theory, be able to share things like shared libraries and other "read-only" memory with the parent, rather than creating their own copy. Copy-on-write *just happens*. {% sidenote 6 "You can't really 'support' copy-on-write so much as just 'make it more effective at saving you memory'." %} It can't be "turned off", but you can make it more effective. Basically, we want to load all of our application *before* forking. Most Ruby webapp servers call this "preloading". All it does is change *when* `fork` is called - before or after your application is initialized. You'll also need to re-connect to any databases you're using after forking. For example, with ActiveRecord: ```ruby # Puma preload_app! on_worker_boot do # Valid on Rails 4.1+ using the `config/database.yml` method of setting `pool` size ActiveRecord::Base.establish_connection end # Unicorn preload_app true after_fork do |server, worker| ActiveRecord::Base.establish_connection end # Passenger uses preloading by default, so no need to turn it on. # Passenger automatically establishes connections to ActiveRecord, # but for other DBs, you will have to: PhusionPassenger.on_event(:starting_worker_process) do |forked| if forked reestablish_connection_to_database # depends on the DB end end ``` In theory, you have to do this for every database your application uses. However, in practice, Sidekiq doesn't try to connect to Redis until you actually try to do something, so unless you're running Sidekiq jobs during application boot, you don't have to reconnect after fork. Unfortunately, there are limits to the gains of copy-on-write. Transparent Huge Pages can cause even a 1-bit memory modification to copy an entire 2MB page, and [fragmentation can also limit savings](https://brandur.org/ruby-memory). But it doesn't hurt, so turn on preloading anyway. ### Container size {% marginnote_lazy hungry.gif|Gimme some of that memory, boi|true %} In general, we want to make sure we're utilizing 70-80% of our server's available CPU and memory. These needs will differ between applications, and the ratio between CPU cores and GB of memory will differ in turn. One application might be happiest on a 4 vCPU / 4 GB of RAM server with 6 Ruby processes, while another less-memory-hungry and more CPU-heavy application might do well with 8 vCPUs and 2GB of RAM. There's no one perfect container size, but the ratio between CPU and memory should be chosen based on your actual production metrics. {% marginnote_lazy spicywinner.jpg||true %} The **amount of memory available to our server or container** is probably one of the most important resources we can tune. On many providers, this number is exceedingly low - 512MB on the standard Heroku dyno, for example. Ruby applications, especially sufficiently complex and mature ones, are memory hungry, and the amount of memory we have to work with is probably one of our most important resources. Because most Rails applications use ~300MB of RAM and I think everyone should be running at least 3 processes per server, most Rails applications will need a server with at least 1 GB of RAM. **Our server's CPU resources** are another important lever we can tune. We need to know how many CPU cores are available to us, and how many threads we can execute at a single time (basically, does this server support Hyper-Threading or not?). As I mentioned in the discussion of child process counts, **your container should support at least 3 child processes**. Even better would be 8 or more processes per server/container. Higher process counts per container improves request routing and decreases latency. ## TL;DR This was an overview of how to best maximize the throughput of your Ruby web application servers. In a short, list format, here's the steps: 1. **Figure out how much memory 1 worker with 5 threads uses.** If you're using Unicorn, obviously no threads required. Run just a few workers on a single server under production load for at least 12 hours without restarting. Use `ps` to get the memory usage of a typical worker. 2. **Choose a container size with memory equal to at least 3X that number**. Most Rails applications will use ~300-400MB of RAM per worker. So, most Rails apps will need at least 1 GB container/server. This gives us enough memory headroom to run at least 3 processes per server. You can run a number of child processes equal to (`TOTAL_RAM` / (`RAM_PER_PROCESS` * 1.2)). 3. **Check CPU core/hyperthread counts** If your container has *fewer* hyperthreads (vCPUs on AWS) than your memory can support, you can either choose a container size with less memory or more CPU. Ideally, the number of child processes you run should equal 1.25-1.5x the number of hyperthreads. 4. **Deploy and watch CPU and memory consumption**. Tune child process count and container size as appropriate to maximize usage. --- ## Is Ruby Too Slow For Web-Scale? URL: https://www.speedshop.co/blog/is-ruby-too-slow-for-web-scale/ {% sidenote 1 "Okay, okay, I know. [Betteridge's Law of Headlines](https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines). Of course Ruby and Rails are fast enough for big websites - Shopify makes it work and they're one of the largest in the world. But some people *genuinely do seem to think* that Rails 'isn't fast enough'. That's what this article is about." %} How does one choose a framework or programming language for a new web application? You almost certainly need one, unless you're doing something pretty trivial. All web applications have a lot of boilerplate they need to get running: security, object-relational mapping, templating and testing. So how do you know which one to choose? {% marginnote_lazy Tiny-trains-on-track.jpg|This is what Rails is, right?|true %} Well, you certainly don't want to pick a *slow* framework, do you? That wouldn't be good - we want a *fast*, *modern*, and *lightweight* web framework, not some *heavy*, *old*, *slow*, web framework. Heavy, old, and slow...like Ruby on Rails, right? Ruby on Rails, the king of the all-in-one web framework space for the last 10 years, is constantly under assault by faster, nippier, lighter competitors. Is Rails a dinosaur that can no longer compete? Well, we could look at some benchmarks to find out. Surely a "fast" and "lightweight" framework would do well on a benchmark, while a old, busted framework would do poorly. {% marginnote_lazy rails-sucks.png|Yeah! Rails sucks!|true %} You would be forgiven for thinking that Ruby on Rails was somehow irretrievably graveyard-bound if you looked at the benchmarks posted by sites such as [TechEmpower](https://www.techempower.com/benchmarks/). Sequel author Jeremy Evans recently pointed out that even [other Ruby frameworks can bury Rails](https://twitter.com/jeremyevans0/status/864212426618675200) in these comparisons. You look at those benchmarks at think: "Wow, Sequel is *ten times faster* than ActiveRecord and Rails!" And in a narrow sense, you'd be right. Benchmarks are like statistics - it's easy to give the right answer to the wrong question, and allow the reader to draw a conclusion which isn't supported by the data. If you looked at those benchmarks and thought: "If I take my Rails application and rewrite it in Sequel and Sinatra, it will be ten times faster than it is now!", you would be wrong. And, even if it *was* faster, would it matter? **Is there such a thing as a *fast enough* web application?** Just how important is performance when choosing a web framework or even a programming language for a web application? ## Latency and Throughput Let's start with some definitions. {% marginnote_lazy funnel.jpg|Servers are like funnels: latency is how long it takes one molecule of water to pass through the funnel, throughput is how much water passes throught the funnel every second. A high-latency high-throughput server would be something like a long, wide tube, and a low-latency low-throughput server would look like a short, wide disc with a narrow opening.|true %} In server application design, *latency* and *throughput* are king. Latency is the amount of time it takes our server to respond to a single request. *Throughput* is how many requests we can serve at the same time, usually measured in a unit like responses/second. *Throughput* of a web application is generally governed by CPU and parallelism - how many CPU cycles does it take to respond to a web request, and how efficiently can you saturate all the CPU cores of the host machine? The amount of CPU cycles is governed by the application's domain, framework, and language - complicated apps take more time, and dynamic languages like Ruby generate more CPU instructions than compiled languages like C or Rust. Efficiently using all the available CPU resources varies depending on the language - Go's goroutines, Elixir's "processes", multi-process servers to get around global VM locks like Python and Ruby, event-driven architectures like Node, or true threading like Java. *Latency*, however, is even more important. This is because *latency is inversely proportional to throughput*. If we halve the latency of our web application, we double its maximum throughput. Latency also affects the end-user experience - a 500 millisecond response time manifests as an extra 500 milliseconds the user must spend waiting for the webpage to load. ## Benchmark Trip-Ups {% marginnote_lazy topfuel.jpg|TechEmpower's servers|true %} Let's take a look at the [TechEmpower web framework benchmarks](https://www.techempower.com/benchmarks/). TechEmpower measures latency and maximum throughput across six synthetic benches. These benchmarks are run on pretty fat servers - they've got 4 cpus with 10 cores and 20 threads *each* (so, 40 cores and 80 hyperthreads in total). Oh yeah, and 528 GB of RAM. {% marginnote_lazy multiquery.png|Rails implementation of the benchmark|true %} One of the more relevant benchmarks is the multiple-queries benchmark. It's pretty simple - it executes 20 queries, sequentially, against a SQL database, and then returns the result. This is a pretty common web application workload - most Rails applications I've worked on roughly look like this. As we render the template, we execute a few SQL queries to get the results to populate the template, and return it. In Round 14, the typical Rails setup (puma-mri-rails) clocks in at a measly ~531 requests per second. [Roda](https://github.com/jeremyevans/roda), an *extreme* lightweight Ruby web framework, when used with Sequel, clocks in at about 7000 requests/second, depending on the webserver used. So does that mean Rails is more than 10 times slower than Roda and Sequel? On an 80 core machine, is 531 requests/second really all you can get out of Ruby on Rails? TechEmpower's Rails setup is unbelievably crippled compared to their Roda setup. [Their Puma server is configured to run just 8 processes](https://github.com/TechEmpower/FrameworkBenchmarks/blob/e784c36f255b318611d3a0a2c91ad57255eb19d5/frameworks/Ruby/rails/run_mri_puma.sh#L7), while [Roda auto-tunes itself](https://github.com/TechEmpower/FrameworkBenchmarks/blob/e784c36f255b318611d3a0a2c91ad57255eb19d5/frameworks/Ruby/roda-sequel/config/mri_puma.rb), ending up with around 100 processes. So the Rails benchmark is using, at best, about 15-20% of the available hyperthreads, while the Roda benchmark is using all of them. So that's *at least* a 5-8x throughput penalty for the Rails benchmark *out of the gate*. But that's fixable - TechEmpower is open source and [we can just open a pull request and fix this](https://github.com/TechEmpower/FrameworkBenchmarks/pull/2850), and we'll get better results for Round 15. Let's take a look at another TechEmpower measurement - average request latency. Focusing on request latency allows to put all languages and frameworks on a somewhat more even footing, because things like global VM locks and other concurrency features usually don't really matter when processing a single request.{% sidenote 2 "Concurrency features generally increase throughput, not decrease latency." %} On the multiple-query database test, Puma and Rails clock in at 129 milliseconds. The Roda/Sequel/Puma stack clocks in at 31.3 milliseconds. Now, as I said, the Puma settings for Rails on TechEmpower are incredibly crippled compared to the Roda settings, so Rails could probably still shave a lot off of that time, but let's take it as it is. Let's just say Rails adds **one hundred milliseconds** of latency to the average web application response over a microframework or other competing platform like Phoenix. (Actually, Phoenix is slower on this test than Rails. [The framework creators dispute this result though](https://www.reddit.com/r/elixir/comments/48ke69/any_reason_why_elixirphoenix_did_so_badly_in/), and I don't doubt it if the Rails benchmark is this gimped too). ## The Computer Changes, But The Human Does Not {% marginnote_lazy room-sized-computer.jpg||false %} The funny thing about computers is that although they keep getting faster, squishy human beings stay the same speed. Just *how fast* a human-computer interaction has to be has been studied since the 1960s. You can understand their interest in this, back in the times when computers were the size of rooms and computations took hours rather than microseconds. If the computer was going to move out of the mainframe and the science lab and into public life, it was going to have to be faster. But *how much* faster? [Jakob Nielsen summarized the results in 1993:](https://www.nngroup.com/articles/response-times-3-important-limits/) {% marginnote_lazy jakob_mouse_big.jpg|Jakob Nielsen. I am glad that this photo exists.|false %} > 0.1 second: Limit for users feeling that they are directly manipulating objects in the UI. (...) > > 1 second: Limit for users feeling that they are freely navigating the command space without having to unduly wait for the computer. (...) > > 10 seconds: Limit for users keeping their attention on the task. (...) You can read his full article on the topic [here](https://www.nngroup.com/articles/response-times-3-important-limits/). ### On the web, how fast is fast enough? Let's assume that all our little web application does is return an HTML response with *no* JavaScript or CSS. It's just a flat, HTML document with the default browser styling.{% sidenote 3 "Imagine if you would, for a moment, a website whose styling is even more boring than this one." %} How long would it take for a user to visit `www.oursite.com` and receive a response? Well, if our user is on a desktop computer in the same country as our servers, it will take about 20 milliseconds for their packets to get from their computer to our servers, and another 20 milliseconds back. This is a *best case scenario*: if they're on the other side of the world, this could easily be 100 milliseconds each way. If they're on a mobile cellular connection, we're talking ~300-400 milliseconds. My home DSL connection fluctuates from 50-150 milliseconds to most US servers. {% marginnote_lazy brentrambo.gif|150 milliseconds time-to-first-byte? That's Brent Rambo Approved.|false %} So, if we've already got ~40 milliseconds of round-trip network latency in the first place, will our users be able to perceive the difference in a web application which renders a response in 1 millisecond or 100 milliseconds? That is, one application will take 41 milliseconds in total and the other 141. The answer **is emphatically no**. Both applications will appear almost instantaneous to the user. And in the worst cases of network conditions, the difference will completely vanish. So minor latency differences (100 milliseconds or less, as in the difference between web frameworks) only matter in their contribution to improving throughput. ### Your Server is Just a Small Part of the User Experience {% marginnote_lazy modern-web.png|WELCOME TO THE MODERN WEB, BITCH.|false %} It's 2017 and web applications don't return flat HTML files anymore. Websites are gargantuan, with JavaScript bundles stretching into the size of megabytes and stylesheets that couldn't fit in ten Apollo Guidance Computers. So how much of a difference does a web application which responds in 1 millisecond or less make in this environment? Vanishingly little. Nowadays, the average webpage takes 5 seconds to render. Some JavaScript single-page-applications can take 12 seconds or more on initial render. Server response times simply make up a teeny-tiny part of the actual user experience of loading and interacting with a webpage - cutting 99 milliseconds off the server response time just doesn't make a difference. ### There's a Ceiling: Web Apps Aren't Video Games In the video gaming world, speed matters. Faster languages can mean more polygons on the screen per frame. There's really no upper limit for this - more polygons will always be good, so a faster language will always help with increasing the fidelity of the simulation. {% marginnote_lazy sortafast.gif|[for those unfamiliar with the meme](https://www.youtube.com/watch?v=hU7EHKFNMQg)|false %} Web applications are not like this. Fundamentally, 90% of them are simple CRUD applications. A faster language does not open more possibilities for functionality or features, it just takes the same HTML webform we've been rendering and renders it a few milliseconds faster. There's a *ceiling* on the usefulness of reduced request latency. ### Ruby is Slow, so More Ruby is Slower {% marginnote_lazy hashtables.png|[Mike Perham](https://web.archive.org/web/20190502061737/https://twitter.com/mperham/status/884126933255995392). And, ultimately, most of Ruby's internals boil down to hash tables, so...|true %} Ruby isn't a fast language. So, if you execute less of it, you'll have a faster benchmark result. Feature-rich frameworks like Rails have a *lot* of code, and execute a lot more on each request because they are *doing more stuff*. This seems like 101-level stuff, but again, TechEmpower and other benchmarks typically do *not* make the difference in features obvious. On TechEmpower, all you get is this impossible-to-skim array of tags. {% marginnote_lazy techempower.png|Yes, this is is an easy-to-understand feature comparison which humans can read.|true %} On throughput microbenchmarks like TechEmpower, where differences are measured in milliseconds (or even microseconds), what you're really measuring is how many *CPU instructions* a particular language runtime generates in response to a particular request. And since there's no real way to compare featuresets between frameworks on TechEmpower, all frameworks are placed on an "equal footing" and you'll think that Rails is the slowest web framework in the world. The truth is that Rails does *a lot* on every request. Just create a new Rails app and look at the middleware stack (`rake middleware`). There's a lot of work being done here that *every good web application should do* but many frameworks *do not do for you*, at least by default. ### Performance is More Complicated than CPU or Maximum Throughput While on TechEmpower CPU usage is the bottleneck, in the real world, the CPU performance of language or the framework is almost never the *bottleneck* for a web application's performance. Web applications are fairly I/O heavy, especially as they grow more complicated. The modern Rails application may interact with three separate databases or more - their SQL database, Redis for their backend job processor, and Memcache for caching. Often, time spent interacting with these databases can make up 25% or more of a response. In addition, as a Ruby on Rails performance consultant, I've seen so many problems with application deployments that have nothing to do with the CPU performance of the framework or language: poor server configurations, memory leaks or bloat, or poor use of caching. Programmers, mysteriously, seem to find a way to completely degrade the performance of their application all on their own! Finally, most mature web applications spend *at most* 50% of their execution time in the framework itself, and far more time in the actual application code and other added dependencies. This is pretty easy to see in Ruby - take a look at a stacktrace and count how many of the top frames are from your framework. It won't be many. If your application could be rewritten in a faster framework in the same language, you would halve its response times *at best*. ## Rewrite Your Entire Application to Save $1,000/month What I worry about is what people do with the information presented in relative benchmarks like TechEmpower. Do they go home and rewrite their applications in the flavor-of-the-week framework or stack? Or, when choosing a stack for a *new* product or service, do people choose the "faster" stack over the "slower" one? [Heck, Pinterest rewrote it's Ads API in Elixir and now they have response times of less than a millisecond.](https://medium.com/@Pinterest_Engineering/introducing-new-open-source-tools-for-the-elixir-community-2f7bb0bb7d8c) Surely, that's just *better*, right? The question is, *why*? As we've already established, there's no difference for the end-user experience. So there's really only two reasons to choose a framework over another: a) it's faster and therefore I'll spend less on server costs to host it b) it's easy to develop with, and helps me ship quality features faster. Let's take a look at that server cost one, for a second. The majority of web applications handle far less than 1000 requests per second. I'd go as far as to say that most web application developers are employed by a company whose entire webapp does far less than 1000 requests/second. Most of them do less than 1000 requests/*minute*. Let's say you have a Rails application which serves 20,000 RPM (request/minute, or about 300 req/sec) at an average response time of 250 milliseconds. That's a pretty average profile for a large, mature Rails application. Such an application will take about 200 Puma processes to serve properly. That's equal to roughly a dozen Performance-L dynos on Heroku, or $6,000/month. Now, let's say you rewrite it in Phoenix, Node, or whatever flavor of the week you want and reduce that to 125 milliseconds. Before you jump out of your seat, remember that you're not going to reduce latency to 12 milliseconds or some other stupid-low amount: you're still going to be limited by I/O to the databases that back this application. Halving our application's latency means we need about half the amount of servers we needed before. So, congratulations: you rewrote your application (or chose your framework) to save $3,000/month. The load on the relational database backing this application won't change, so those costs will remain the same. When your application is big enough to be doing 20,000 RPM, you will have anywhere from a half-dozen to even fifty engineers, depending on your application's domain. A single software engineer costs a company at least $10,000/month in employee benefits and salary. So we're choosing our frameworks based on saving one-third of an engineer per month? And if that framework caused your development cycles to slow down by even *one third* of a mythical man-month, you've *increased* your costs, not decreased them. Choosing a web framework based on server costs is clearly a sucker's game. Why cargo-cult engineering practices from huge companies where a few milliseconds can save tens of thousands per month? You're not Pinterest (or Netflix, or...), you have different problems, and that's OK. ### It Isn't Getting Worse Computers aren't getting slower. While [Wirth's Law](https://en.wikipedia.org/wiki/Wirth%27s_law){% sidenote 4 "Software is getting slower more rapidly than hardware becomes faster." %} certainly holds for most end-user applications like your mobile phone apps, it doesn't really hold for your typical web application. Ruby web applications (and any web application) will continue to get faster because the slow grind of progress in hardware will continue to find ways to jam more CPU instructions into a clock cycle, or to make those clock cycles even faster, or to cram more cores onto a die. And the language isn't getting slower, either. Noah Gibbs of Appfolio has shown that [each minor version of Ruby decreases average response times by about 5-10%.](http://engineering.appfolio.com/appfolio-engineering/2017/5/22/rails-speed-with-ruby-240-and-discourse-180) ## Let's Talk About Happiness The performance doomsayers have always been wrong, and will continue to be wrong. [Take this gentleman from 2007](http://archive.oreilly.com/pub/post/multicore_hardware_and_the_fut.html): > No matter what implementation becomes the next de-facto Ruby platform, one thing is clear: People are interested in taking advantage of their newer, more powerful multi-core systems (as the recent surge in interest in Erlang in recent RailsConf and RubyConfs has shown). As Ruby becomes increasingly part of solutions that deal in high volumes of data processing, this demand can only increase. Ten years later, and scaling across multiple cores through preforking webservers like Puma and Unicorn is still plenty Good Enough. Ruby still isn't dead. I'm excited for the possibilities afforded by the [proposed Guild model](http://olivierlacan.com/posts/concurrency-in-ruby-3-with-guilds/), but is the language unusable until then? Nope. What I want is for the conversation around web frameworks and programming languages to change. There's too much talk of performance and concurrency, when in reality the margins are narrow and the costs minimal and getting lower. Languages aren't dying based on their concurrency or performance features alone. The better conversation, the more meaningful and impactful one, is which framework **helps me write software faster, with more quality, and with more happiness**. I know what the answer to that question is for me, and maybe the answer is different for you. ### "Polyglots" and "The New Hotness Stack" There's a subset of engineers who will never be happy writing software which isn't on the "new hotness stack". Engineers are always looking for a new problem to solve, something new to learn - and that's great! I've never related. GORUCO, the NYC Ruby conference started calling itself a "polyglot conference" this year, and the speaker schedule features talks on Python, Elixir, Rust, React and static typing. [Conference organizer Mike Dalessio's blog post announcing this](https://medium.com/@flavorjones/ruby-values-7b5ffe45aea7) reads like a tombstone. Benchmarks are often waved around in this "is X dead" discussion. As I hope I've shown above, there really is no benchmark which can prove that any language or framework is not suitable for writing web applications. Performance isn't the concern. Instead, the performance discussion regarding web applications is mostly FUD, spread by those trying to justify the engineering time they just spent rewriting their entire stack or what they're telling management so that they get to play with the coolest new toy they saw on Hacker News. Programmers are perpetually terrified of career obsolescence. Some are afraid of intellectual stagnation - that they'll become the crusty old person in the back office writing RPG to keep a truck parts company's order system running. But almost all of them are afraid of unemployment. They're worried that the world will move on from their particular stack, leaving their salaries and jobs in jeopardy. These fears are real - but let's realize that most of the discussion around "is stack X dead?!" are driven by *fear*, not concerns for the *requirements* of web applications. ## Fun and Games Let's be clear - performance still matters. Most organizations can and should save on server costs by focusing on speeding up their endpoints, and particularly slow endpoints probably do impact the customer experience [or the bottom line](https://wpostats.com/tags/revenue/) and should be sped up. What I've talked about above is just how little *framework choice* matters in the performance of your web application. Also, I'm not ragging on TechEmpower. It's a massive project, and they depend on domain experts creating PRs that fix any problems with the results. They're genuinely good people in my opinion, and aren't trying to push an agenda or participate in benchmarketing in favor of any particular stack. In conclusion, JavaScript, Go, Elixir and Python all suck, write Ruby :) No, of course not - write what you're productive in. If you're a web programmer, count your lucky stars that you get to choose your tools based on ergonomics, not on performance. --- ## Railsconf 2017: The Performance Update URL: https://www.speedshop.co/blog/railsconf-2017-the-performance-update/ {% marginnote_lazy sleepycat.gif|When you just can't conf any more|true %} Hello readers! Railsconf 2017 has just wrapped up, and as I did for [RubyConf 2016](/blog/rubyconf-2016-performance-update/), here’s a rundown of all the Ruby-performance-related stuff that happened or conversations that I had. ## Bootsnap Shopify recently released [bootsnap](https://github.com/Shopify/bootsnap), a Rubygem designed to boot large Ruby apps faster. It was released just a week or so before the conference, but Discourse honcho [Sam Saffron](https://twitter.com/samsaffron) was telling everyone about how great it was. It's fairly infrequently that someone is able to come up with one of these "just throw it in your Gemfile and voila your app is faster" projects, but it looks like this is one of them. {% marginnote_lazy ohno.gif|50% faster, you say?|true %} [Bootsnap reduced bootup time in development for Discourse by 50%.](https://gist.github.com/SamSaffron/d4f733108fe261815678b52b1a22f4b7) You may have heard of or used [bootscale](https://github.com/byroot/bootscale) - Bootsnap is intended to be an evolution/replacement of that gem. How does it work? Well, unlike a lot of performance projects, [Bootsnap's README is actually really good](https://github.com/Shopify/bootsnap) and goes into depth on how it accomplishes these boot speedups. Basically, it does two big things: makes `require` faster, and caches the compilation of your Ruby code. The `require` speedups are pretty straightforward - `bootsnap` uses caches to reduce the number of system calls that Ruby makes. Normally if you `require 'mygem'`, Ruby tries to open a file called `mygem.rb` on *every folder on your LOAD_PATH*. Ouch. Bootsnap thought ahead too - your application code is only cached for 30 seconds, so no worries about file changes not being picked up. The second feature is caching of compiled Ruby code. This idea has been around for a while - if I recall, [Eileen Uchitelle](https://twitter.com/eileencodes) and [Aaron Patterson](https://twitter.com/tenderlove) were working on something like this for a while but either gave up or got sidetracked. Basically, Bootsnap stores the compilation results of any given Ruby file *in the extended file attributes of the file itself*. It's a neat little hack. Unfortunately it doesn't really work on Linux for a few reasons - if you're using ext2 or ext3 filesystems, you probably don't have extended file attributes turned on, and even if you did, [the maximum size of xattrs on Linux is very, very limited](https://man7.org/linux/man-pages/man7/xattr.7.html) and probably can't fit the data Bootsnap generates. There was some discussion at the conference that, eventually, the load path caching features could be merged into Bundler or Rubygems. ## Frontend Performance {% marginnote_lazy noooo.gif|When the conf wifi doesn't co-operate|true %} I gave a workshop entitled "Front End Performance for Full-Stack Developers". The idea was to give an introduction to using Chrome's Developer Tools to profile and diagnose problems with first page load experiences. I thought it went *okay* - on conference wifi, many of the pages I had planned to use as examples suddenly had far far different load behaviors than what I had practiced with, so I felt a little lost! However, it must have gone *okay*, as Richard managed to [halve CodeTriage's paint times](https://github.com/codetriage/codetriage/pull/540) by marking his Javascript bundle as `async`. ## Application Server Performance After a recent experience with a client, I had a mini-mission at Railsconf to try to diagnose and improve some issues with performance in [`puma`](https://github.com/puma/puma/). The issue was with how Puma processes accept requests for processing. Every Puma process ("worker") has an internal "reactor". The reactor's job is to [listen to the socket](https://github.com/puma/puma/blob/master/lib/puma/reactor.rb#L29), buffer the request, and then hand requests to available threads. {% marginnote_lazy pumareactor.gif|Puma's reactor, accepting requests|true %} The problem was that Puma's default behavior is for the reactor to *accept as many requests as possible, without limit*. This leads to poor load-balancing between Puma worker processes, especially during reboot scenarios. Imagine you've restarted your `puma`-powered Rails application. While you were restarting, 100 requests have piled up on the socket and are now waiting to be processed. What *could* sometimes happen is that just a *few* of those Puma processes could accept a majority of those requests. This would lead to excessive request queueing times. This behavior didn't make a lot of sense. If a Puma worker has 5 threads, for example, why should it *ever* accept more than 5 requests at a time? There may be other worker processes that are completely empty and waiting for work to do - we should let those processes accept new work instead! So, [Evan fixed it](https://github.com/puma/puma/commit/482ea5a24abaccf33c49dc9238a22e2a9affe288). Now, Puma workers will not accept more requests than they could possibly process at once. This should really improve performance for single-threaded Puma apps, and should improve performance for multithreaded apps too. In the long term, I still think request load-balancing could be improved in Puma. For example - if I have 5 Puma worker processes, and 4 currently have a request being processed and 1 is completely empty, it's possible that a new request could be picked up by one of the already-busy workers. For example, if we're using MRI/CRuby and one of those busy workers hits an IO block (say it's waiting on a result from the database), it could pick up a new request instead of our totally-free worker. That's no good. And, as far as I know, routing is *completely random* between all the processes available and listening to the socket. Basically, the only way Puma can "get smarter" with it's request routing is to put some kind of "master routing process" on the socket, instead of letting the Puma workers listen directly to the socket themselves. One idea Evan had was to just put the Reactor (the thing that buffers and listens for new requests) in Puma's "master" process, and then have the master process decide which child process to give it to. This would let Puma implement more complex routing algorithms, such as round-robin or Passenger's ["least-busy-process-first"](https://www.phusionpassenger.com/library/indepth/ruby/request_load_balancing.html). Speaking of Passenger, Phusion founder Hongli spitballed the idea that Passenger could even act as a reverse proxy/load-balancer for Puma. It could definitely work (and would give Puma other benefits like offloading static file serving to Passenger) but I think Puma using the master process as a kind of "master reactor" is more likely. ## rack-freeze {% marginnote_lazy dicey.gif|Is my app threadsafe? Survey says... definitely maybe.|true %} One question that frequently comes up around performance is "how do I know if my Ruby application is thread-safe or not?" My stock is answer is usually to [run your tests in multiple threads](https://github.com/seattlerb/minitest/blob/master/lib/minitest/test.rb#L46-L46). There are two problems with this suggestion though - one, you can't run RSpec in multiple threads, so this is Minitest-only, and two, this really only helps you find threading bugs in your unit tests and application units, it doesn't cover most of your dependencies. One source of threading bugs is Rack middleware. Basically, the problem looks something like this: ```ruby class NonThreadSafeMiddleware def initialize(app) @app = app @state = 0 end def call(env) @state += 1 return @app.call(env) end end ``` A interesting way to surface these problems is to just `freeze` everything in all of your Rack middlewares. In the example above, `@state += 1` would now blow up and return a RuntimeError, rather than just silently adding incorrectly in a multithreaded app. That's exactly what [rack-freeze](https://github.com/ioquatix/rack-freeze) does (which is where the example above is from). Hat-tip to @schneems for bringing this up. ## snip_snip When talking to Kevin Deisz in the hallway (I don't recall what about), he told me about his gem called [`snip_snip`](https://github.com/kddeisz/snip_snip). Many of you have probably tried `bullet` at some point - [`bullet`](https://github.com/flyerhzm/bullet)'s job is to help you find N+1 queries in your app. `snip_snip` is sort of similar, but it looks for database columns which you `SELECT`ed but didn't use. For example: ```ruby class MyModel < ActiveRecord::Base # has attributes - :foo, :bar, :baz, :qux end class SomeController < ApplicationController def my_action @my_model_instance = MyModel.first end end ``` ...and then... ``` # somewhere in my_action.html.erb @my_model_instance.bar @my_model_instance.foo ``` ...then `snip_snip` will tell me that I `SELECT`ed the `:baz` and `:qux` attributes but didn't use them. I could rewrite my controller action as: ```ruby class SomeController < ApplicationController def my_action @my_model_instance = MyModel.select(:bar, :foo).first end end ``` Selecting fewer attributes, rather than *all* of the attributes (default behavior) can provide a decent speedup when you're creating many (hundreds or more, usually) ActiveRecord objects at once, or when you're grabbing objects which have many attributes (User, for example). ## Inlining Ruby In a hallway conversation with [Noah Gibbs](https://twitter.com/codefolio?lang=en), Noah mentioned that he's found that increasing the compiler's *inline threshold* when compiling Ruby can lead to a minor speed improvement. The *inline threshold* is basically how aggressively the compiler decides to copy-paste sections of code, *inlining* it into a function rather than calling out to a separate function. Inlining is usually always faster than jumping to a different area of a program, but of course if we just inlined the entire program we'd probably have a 1GB Ruby binary! [Noah found that increasing the inline threshold a little led to a 5-10% speedup on the optcarrot benchmark](https://bugs.ruby-lang.org/issues/12599), at the cost of a ~3MB larger Ruby binary. That's a pretty good tradeoff for most people. Here's how to try this yourself. We can pass some options to our compiler using the `CFLAGS` environment variable - if you're using Clang (if you're on a Mac, this is the default compiler): ``` CFLAGS="-O3 -inline-threshold=5000" Example with ruby-install ruby-install ruby 2.4.0 -- --enable-jemalloc CFLAGS="-O3 -inline-threshold=5000" ``` If you're using GCC: ``` CFLAGS="-O3 -finline-limit=5000" ``` I wouldn't try this in production *just yet* though - it seems to cause a few segfaults for me locally from time to time. Worth playing around with on your development box though! ## Your App Server Config is Wrong I gave a sponsored talk for Heroku that I titled "Your App Server Config is Wrong". Confreaks still hasn't posted the video, but [you can follow me on Twitter](https://twitter.com/nateberkopec) and I'll retweet it as soon as it's posted. Basically, the number one problem I see when consulting on people's applications is misconfigured app servers (Puma, Unicorn, Passenger and the like). This can end up costing companies thousands of dollars a month, or even costing them 30-40% of their application's performance. Bad stuff. Give the talk a watch. ## Performance Panel On the last day of the conference, Sam Saffron hosted a panel on performance with Richard, [Eileen](https://twitter.com/eileencodes), [Rafael](https://twitter.com/rafaelfranca) and myself. [Here's the video.](http://confreaks.tv/videos/railsconf2017-panel-performance-performance) Attenddee Savannah made this cool mind-mappy-thing: ## More Performance Talks There are a few more talks from Railsconf you should watch if you're interested in Ruby performance: * [5 Years of Scaling Rails to 80,000 RPS](http://confreaks.tv/videos/railsconf2017-5-years-of-rails-scaling-to-80k-rps) with Simon Eskildsen of Shopify. Simon's talks are always really good to begin with, so if you want to hear how Rails is used at one of the top-100 sites by traffic *in the world*, you should probably watch this talk. * [The Secret Life of SQL: How to Optimize Database Performance](http://confreaks.tv/videos/railsconf2017-the-secret-life-of-sql-how-to-optimize-database-performance) A (short) introduction to making those SQL queries as fast as possible from Bryana Knight, mostly discussing indexes and how you know if they're being used. * [High Performance Political Revolutions](http://confreaks.tv/videos/railsconf2017-high-performance-political-revolutions) Another "performance war story" from Braulio Carreno. ## Secret Project So, I won't go into *too much* detail here, but *somebody* showed me a very cool JavaScript project which was basically a "Javascript framework people who don't have a single-page-app". It looked like it would work extremely well with Turbolinks applications, or just apps which have a lot of Javascript behaviors but don't already use another framework. If you could imagine "Unobtrusive JavaScript: The Framework", that's what this looked like. I'll let you know when this project gets a public release. {% marginnote_lazy nogoingback1.gif|Son, once you start adding stuff to $(document).ready...|true %} One of Turbolinks' problems, IMO, is that it lacks a lot of teaching resources or pedagogy around "How To Build Complex Turbolinks-enabled Applications". Turbolinks requires a different approach to JavaScript in your app, and if you try to use an SPA framework such as Backbone or Angular with it, or if you try to just write your JavaScript the way you had before by dumping the kitchen sink into `turbolinks:load` hooks, you're Gonna Have a Bad Time. This framework looks like it could fix that by providing a "golden path" for attaching behaviors to pages. ## HTTP/2 This was touched on briefly in Aaron's keynote, but in hallway conversations with [Aaron](https://github.com/tenderlove) and [Evan](https://github.com/evanphx), the path forward on HTTP/2 support in Rack was discussed. I've advocated that you [just throw an HTTP/2-enabled CDN in front of your app and Be Done With It](/blog/what-http2-means-for-ruby-developers/) before, and Aaron and I pretty much agree on that. Aaron wants to add an HTTP/2-specific key to the Rack env hash, which could take a callback so you can do whatever fancy HTTP/2-y stuff you want in your application if Rack tells you it's an HTTP/2-enabled request. I see the uses of this being pretty limited, however, as Server Push can mostly [be implemented by your CDN](https://blog.cloudflare.com/announcing-support-for-http-2-server-push-2/) or [your reverse proxy](https://h2o.examp1e.net/configure/http2_directives.html). ## RPRG/Chat Update In [my Rubyconf 2016 update](/blog/rubyconf-2016-performance-update/), I said: > Finally, there was some great discussion during the Performance Birds of a Feather meeting about various issues. Two big things came out of it - the creation of a Ruby Performance Research Group, and a Ruby Performance community group. I want to say I'm *still working* on both of these projects. You should see something about the Research Group *very soon* (I have something *I* want to test surrounding memory fragmentation in highly multithreaded Ruby apps) and the community group some time after that. ## And Karaoke! {% marginnote_lazy karaoke.gif|[Jon McCartie](https://twitter.com/jmccartie), everyone|true %} That pretty much sums up my Railsconf 2017. Looking forward to next year, with even more Ruby performance and karaoke. --- ## Understanding Ruby GC through GC.stat URL: https://www.speedshop.co/blog/a-guide-to-gc-stat/ {% marginnote_lazy garbage.gif|I call that an object leak.|true %} Most Ruby programmers don't have any idea how garbage collection works in their runtime - what triggers it, how often it runs, and what is garbage collected and what isn't. That's not entirely a bad thing - garbage collection in dynamic languages like Ruby is usually pretty complex, and Ruby programmers are better off just focusing on writing code that matters for their users. But, occasionally, you get bitten by GC - either it's running too often or not enough, or your process is using tons of memory but you don't know why. Or maybe you're just curious about how GC works! One way we can learn a bit about garbage collection in CRuby (that is, the standard Ruby runtime, written in C) is to look at the built-in `GC` module. If you haven't [read the docs](https://ruby-doc.org/core-2.4.0/GC.html) of this module, check it out. There's a lot of interesting methods in there. But for right now, we're just going to look at one: `GC.stat`. {% marginnote_lazy bunneh.gif|Me, reading MRI source.|true %} `GC.stat` outputs a hash with a bunch of different numbers, but none of these numbers are really well documented, and some are just completely confusing unless you actually read the C code for Ruby's GC! Rather than having you do that yourself, I've done it for you. Let's take a look at all the information in `GC.stat` and see what we can learn about GC in Ruby. Here's what my `GC.stat` looks like in a just-booted `irb` session using Ruby 2.4.0: ```ruby { :count=>15, :heap_allocated_pages=>63, :heap_sorted_length=>63, :heap_allocatable_pages=>0, :heap_available_slots=>25679, :heap_live_slots=>25506, :heap_free_slots=>173, :heap_final_slots=>0, :heap_marked_slots=>17773, :heap_eden_pages=>63, :heap_tomb_pages=>0, :total_allocated_pages=>63, :total_freed_pages=>0, :total_allocated_objects=>133299, :total_freed_objects=>107793, :malloc_increase_bytes=>45712, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>13, :major_gc_count=>2, :remembered_wb_unprotected_objects=>182, :remembered_wb_unprotected_objects_limit=>352, :old_objects=>17221, :old_objects_limit=>29670, :oldmalloc_increase_bytes=>46160, :oldmalloc_increase_bytes_limit=>16777216 } ``` Ok, there's a lot there. That's 25 *undocumented* keys! Yay! First, let's talk about the **GC counts**: ```ruby { :count=>15, # ... :minor_gc_count=>13, :major_gc_count=>2 } ``` {% marginnote_lazy rgengc.png|RGenGC, introduced in Ruby 2.1. [Slide from Koichi Sasada.](https://engineering.heroku.com/blogs/2015-02-04-incremental-gc/)|true %} These are pretty straightforward. `minor_gc_count` and `major_gc_count` are just counts of each type of GC since the start of this Ruby process. In case you didn't know, since Ruby 2.1 there have been *two* types of garbage collections, major and minor. A minor GC will only attempt to garbage collect objects which are "new" - that is, they have survived 3 or less garbage collection cycles. A major GC will attempt to garbage collect *all* objects, even ones which have already survived more than 3 GC cycles. `count` will always equal `minor_gc_count` + `major_gc_count`. For more about this, see my talk at FOSDEM about [the history of Ruby Garbage Collection](https://www.youtube.com/watch?v=lcQ-hIfiljA). Tracking GC counts can be useful for a few reasons. We can figure out if a particular background job, for example, always triggers GCs (and how many it triggers). For example, here's a Rack middleware that logs the number of GCs that occurred while a web request was processing: ```ruby class GCCounter def initialize(app) @app = app end def call(env) gc_counts_before = GC.stat.select { |k,v| k =~ /count/ } @app.call(env) gc_counts_after = GC.stat.select { |k,v| k =~ /count/ } puts gc_counts_before.merge(gc_counts_after) { |k, vb, va| va - vb } end end ``` This won't be 100% accurate if your application is multithreaded, because another thread executing may have actually created the memory pressure which triggered these GC's, but it's a starting point! Now, let's move on to the **heap numbers**. ```ruby { # Page numbers :heap_allocated_pages=>63, :heap_sorted_length=>63, :heap_allocatable_pages=>0, # Slots :heap_available_slots=>25679, :heap_live_slots=>25506, :heap_free_slots=>173, :heap_final_slots=>0, :heap_marked_slots=>17773, # Eden and Tomb :heap_eden_pages=>63, :heap_tomb_pages=>0 } ``` In this context, the `heap` is a C data structure, sometimes also called the `ObjectSpace`, in which we keep references to of all the currently live Ruby objects. On a 64-bit system, each heap *page* contains approximately 408 *slots*. Each slot contains information about a single live Ruby object. First, you've got some information about the overall size of the entire Ruby object space. `heap_allocated_pages` is just the number of currently allocated heap pages (er, duh). These pages may be completely empty, completely full, or somewhere in between. `heap_sorted_length` is the actual size of the heap in memory - if we have 10 heap pages, and then free the 5th page (or some other random page from the middle), the *length* of the heap is still 10 pages (since we cannot move pages around in memory). `heap_sorted_length` will always be greater than or equal to the number of actually allocated pages. Finally, we've got `heap_allocatable_pages` - these are heap-page-sized chunks of memory that Ruby currently owns (i.e., has already `malloc`ed) that we could allocate a new heap page in. If Ruby needs a new heap page for additional objects, it will use this memory space first. Okay, now we've got a lot of numbers relating to the individual object `slots`. `heap_available_slots` is obviously the total number of slots in heap pages - `GC.stat[:heap_available_slots]` divided by `GC::INTERNAL_CONSTANTS[:HEAP_PAGE_OBJ_LIMIT]` will always equal `GC.stat[:heap_allocated_pages]`. `heap_live_slots` is the number of live objects, and `heap_free_slots` are slots in heap pages which are empty. `heap_final_slots` are object slots which have *finalizers* attached to them. Finalizers are sort of an obscure feature of Ruby - they're just Procs which run when an object is freed. Here's an example: ```ruby ObjectSpace.define_finalizer(self, self.class.method(:finalize).to_proc) ``` `heap_marked_slots` are pretty much the count of *old objects* (objects that have survived more than 3 GC cycles) plus *write barrier unprotected objects*, which we're going to get to a minute. As for practical use of the slot counts in `GC.stat`, I'd suggest monitoring `heap_free_slots` if you're having memory bloat issues. Large numbers of free slots (like more than 300,000) usually indicates that you have a few controller actions which are allocating large numbers of objects and then freeing them, which can permanently bloat the size of your Ruby process. For more about fixing that, [check out my Rubyconf talk of memory issues in Ruby.](https://www.youtube.com/watch?v=kZcqyuPeDao) {% marginnote_lazy tales-from-the-crypt.jpg|WELCOME TO THE TOMB PAGES|true %} Now we've got `tomb_pages` and `eden_pages`. Eden pages are heap pages which contain *at least one* live object in them. Tomb pages *contain no live objects*, and so have completely free slots. The Ruby runtime can *only release tomb pages back to the operating system*, eden pages can never be freed. Briefly, there are a few **cumulative allocated/freed numbers**. ```ruby { :total_allocated_pages=>63, :total_freed_pages=>0, :total_allocated_objects=>133299, :total_freed_objects=>107793 } ``` These numbers are *cumulative* for the life of the process - they are never reset and will not go down. They're pretty self explanatory. Finally, we have the **garbage collection thresholds**. ```ruby { :malloc_increase_bytes=>45712, :malloc_increase_bytes_limit=>16777216, :remembered_wb_unprotected_objects=>182, :remembered_wb_unprotected_objects_limit=>352, :old_objects=>17221, :old_objects_limit=>29670, :oldmalloc_increase_bytes=>46160, :oldmalloc_increase_bytes_limit=>16777216 } ``` So, one major misconception Ruby developers have is about *when* garbage collection is triggered. We can trigger GCs manually with `GC.start`, but that doesn't happen in production. Many seem to think that GC runs on some sort of timer - every X seconds or requests. That's not true. Minor GCs are triggered by a lack of free slots. Ruby doesn't automatically GC anything - it only GCs when it runs out of space. So when there are no `free_slots` left, we run a minor GC - marking and sweeping all of the "new" (i.e. not old, have survived fewer than 3 GCs) objects and objects in the *remember set* and those which are not *protected by the write-barrier*. I'll define those terms in a second. Major GCs can be triggered by a lack of free slots *after* a minor GC, or any of the following 4 thresholds being exceeded: oldmalloc, malloc, old object count, or the "shady"/writebarrier-unprotected count. The part of GC.stat we're looking at here shows each of those four thresholds (the `limit`) and the current state of the runtime on the way to that threshold. `malloc_increase_bytes` refers to when Ruby allocates space for objects *outside* of the "heap" we've been discussing so far. Each object slot in the heap pages is only 40 bytes (see `GC::INTERNAL_CONSTANTS[:RVALUE_SIZE]`) so what happens when we have an object larger than 40 bytes (say, a long string)? We `malloc` some space just for that object somewhere else! If we allocate 80 bytes for a string, for example, `malloc_increase_bytes` will increase by 80. When this number reaches the limit, we trigger a major GC. `oldmalloc_increase_bytes` is the same thing, but only includes objects that are *old*. `remembered_wb_unprotected_objects` is a count of objects which are not protected by the *write-barrier* and are part of the *remembered set*. Let's define both of those terms. The write-barrier is simply a interface between the Ruby runtime and an object, so that we can track references to and from the object when they're created. C-extensions can create new references to objects without going through the write-barrier, so objects which have been touched by C-extensions are called "shady" or "write-barrier unprotected". The remembered set is a list of *old* objects which have a reference to a *new* object. `old_objects` is just a count of object slots marked as old. Tracking these thresholds might be helpful if you're having trouble with a large number of major GCs. I hope this has been an educational look at GC.stat - it's an informative hash which can be used to build temporary debugging solutions for when you've got bad GC behavior that you need to fix. --- ## Rubyconf 2016: The Performance Update URL: https://www.speedshop.co/blog/rubyconf-2016-performance-update/ {% marginnote_lazy tired.gif|Post-conference haze|true %} Woo! I just got back from RubyConf. It was a great conference (as usual), and so nice to meet a few of you, my readers, there. I've got a lot to report on the performance front, so let's jump right in. ## JRuby+Truffle {% marginnote_lazy chrisseaton.jpeg|Chris Seaton %} JRuby+Truffle member Chris Seaton presented an excellent talk on the problem with C-extensions in Ruby and what he (and other Ruby implementations) are doing about it. JRuby+Truffle is a research project, sponsored by Oracle, which combines JRuby with the Graal and Truffle projects. It's sort of an *alternative* to the *alternative* Ruby implementation (JRuby). Though it runs on the JVM, like JRuby, it uses the [Truffle language framework](https://github.com/graalvm/truffle) to give itself nearly automatic just-in-time compilation and a host of other optimizations. It's a lot further behind than JRuby in terms of compatibility, but it's getting there. C-extensions have always been a problem for alternative Ruby implementations because Ruby's C API was never clearly defined, so C-extensions essentially just accessed the private internals of MRI. This meant that other Ruby implementations like JRuby had to *pretend* they were actually MRI to get C-extensions to work. {% marginnote_lazy truffle_rails.jpg | [From Chris' slides.](http://chrisseaton.com/rubytruffle/rubyconf16/rubyconf16-cexts.pdf) JRuby+Truffle progress on Rails tests. %} I learned alot about the JRuby+Truffle project from Chris' talk, and, if it can achieve greater compatibility with Rails, it could be an amazing alternative implementation. Interestingly, JRuby+Truffle is actually the largest paid Ruby implementation team, with more paid developers than even MRI! They're most of the way to running Rails applications, but C-extensions (especially Nokogiri and OpenSSL) remain the main stumbling block. Chris said that almost 25% of all lines of code in Rubygems are actually C-extensions - ouch! {% marginnote_lazy menard.jpg | A "rope" style string representation %} A lot of the things that the project does are really radical: see this talk from [Kevin Menard about how JRuby+Truffle represents strings as ropes](https://www.youtube.com/watch?v=UQnxukip368), which no other Ruby implementation does. In addition, because of the way the Graal compiler works in combination with the [Sulong interpreter](https://github.com/graalvm/sulong), JRuby+Truffle can optimize C code and Ruby code together, and at the same time. That is, from the compilers perspective, both Ruby and C code are identical. That's powerful stuff! All of this means that, on some specific, limited benchmarks, JRuby+Truffle can be 30-100x faster than MRI! ## Upcoming Changes to CRuby {% marginnote_lazy deopt.jpg | [Shyouhei's slides](https://speakerdeck.com/shyouhei/optimizing-ruby) showed impressive benchmarks. %} Shyouhei Urabe gave a talk about a de-optimizing engine for CRuby. Basically, compilers can optimize VM instructions when certain assumptions are made - for example, we can speed up "2 + 2" if we know "+" is not overridden. To make those optimizations, though, we also need to de-optimize if someone *does* override the "+" operator. JRuby has been doing this for a long time now, but we've never had anything of the sort in CRuby. So, since basically anything can be overridden in Ruby, a de-optimizer is actually required before we can start on any optimizations. Shyouhei has proposed one - the details are pretty technical, but [you can read more about it here](https://github.com/ruby/ruby/pull/1419). He showed that in the worst case, it makes a Rails app about 5% slower and uses no additional memory. Of course, the Rails app will be *faster* (and probably use more memory) once the optimizations are built on top of the de-optimizer. {% marginnote_lazy aaron_talk.jpg | Aaron cheated at #rubyfriends. | true %} Aaron Patterson gave a great overview of garbage collection and memory management in Ruby as a prelude to his optimizations for Ruby's heap structure. Basically, we can improve copy-on-write performance and total RSS usage if we allocate objects into two separate areas - probably old (objects which won't be GC'd, like Classes, Modules, etc) and probably new (everything else). RSS usage on Github improved by about 10%. [You can see his PR to Github's Ruby fork here](https://github.com/github/ruby/pull/32). {% marginnote_lazy heap_compact.png | [Read more about heap compaction on Wikipedia](https://en.wikipedia.org/wiki/Mark-compact_algorithm). %} In addition, there was some hallway discussion about a compacting garbage collector for CRuby. This would be a *very* big deal for total memory usage. Previously, Ruby hasn't had a compacting GC because C-extensions can hold memory addresses directly to Ruby objects - moving the object in memory would cause a segfault. However, Ruby 2.1 introduced "sunny/shady" objects - sunny objects have never been accessed by C-extensions, vice versa for shady objects. CRuby *could* move sunny objects around the heap to optimize total memory usage. Aaron Patterson has said on Twitter that he's experimenting with it now, and it looks like he's making great progress. Matz clarified the goal of Ruby3x3 (making Ruby "3 times faster"). One of the main ways the core team are measuring that progress is [through the optcarrot benchmark](https://github.com/mame/optcarrot). Ruby 3 should run the optcarrot benchmark 3 times faster than Ruby 2.0. ## Everything Else {% marginnote_lazy tokyo2020.jpg | Another reason to look forward to the next Summer Olympics. | true %} Don't look for true static typing in Ruby anytime soon. Matz said that he thinks type annotations aren't DRY and aren't human-friendly. He did say he liked Crystal though! Instead, Matz re-iterated his proposal for "soft" or "inferred" types in Ruby - if the compiler can tell that you're going to call `to_s` on an object that doesn't define that method, it will throw an error. Look for this in Ruby 3 (which Matz has said has a target release date of "before the Tokyo Olympics in 2020"). I gave a talk on reducing memory usage in Ruby applications. [You can see the slides and notes here](https://gist.github.com/nateberkopec/2b1f585046adad9a55e7058c941d3850). If you purchased [the Complete Guide to Rails Performance](https://www.railsspeed.com), there's probably not much new there to you, but if you haven't, go buy it! {% marginnote_lazy pumacore.jpg | | true %} Evan Phoenix has added myself and Richard Schneeman (of Heroku) to Puma. We're going to try to reduce the issue/PR backlog, but do send us more bug reports if you're having trouble with Puma! {% marginnote_lazy killthreads.jpg | It's hard to read, I know, but Koichi's shirt really does say "Kill Threads". | true %} One interesting aspect of the conference was how much the Ruby core team (Matz and Koichi, mostly) were hostile to Threads. Matz said in his opening keynote that in retrospect, he wished he had never added Thread to Ruby. It seemed like, from a language designer's perspective, he thought it was a poor abstraction and was too difficult to use. Koichi even wore a "Kill Threads!" shirt while presenting about Guilds, the new proposed Ruby concurrency model. Speaking of Guilds, Koichi discussed some more details around the proposed model. GC will remain global and will not be per-Guild. Overhead for creating a new Guild should be extremely low - akin to creating a new Thread. Transferring big objects (like huge Hashes) between Guilds will probably require a new datastructure, like "BigHash" or "BigArray". Feedback has been very positive to Guilds so far, and many believe it could be just what we need. It seems like we should see Guilds in Ruby some time prior to Ruby 3 - maybe in a few years, so Ruby 2.6 or 2.7. There was a great talk by Ariel Caplan on the performance issues behind OpenStruct, that oft-forgotten bit of the core library. If, like me, you thought OpenStruct was slow because it invalidated the global method cache, you're wrong! That was fixed in Ruby 2.1. However, there are plenty more things that slow down OpenStruct, which Ariel discussed in his talk. If you're interested, [checkout his Github repo for a far faster OpenStruct-like implementation.](https://github.com/amcaplan/dynamic_class) Colin Jones gave a great talk on DTrace, the performance profiler - [here's his slides](https://speakerdeck.com/trptcolin/diving-into-the-details-with-dtrace-rubyconf-2016-edition). I'll be digging in to DTrace more in the next few months. It's an extremely powerful tool. ## Two New Community Initiatives Finally, there was some great discussion during the Performance Birds of a Feather meeting about various issues. Two big things came out of it - the creation of a Ruby Performance Research Group, and a Ruby Performance community group. Let's discuss each. First, the Research Group. Companies with production Ruby applications want faster Ruby, and Ruby implementors want more production data to figure out if the decisions they're making are making people's apps faster or slower. While open-source benchmarks exist, they're often highly synthetic and don't match real-world usage. And the open-source apps that do exist are limited and may not provide access to the production environment. Additionally, giving any Rails application to a researcher or implementor is a huge pain because setting up even a trivial app can take hours. So, we proposed the creation of a Research Group. The purpose of the group would be to allow researches (core team from MRI, JRuby, JRuby+Truffle, and performance-sensitive projects like Rails, Sidekiq and Puma) to run limited, small experiments on production Ruby web applications. These experiments would produce data, which would be returned back to the researchers. Some example experiments might be "how often is `object_id` called on objects?" or "how often are global variables set?". Experiments may be as simple as installing a Rubygem, or as complex as using a patched Ruby version. I'm taking the lead on this project, so expect to hear more before Christmas. Second, we discussed the need for an open community of Ruby performance enthusiasts. I noted that the CGRP Slack channel (included with the purchase of the course) was already pretty much what was desired. I'm considering opening up the community with a nominal payment (like $5/year) or proof that you've had a performance-related PR accepted to Ruby or Rails. You'll probably hear more from me about that topic soon. For updates on both of these projects, you're probably best off subscribing to me newsletter below. ## A Great Conference! I had a great time at Rubyconf. Plus, who knew downtown Cincinatti had a huge casino? Thanks, [ballot initiatives](https://ballotpedia.org/Ohio_Casino_Approval_and_Tax_Distribution,_Amendment_3_(2009))! Looking forward to seeing all of you at Railsconf in the spring. [By the way, did you know the CFP is already open?](https://railsconf.org) --- ## What HTTP/2 Means for Ruby Developers URL: https://www.speedshop.co/blog/what-http2-means-for-ruby-developers/ {% marginnote_lazy yC6kwyY.gif|Okay, way too much magical pixie dust|true %} HTTP/2 is coming! No, wait, HTTP/2 is here! [After publication in Q1 of 2015](https://github.com/http2/http2-spec), HTTP/2 is now an "official thing" in Web-land. As of writing (December 2015), [caniuse.com estimates about 70% of browsers globally can now support HTTP/2](http://caniuse.com/#feat=http2). So, I can use HTTP/2 in my Ruby application *today*, right? After all, Google says that [some pages can load up to 50% faster just by adding HTTP/2/SPDY support](https://www.chromium.org/spdy/spdy-whitepaper/), it's magical web-speed pixie dust! Let's get it going! {% marginnote_lazy rHXhQoM.jpg|Uh, hello Aaron? Yeah, could you like, fix Rack please? %} Well, no. Not really. Ilya Grigorik has written an experimental HTTP/2 webserver in Ruby, but it's not compatible with Rack, and therefore not compatible with any Ruby web framework. While [@tenderlove](http://tenderlovemaking.com/) has done [some](https://github.com/tenderlove/the_metal) [experiments](https://github.com/tenderlove/arghhh) [with HTTP/2](https://twitter.com/tenderlove/status/626044968419721217), Rack remains firmly stuck in an HTTP/1.1 world. [While it was discussed that this would change with Rack 2 and Rails 5](https://github.com/tenderlove/the_metal/issues/5), very little actually changed. Until the situation changes at the Rack level, Rails and all other Ruby web frameworks are stuck with HTTP/1.1. Part of the reason why progress has been slow here (other than, apparently, that [@tenderlove](http://tenderlovemaking.com/) is the only one that wants to work on this stuff) is that Rack is thoroughly designed for an HTTP/1.1 world. In a lot of ways, HTTP/2's architecture will probably mean that whatever solution we come up with will bear more resemblance to ActionCable than it does to to Rack 1.0. Ilya Grigorik, Google's public web performance advocate, [has laid out 4 principles for the web architecture of the future](https://www.igvita.com/2012/01/18/building-a-modern-web-stack-for-the-realtime-web/). Unfortunately, Rack is incompatible with most of these principles: * **Request and Response streaming should be the default**. While it isn't the default, Rack at least supports streaming responses (it has for a while, at least). * **Connections to backend servers should be persistent**. I don't see anything in Rack that stops us from doing this at the moment. * **Communication with backend servers should be message-oriented**. Here's one of the main hangups - Rack is designed around the request/response cycle. Client makes a request, server makes a response. While we have some limited functionality for server pushes (see [ActionController::Live::SSE](http://api.rubyonrails.org/classes/ActionController/Live/SSE.html)), communication in Rack is mostly designed around request/response, not arbitrary messages that can go in either direction. * **Communication between clients and backends should be bi-directional**. Another problem for Rack - it isn't really designed for pushes straight from the server without a corresponding request. Rack essentially assumes it has direct read/write access to a socket, but HTTP/2 complicates that considerably. If you're paying attention, you'll realize these 4 principles sound a hell of a lot like WebSockets. HTTP/2, in a lot of ways, obviates Ruby developers' needs for WebSockets. [As I mentioned in my guide to ActionCable](/blog/action-cable/), WebSockets are a layer *below* HTTP, and one of the major barriers of WebSocket adoption for application developers will be that many of the things you're used to with HTTP (RESTful architecture, HTTP caching, redirection, etc) need to be *re-implemented* with WebSockets. Once HTTP/2 gets a JavaScript API for opening bi-directional streams to our Rails servers, the reasons for using WebSockets at all pretty much evaporate. When these hurdles are surmounted, HTTP/2 could bring, potentially, great performance benefits to Ruby web applications. ## HTTP/2 Changes That Benefit Rubyists Here's a couple of things that will benefit almost every web application. ### Header Compression One of the major drawbacks of HTTP 1.1 is that headers cannot be compressed. Recall that a traditional HTTP request might look like this: ``` accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 accept-encoding:gzip, deflate, sdch accept-language:en-US,en;q=0.8 cache-control:max-age=0 cookie:_ga=(tons of Base 64 encoded data) upgrade-insecure-requests:1 user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73 Safari/537.36 ``` Cookies, especially, can balloon the size of HTTP requests and responses. Unfortunately, there is no provision in the HTTP 1.x specification for compressing these - unlike response bodies, which we can compress with things like `gzip`. {% marginnote_lazy ITu2NDW.jpg|Huffman coding, duh! %} Headers can make up 800-1400KB of a request or response - multiply this to Web Scale and you're talking about a *lot* of bandwidth. HTTP/2 will reduce this *greatly* by compressing headers with something fancy called Huffman coding. You don't really need to understand how that works, just know this - HTTP/2 makes HTTP headers smaller by nearly 80%. And you, as an application author, won't need to do anything to take advantage of this benefit, because the compression/decompression will happen at lower levels (probably in Rack or some new layer directly below). This compression will probably be one of the first HTTP2 features that Rails apps will be able to take advantage of, since header compression/decompression can happen at the load balancer or at the web server, before the request gets to Rack. You can take advantage of header compression today, for example, by placing your app behind Cloudflare’s network, which provides HTTP/2 termination at their load balancers. ### Multiplexing {% marginnote_lazy mcxYuDb.gif|Damn, shoulda multiplexed.|true %} Multiplexing is a fancy word for two-way communication. HTTP 1.x was a one-way street - you could only communicate in one direction at a time. This is sort of like a walkie-talkie - if one person is transmitting with a walkie-talkie, the person on the other walkie-talkie can't transmit until the first person lets off the "transmit" button. On the server side, this means that we can send multiple responses to our client over a *single connection* at the *same time*. This is nice, because setting up a new connection is actually sort of expensive - it can take 100-500ms to resolve DNS, open a new TCP connection, and perhaps negotiate SSL. Multiplexing will completely eliminate the need for domain sharding, a difficult-to-use HTTP 1.x optimization tactic where you spread requests across multiple domains to get around the browser's 6-connections-per-domain limit. Instead of each request we want to make in parallel needing a new connection, a client browser can request several resources across the same connection. I mentioned domain sharding was fraught with peril - that's because it can cause network congestion. The entire reason the 6-connections-per-domain limit even exists is to limit how much data the server can spit back at the client at one time. By using domain sharding, we run the risk of *too much data* being streamed back to clients and causing packet loss, ultimately slowing down page loads. [Here's an awesome deconstruction of how domain sharding too much actually slowed down Etsy's page loads by 1.5 seconds](http://calendar.perfplanet.com/2013/reducing-domain-sharding/). One area where Rails apps can take advantage of multiplexing today is by using an HTTP/2 compatible CDN for serving their assets. ### Stream Prioritization HTTP/2 allows clients to express preferences as to which requests should be fulfilled first. For example, browsers can optimize by asking for JS and CSS before images. They can *sort of* do this today by *delaying* requests for resources they don’t want right away, but that's pretty jank and fraught with peril. As an example, [here's an article about how stream prioritization sped up a site's initial paint times by almost 50%](http://blog.kazuhooku.com/2015/04/dependency-based-prioritization-makes.html). Again, your Ruby app can take advantage of this right now by using an HTTP/2 compatible CDN. ### Latency Reduction HTTP/2 will especially benefit users in high-latency environments like mobile networks or developing countries. [Twitter found that SPDY (the predecessor to HTTP/2) sped up requests in high-latency environments much more than in low-latency ones](https://blog.twitter.com/2013/cocoaspdy-spdy-for-ios-os-x). ### Binary {% marginnote_lazy /assets/posts/img/computers.gif|I'm a computer!|true %} HTTP/2 is a binary protocol. This means that, instead of plain text being sent across the wire, we're sending 1s and 0s. In short, this means HTTP/2 will be easier for implementers, because plain-text protocols are often more difficult to control for edge-cases. But for clients and servers, we should see slightly better bandwidth utilization. Unfortunately, this means you won't be able to just `telnet` into an HTTP server anymore. To debug HTTP/2 connections, you're going to need to use a tool that will decode it for you, such as the browser's developer tools or something like WireShark. ### One connection means one TLS handshake One connection means TLS handshakes only need to happen once per domain, not once per connection (say, up to 6 TLS handshakes *for the same domain* if you want to download 6 resources from it in parallel). Rails applications can experience the full benefit of this HTTP/2 feature today by being behind an HTTP/2 compatible web server or load balancer. ## How Rails Apps Will Change with HTTP/2 All of the changes I've mentioned so far will generally benefit all Ruby web applications - but if you'll permit me for a minute, let's dive in to Rails as a specific example of your applications may have to change in the future to take full advantage of HTTP/2. Primarily, HTTP/2 will almost completely upend the way Rails developers think about assets. ### Concatenation is no more In essence, all HTTP/2 does is make requests and responses cheaper. If requests and responses are cheap, however, suddenly the advantages of asset concatenation become less clear. HTTP/2 can transport a JS file in 10 parts pretty much as fast as it can transport that same file in 1 part - definitely not the case in HTTP/1.x. In HTTP/1.x-world, we've done a lot of things to get around the fact that opening a new connection to download a sub-resource was expensive. Rails concatenated all of our Javascript and CSS into a single file. Some of us used frameworks like Compass to automatically sprite our images, turning many small .pngs into one. But since HTTP/2 makes many-files just as cheap as one-file, that opens up a whole new world of advantages for Rails: * Development mode will get waaaay faster. In development mode, we don't concatenate resources, meaning a single page often requires dozens of scripts and css files. HTTP/2 should make this just as fast as a single concatenated file in production. * We can experiment with more granular HTTP caching schemes. For example, in todays Rails' world, if you change *a single line* in your (probably massive) application.js, the entire file will need to be re-downloaded by *all* of your clients. With HTTP/2, we'll be able to experiment with breaking our one-JS and one-CSS approach into several different files. Perhaps you'll split out high-churn files so that low-churn CSS won't be affected. * We can amortize large amounts of CSS and JS over several page loads. In today's Rails world, you have to download *all* of the CSS and JS for the *entire application* on the first page load. With HTTP/2 and it's cheap connections, we can experiment with breaking up JS and CSS on a more granular basis. One way to do it might be per-controller - you could have a single base.css file and then additional css files for each controller in the app. Browsers could download bits and pieces of your JS and CSS as they go along - this would effectively reduce homepage (or, I guess, first-page) load times while not imposing any additional costs when pages included several CSS files. ### Server push really makes things interesting HTTP/2 introduces a really cool feature - server push. All this means is that servers can proactively *push* resources to a client that the client *hasn't specifically requested*. In HTTP/1.x-land, we couldn't do this - each response from the server had to be tied to a request. Consider the following scenario: 1. Client asks for `index.html` from your Rails app. 2. Your Rails server generates and responds with `index.html`. 3. Client starts parsing `index.html`, realizes it needs `application.css` and asks your server for it. 4. Your Rails server responds with `application.css`. With server push, that might look more like this: 1. Client asks for `index.html` from your Rails app. 2. Your Rails server generates and responds with `index.html`. While it's doing this, it realizes that `index.html` *also* needs `application.css`, and starts sending that down to the client as well. 3. Client can display your page without requesting any additional resources, because it already has them! Super neato, huh? This will especially help in high-latency situations where network roundtrips take a long time. Interestingly, I think some of this means we might need to serve different versions of pages, or at least change Rails' server behavior, based on whether or not the connection is HTTP/2 or not. Hopefully this will be automatically done by the framework, but who knows - nothing has been worked on here yet. ## How to Take Advantage of HTTP/2 Today If you're curious about where we have to go next with Rack and what future interfaces might look like in Rails for taking advantage of HTTP/2, [I find that this Github thread is extremely illuminating](https://github.com/tenderlove/the_metal/issues/5). For all the doom-and-gloom I just gave you about HTTP/2 still looking a ways off for Ruby web frameworks, take heart! There are ways to take advantage of HTTP/2 today *before* anything changes in Rack and Rails. ### Move your assets to a HTTP/2 enabled CDN An easy one for most Rails apps is to use a CDN that has HTTP/2 support. Cloudflare is probably the largest and most well-known. There's no need to add a subdomain - simply directing traffic through Cloudflare should allow browsers to upgrade connections to HTTP/2 where available. The page you're reading right now is using Cloudflare to serve you with HTTP/2! Open up your developer tools to see what this looks like. ### Use an HTTP/2 enabled proxy, like nginx or h20. You should receive most of the benefits of HTTP/2 just by proxying your Rails application through an HTTP/2-capable server, such as nginx. For example, Phusion Passenger can be deployed as an nginx module. nginx, as of 1.9.5, supports HTTP/2. Simply configure nginx for HTTP/2 as you would normally, and you should be able to see some of the benefits (such as header compression). With this setup, however, you still won't be able to take advantage of server push (as that has to be done by your application) or the websocket-like benefits of multiplexing. --- ## How Changing WebFonts Made Rubygems.org 10x Faster URL: https://www.speedshop.co/blog/how-changing-webfonts-made-rubygems-10x-faster/ {% marginnote_lazy https://imgur.com/nzECFNz.jpg ||true %} I'm passionate about fast websites. That's a corny thing to say, I realize - it's something you'd probably read on a resume, next to a description of how "detail-oriented" and "dedicated" I am. But really, I love the web. The openness of the Web has contributed to a global coming-together that's created beautiful things like Wikipedia or the FOSS movement. As Jeff Bezos said {% sidenote 1 "\"Investments in speed are going to pay dividends forever.\" [Basecamp, Signal vs. Noise](https://signalvnoise.com/posts/3112-how-basecamp-next-got-to-be-so-damn-fast-without-using-much-client-side-ui)"%}, nobody is going to wake up 10 years from now and wish their website was slower. By making the web faster, we can make bring the Web's amazing possibilities for collaboration to an even wider global audience. Internet access is not great everywhere - Akamai puts the global average connection bandwidth at 5.1 Mbps {% sidenote 2 "Read this and you'll wish you lived in Bulgaria. [Akamai State of the Internet, 2015](https://www.akamai.com/us/en/multimedia/documents/content/state-of-the-internet-2015-executive-review-volume-02.pdf)". %} {% marginnote_lazy https://i.imgur.com/zGunpp4.gif|Using rubygems.org on a slow connection %} For those of you doing the math at home, that's a measly 625 kilobytes per second. The US average isn't much better - 12.0 Mbps, or just 1.464 megabytes per second. When designing the website for a project that wants to encourage global collaboration, as most FOSS sites do, we need to be thinking about our users in low-bandwidth areas (which is to say, the majority of global internet users). We don't want to make a high-bandwidth connection a barrier to learning a programming language or contributing to open-source. It's with this mindset that I've been looking at the performance of [Rubygems.org](https://rubygems.org) for the last few weeks. As a Rubyist, I want people all over the world to be able to use Ruby - fast connection or no. Rubygems.org is one of the most critical infrastructure pieces in the Ruby ecosystem - you use it every time you `gem install` (or `bundle install`, for that matter). Rubygems.org also has a web application, which hosts a gem index and search function. It also has some backend tools for gem maintainers. I decided to dig in to the frontend performance of Rubygems.org for these reasons. ## Diagnosing with Chrome Timeline {% marginnote_lazy https://i.imgur.com/5fnVtiy.png|For more about Chrome Timeline, [see my guide.](/blog/frontend-performance-chrome-timeline/) %} When diagnosing a website's performance, I do two things straight off the bat: * Open the site in Chrome. Open DevTools, and do a hard refresh while the Network tab is open. * Run a test on [webpagetest.org](https://www.webpagetest.org). Both webpagetest.org and Google Chrome's Network tools pointed out an interesting fact - while total page weight was reasonable (about 600 KB), over 72% of the total page size was WebFonts (434 KB!). Both of these tools were showing that page loads were being heavily delayed by waiting for these fonts to download. I plugged Akamai's bandwidth statistics into DevTool's network throttling function. Using DevTool's throttler is a bit like running your own local HTTP proxy that will artificially throttle down network bandwidth to whatever values you desire. The results were pretty dismal. {% sidenote 3 "Lest you try this on your own site, don't immediately discard the results if you think they're \"way too slow, our site never loads like that!\" At 625 KB/s, Twitter still manages to paint within 2 seconds. Google's homepage does it half a second." %} | | Time to First Paint | Time to Paint Text (fonts loaded) | Time to `load` Event | | --- | --- | --- | --- | | US (1.4 MB/s) | 3.56s | 3.83s | 3.96s | | Worldwide (625 KB/s) | 7.41s | 7.59s | 8.20s | Ouch! I used DevTool's Filmstrip view to get a rough idea of when fonts were loaded in as well. You can use the fancy new [Resource Timing API](http://googledevelopers.blogspot.com/2013/12/measuring-network-performance-with.html) to get this value precisely (and on client browsers!) but I was being lazy. {% marginnote_lazy https://i.imgur.com/acKj5tD.png|When these standards were discovered (1968), [The Nova Minicomputer](https://en.wikipedia.org/wiki/Data_General_Nova) had just been released. 1968 was a good year for computing - [Djikstra wrote GOTO considered harmful](https://www.cs.utexas.edu/~EWD/transcriptions/EWD02xx/EWD215.html), the [Apollo Guidance Computer](https://en.wikipedia.org/wiki/Apollo_Guidance_Computer) left the atmosphere on Apollo 8, and [The Mother of All Demos](https://www.youtube.com/watch?v=yJDv-zdhzMY) was presented. %} When evaluating the results of any performance test, I use the following rules-of-thumb. These guidelines for human-computer interaction speeds have remained constant since [they were first discovered in the late 60's](https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two): * **0.1 seconds** is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result. * **1.0 second** is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data. * **10 seconds** is about the limit for keeping the user's attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect. {% sidenote 4 "This is the Nielsen Norman group's interpretation of the linked paper. See the rest of their take on response times here." %} Most webpages become *usable* (that is, the user can read and begin to interact with them) in the range of 1 to 10 seconds. This is *good*, but it's possible that for many connections we can achieve websites that, on first/uncached/cold loading, can be usable in less than 1 second. Using these rules-of-thumb, I decided we had some work to do to improve Rubygems.org's paint and loading times on poor connections. As fonts comprised a majority of the site's page weight, I decided to start there. ## Auditing font usage WebFonts are awesome - they really make the web beautiful. The web is typography {% sidenote 4 "[Web Design is 95% Typography](https://ia.net/topics/the-web-is-all-about-typography-period)" %}, so changing fonts can have a huge effect on the character and feel of a website. For these reasons, WebFonts have become extremely popular *very* quickly - HTTP Archive estimates about 51% of sites currently use WebFonts {% marginnote_lazy https://imgur.com/KzlGyN9.png|[via HTTP Archive](http://httparchive.org/trends.php#perFonts) %}, and that number is still growing. WebFonts are here to stay, but that doesn't mean it's impossible to use them poorly. Rubygems.org was using Adobe Typekit - a common setup - and using a single WebFont, Aktiv Grotesk, for all of the site's text. By using Chrome's Network tab, I realized that Rubygems.org was loading more than a dozen individual weights and styles of the site font, Aktiv Grotesk. Immediately some red flags started to go up - how could I possibly audit all of the site's CSS and determine if each of these weights and styles was actually being used? Instead of taking a line-by-line approach of combing through the CSS, I decided to approach the problem from first principles - what was the intent of the design? *Why* was Rubygems.org using WebFonts? ### Deciding on Design Intent {% marginnote_lazy https://i.imgur.com/ubws6J0.jpg|Not pictured: me. %} Now, I am not a designer, and I don't pretend to be one on the internet. As developers, our job isn't to tell the designers "Hey, you're dumb for including over 500KB of WebFonts in your design!". That's not their job. As performance-minded web developers, our job is to **deliver the designer's vision in the most performant way possible**. {% marginnote_lazy https://i.imgur.com/D26hubK.png %} To the right is a screenshot of Rubygems.org's homepage. Most of the text is set at around a ~14px size, with the notable exception of the main heading, which is set in large type in a very light weight. All text is set in the same font, Aktiv Grotesk, which could be described as a grotesque or neo-grotesque sans-serif. {% sidenote 5 "What's a grotesque? [Wikipedia has a good description.](https://en.wikipedia.org/wiki/Sans-serif#Grotesque)" %} Based on my interpretation of the design, I decided the design's intent was: * For h1 tags, use a very light weight grotesque type. * For all other text, use a grotesque type set at it's usual, context-appropriate weight. * The design should be consistent across platforms. * The design should be consistent across most locales/languages. {% marginnote_lazy https://i.imgur.com/Ty6gt5R.jpg|Image from Martin Silverant's excellent [Why Helvetica is Not Great](http://martinsilvertant.deviantart.com/journal/?offset=1) %} The site's font, Aktiv Grotesk, bears more than a passing resemblance to Helvetica or Arial - they're both grotesque sans-serifs. At small (~14px) sizes, the difference is mostly indistinguishable to non-designers. I already had found a way to eliminate the majority of the site's WebFont usage - use WebFonts only for the h1 header tags. The rest of the site could use a Helvetica/Arial font stack with very little visual difference. **This one decision eliminated *all but one* of the weights and styles required for Rubygems.org!** {% marginnote_lazy https://i.imgur.com/hntGkcE.jpg|If I may make a suggestion as to which system font to use... %} Using WebFonts for "body" text - paragraphs, h3 and lower - seems like a loser's game to me. The visual differences to system fonts are usually not detectable at these small sizes, at least to layman eyes, and the page weight implications can be immense. Body text usually requires several styles - bold, italic, bold italic at least - whereas headers usually appear only in a single weight and style. **Using WebFonts only in a site's headers is an easy way to set the site apart visually without requiring a lot of WebFont downloads.** I briefly considered not using WebFonts at all - most systems come with a variety of grotesque sans-serifs, so why not just use those on our headers too? Well, this would work great for our Mac users. Helvetica looks stunning in a light, 100 weight. But Windows is tougher. Arial isn't included in Windows in anything less than 400 (normal) weight, so it wouldn't work for Rubygems.org's thin-weight headers. And Linux - well, who knows what fonts they have installed? It felt more appropriate to *guarantee* that this "lightweight" header style, so important to the character of the Rubygems.org design, would be visually consistent across platforms. So I had my plan: * Use a WebFont, in a grotesque sans-serif style, to display all the site's h1 tags in a very light weight. * Use the common Helvetica/Arial stack for all other text. ## Changing to Google Fonts {% marginnote_lazy https://www.google.com/logos/doodles/2014/world-cup-2014-47-5450493904027648.5-hp.gif %} Immediately, I knew Typekit wasn't going to cut it for Rubygems.org. Rubygems.org is an open-source project with many collaborators, but issues with fonts had to go through one person (or a cabal of a few people), the person that had access to the Typekit account. With an OSS font, or a solution like Google Fonts (where anyone can create a new font bundle/there is no 'account'), we could all debug and work on the site's fonts. That reason - the "accountless" and FOSS nature of the fonts served by Google Fonts - initially lead me to use Google Fonts for Rubygems.org. Little did I realize, though, that Google Fonts offers a number of performance optimizations over Typekit that would end up making a huge difference for us. ### Serve the best possible format for a user-agent {% marginnote_lazy https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/images/font-crp.png|Image via [Ilya Grigorik/Google](https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/webfont-optimization?hl=en), CC/BY %} In contrast to Typekit, Google Fonts works with a two-step process: * You include an external stylesheet, hosted by Google, in the head tag. This stylesheet includes all the `@font-face` declarations you'll need. The actual font files themselves are linked in this stylesheet. * Using the URLs found in the stylesheet, the fonts are downloaded from Google's servers. Once they're downloaded, the browser renders them in the document. Typekit uses [WebFontLoader](https://github.com/typekit/webfontloader) to load your fonts through an AJAX request. When the browser sends the request for the external stylesheet, Google takes note of what user agent made the request. But why would different browsers need different fonts served? * **Not all font formats are created equal, and browsers require different formats.** Ideally, everyone would support and use WOFF2, the latest open standard. WOFF2 utilizes some awesome compression that can reduce font sizes by up to 30% over the more widely-supported WOFF1. Some browsers (mostly old IE and Safari) require EOT, TTF, even SVG. Google Fonts takes care of all of this *for* you, rather than you having to host and serve each of these formats yourself. * **Google strips out font-hinting information for non-Windows users**{% sidenote 6 "What's font hinting? [Via Wikipedia](https://en.wikipedia.org/wiki/Font_hinting): \"Font hinting (also known as instructing) is the use of mathematical instructions to adjust the display of an outline font so that it lines up with a rasterized grid. At low screen resolutions, hinting is critical for producing clear, legible text.\""%} This is pretty cool. Only Windows usually actually utilizes this information in a font file - Mac and other operating systems have their own "auto-hinting" that ignores most of this information. So, if there is any hinting information in a font file, Google will strip it out for non-Windows users, eliminating a few extra bytes of data. ### Leveraging the power of HTTP caching As I mentioned, Google Fonts are a two-step process: download the (very short) stylesheet from Google, then download the font files from wherever Google tells you. The neat thing is that *these font files are always the same for each user agent*. So if you go to Rubygems.org on a Mac with Chrome, and then navigate to a *different site* that uses the same Google Fonts served Roboto font and weight as we do, you *won't redownload it!* Awesome! And since Roboto is one of the most widely used WebFonts, we can be reasonably expect that at least a minority of visitors to our site *won't have to download anything at all!* Even better, since Roboto is the default system font on Android and ChromeOS, those users won't download anything at all either! Google's CSS puts the *local* version of the font higher up in the font stack: ```css @font-face { font-family: 'Roboto'; font-style: normal; font-weight: 100; src: local('Roboto Thin'), local('Roboto-Thin'), url(https://fonts.gstatic.com/s/roboto/v15/2tsd397wLxj96qwHyNIkxHYhjbSpvc47ee6xR_80Hnw.woff2) format('woff2'); } ``` Google Font's stylesheet has a cache lifetime of 1 day - but the font files themselves have a cache lifetime of 1 year. All in all, this adds up - many visitors to Rubygems.org won't have to download any font data at all! ### Removing render-blocking Javascript One of my main beefs with Typekit (and [webfont.js](https://github.com/typekit/webfontloader)) is that it introduces Javascript into the critical rendering path. Remember - any time the browser's parser encounters a script tag, it must: * Download the script, if it is external (has a "src" attribute) and isn't marked `async` or `defer`. * Evaluate the script. Until it finishes these two things, the browser's parser is *stuck*. It can't move on constructing the page. Rubygems.org's Typekit implementation looked like this: ```html <%= stylesheet_link_tag("application") %> ``` Arrgh! We can't start evaluating this page's CSS until Typekit has downloaded itself and `Typekit.load()` has finished. Unfortunately, if, say, Typekit's servers are slow or are down, `Typekit.load()` will simply block the browser parser until it times out. Ouuccch! This could take your entire site down, in effect, if Typekit ever went down (this has happened to me before - don't be as ignorant as I!). Far better would have been this: ```html <%= stylesheet_link_tag("application") %> ``` At least in this case we can render everything *except* the WebFonts from Typekit. We'll still have to wait around for any of the text to show up until after Typekit finishes, but at least the user will see *some* signs of life from the browser rather than staring at a blank white screen. Google Fonts doesn't use any JavaScript (by default, anyway), which makes it faster than almost any JavaScript-enabled approach. There's really only one case where using Javascript to load WebFonts makes sense - preventing flashes of unstyled text. Certain browsers will immediately render the fallback font (the next font in the font stack) without waiting for the font to download. Most modern browser will instead wait, sensibly, for up to 3 seconds while the font downloads. What this means is that using Javascript (really I mean webfont.js) to load WebFonts makes sense when: * Your WebFonts may reasonably be expected to take more than 3 seconds to download. This is probably true if you're loading 500KB or more of WebFonts. In that case, webfont.js (or similar) will help you keep text hidden for longer while the WebFont downloads. * You're worried about FOUC in old IE or *really* old Firefox/Chrome versions. Simply keeping WebFont downloads fast will minimize this too. ### unicode-range If you look at Rubygems.org in Chrome, Safari, Firefox, and IE, you'll notice something very different in the size of the font download: | Browser | Font Format | Download Size | Difference | | ----- | ---- | --- | --- | | Chrome (Mac) | WOFF2 | 10.0 KB | 1x | | Opera | WOFF2 | 10.0 KB | 1x | | Safari | TrueType | 62.27 KB | 6.27x | | Firefox (Mac) | WOFF | 58.9 KB | 5.89x | | Chrome (Win) | WOFF2 | 14.4 KB | 1.44x | | IE Edge | WOFF | 78.88 KB | 7.88x | What the hell? How is Chrome only downloading 10KB to display our WebFont when Safari and Firefox take almost 6x as much data? Is this some secret backdoor optimization Google is doing in Chrome to make other browsers look bad?! Well, Opera looks pretty good too, so that can't be it (this makes sense - they both use the Blink engine). Is WOFF2 just *that good*? If you take a look at the CSS Google serves to Chrome versus the CSS served to other browsers, you'll notice a crucial difference in the `@font-face` declaration: ```css @font-face { font-family: 'Roboto'; unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000; } ``` What's all this gibbledy-gook? The `unicode-range` property *describes what characters the font supports*. Interesting, right? Rubygems.org, in particular, has to support Cyrillic, Greek and Latin Extended characters. Obviously, normally, we'd have to download extra characters to do that. By telling the browser what characters the font supports, the browser can look at the page, note what characters the page uses, and then *only download the fonts it needs to display the characters actually on the page*. Isn't that awesome? Chrome (and Opera) isn't downloading the Cyrillic, Latin-Extended or Greek versions of this font because it knows it doesn't need to! {% sidenote 7 "[Here's the CSS3 spec on unicode-range for more info](http://www.w3.org/TR/css3-fonts/#unicode-range-desc)." %} Obviously, this particular optimization only really matters if you need to support diferent character sets. If you're just serving the usual Latin set, unicode-range can't do anything for you. There are other ways to slim your font downloads on Google Fonts, though - there's a semi-secret `text` parameter that can be given to Google Fonts to generate a font file that only includes *the exact characters you need*. This is useful when using WebFonts in a limited fashion. This is exactly what I do on this site: ```html ``` This makes the font download required for my site a measly **1.4KB** in Chrome and Opera. Hell yeah. ## But Nate, I want to do it all myself! Yeah, I get it. Depending on Big Bad Google (or any 3rd-party provider) never makes you feel very good. But, let's be realistic: * Are you going to implement `unicode-range` optimization yourself? What if your designer changes fonts? * Are you going to come up with 30+ varieties of the same font, like Google Fonts does, to serve the perfect one to each user agent? * Are you going to strip the font-hinting from your font files to save an extra couple of KB? * What if a new font technology comes out (like WOFF2 did) and even more speed becomes possible? Are you going to implement that yourself? * Are you *absolutely sure* that there's no major benefit afforded by users having already downloaded your font on another site using Google Fonts? There are some very, very strange strategies out there that people use when trying to make WebFonts faster for themselves. There's a few that involve LocalStorage, though I don't see the point when Google Fonts uses the HTTP cache like a normal, respectable webservice. Inlining the fonts into your CSS with data-uri makes intuitive sense - you're eliminating a round-trip or two to Google - but the benefit rarely pans out when compared to the various other optimizations listed above that Google Fonts gets you *for free*. Overall, I think the tradeoff is clearly in Google's favor here. ## TL:DR; * **Do not put Javascript ahead of your stylesheets unless absolutely necessary**. Unfortunately, Typekit only says to "put your embed code near the top of the head tag". If Typekit (or any other font-loading Javascript) is higher up in the `` than your stylesheets, your users will be seeing a blank page *until* Typekit loads. That's not great. * **If you have FOUC problems, either load fewer fonts or use webfonts.js**. Soon, we'll get the ability to control font fallback natively in the browser, but until then, you need to use [WebFontLoader](https://github.com/typekit/webfontloader). It may be worth *inlining* WebFontLoader (or its smaller cousin, [FontFaceObserver](https://github.com/bramstein/fontfaceobserver)) to eliminate a network round-trip. * **Google Fonts does a lot of optimizations you cannot realistically do yourself.** These include stripping font-hinting, serving WOFF2 to capable browsers, and supporting `unicode-range`. In addition, you benefit from *other* sites using Google Fonts which may cause users to have already loaded the font you require! * **Audit your WebFont usage.** Use Chrome DevTools to decipher what's going with your fonts. Use similar system fonts when text is too small to distinguish between fonts. WebFont downloads should almost always be less than 100KB. ## Further Optimization Here are some links for further reading on making WebFonts fast: * [Ilya Grigorik, Optimizing WebFont Rendering Performance](https://www.igvita.com/2014/01/31/optimizing-web-font-rendering-performance/) * [Adam Beres-Deak, Loading webfonts with high performance on responsive websites](http://bdadam.com/blog/loading-webfonts-with-high-performance.html) Using LocalStorage to store and serve WebFonts. Try this one in your browser with Chrome Timeline open - it performs far worse than Google Fonts on first load. * [Patrick Sexton, Webfont options and speed](https://web.dev/articles/font-best-practices) Great overview of the multitude of options available to you outside of Google Fonts. * [Filament Group, Font Loading Revisited](https://www.filamentgroup.com/lab/font-events.html) --- ## Page Weight Doesn't Matter URL: https://www.speedshop.co/blog/page-weight-doesnt-matter/ There's one universal law of front-end performance - **less is more**. Simple pages are fast pages. We all know this - it isn't controversial. Complexity is the enemy. And yet, it's trivial to find a website whose complexity seems to reach astronomical levels. {% sidenote 1 "Literally. The Apollo Guidance Computer had just 64 KB of ROM, but most webpages require more than 1MB of data to render. There are some webpages that are actually 100x as complex as the software that took us to the moon." %} It's perhaps telling that media and news sites tend to be the worst here - most media sites in 2015 take ages to load, not to mention all the time you spend clicking past their paywall popups (NYTimes) or full-page advertisements (Forbes). {% marginnote_lazy https://i.imgur.com/lnAzS1o.jpg|"Dear Adobe: Flash is a dumpster fire. Love, Steve." %} Remember when [Steve Jobs said Apple's mobile products would never support Flash?](https://web.archive.org/web/20100130044557/http://www.apple.com/hotnews/thoughts-on-flash/) For a year or two there, it was a bit of a golden age in web performance - broadband was becoming widespread, 4G started to come on the scene, and, most importantly, websites started dropping Flash cruft. The "loading!" screens and unnecessarily complicated navigation schemes became something of yesteryear. That, is, until the marketing department figured out how to use Javascript. The Guardian's homepage sets advertising tracking cookies across 4 different partner domains. Business Insider thought to one-up their neighbors across the pond and sets **cookies across 17 domains**, requires **284 requests** (to nearly 100 unique domains) and a **4.9MB download** which took a full *9 seconds* to load on my cable connection, which is a fairly average broadband ~20 megabit pipe. {% marginnote_lazy https://i.imgur.com/L8K5kUM.gif|"They think Business Insider is a news site and not just an ad delivery mechanism? That's rich!"|true %} Business Insider is, ostensibly, a news site. The purpose of the Business Insider is to deliver text content. Why does that require 5 MB of *things which are not text*? Unfortunately, it seems, the cry of "complexity is the enemy!" is lost on the ones setting the technical agenda. While trying to load every single tracking cookie possible on your users, you've steered them away by making your site slow on *any* reasonable broadband connection, and nearly *impossible* on any mobile connection. Usually, the boogeyman that gets pointed at is *bandwidth*: users in low-bandwidth areas (3G, developing world) are getting shafted. {% marginnote_lazy https://i.imgur.com/AU7LLZi.jpg|
4 divided by 20 isn't 9...|true %}But the math doesn't *quite* work out. Akamai puts the global connection speed average at **3.9 megabits per second**. So wait a second - why does Business Insider take 9 seconds to load on my 20 megabit pipe, when it's only 4.9MB? If I had an average connection, according to Akamai, shouldn't Business Insider load in 2 seconds, tops? The secret is that "page weight", broadly defined as the simple total file size of a page and all of it's sub-resources (images, CSS, JS, etc), isn't the problem. **Bandwidth is not the problem, and the performance of the web will not improve as broadband access becomes more widespread.** The problem is latency. Most of our networking protocols require a lot of round-trips. Each of those round trips imposes a latency penalty. Latency is governed, at the end of the day, by the speed of light. Which means that latency *isn't going anywhere*. DNS lookup is, and always will be, expensive.{% sidenote 2 "I'm being facetious, of course. In 10 years, we may have invented some better protocols here. But it's fair to say we have to live with the current reality for at least a decade. Look at how long it's taking us to get on board with IPv6."%} TCP connections are, and always will be, expensive. SSL handshakes are, and always will be, expensive. We're going to be doing more of them over the next 10 years. Thanks NSA. Each of these things requires at least one *network round-trip* - that is, a packet going from your computer, across the network, to someone else's. That will never be faster than the speed of light - and even light takes 30 milliseconds to go from New York to San Francisco and back. {% sidenote 3 "Thanks to the amount of hops a packet has to make across the internet backbone, usually the time is much worse - 2-4x." %} What's worse is that these network round-trips must happen sequentially - we have to know the IP address before we start the three-way handshake for TCP, and we have to establish a TCP connection before we can start to negotiate SSL. Setting up a typical HTTPS connection can involve *5.5 round-trips*. That's like 165 milliseconds {% sidenote 4 "In the hypothetical NY-to-SF scenario. Usually it's better than this in the US because of CDNs. But 150ms per connection isn't a bad rule of thumb - and on mobile it's much worse, closer to 300." %} per connection *on a really really good day*. The smart ones among you may already see the solution - well, Nate, 165 milliseconds per connection isn't a problem! We'll just parallelize the connections! Boom! 100 connections opened in 165 milliseconds! The problem is that HTML *doesn't work this way by default*. {% marginnote_lazy https://i.imgur.com/mHImMLs.png|Business Insider's network utilization over time - hardly pegged at 100%. %} We'd like to imagine that the way a webpage loads is this: 1. Browser opens connection to yoursite.com, does DNS/TCP/SSL setup. 2. Browser downloads the document (HTML). 3. As soon as the browser is done downloading the document, the browser starts downloading all the document's sub resources *at the same time*. 4. Browser parses the document and fills in the necessary sub resources once they've been downloaded. Here's what actually happens: 1. Browser opens connection to yoursite.com, does DNS/TCP/SSL setup. 2. Browser downloads the document (HTML). 3. Browser starts parsing the document. When the parser encounters a subresource, it opens a connection and downloads it. {% marginnote_lazy https://i.imgur.com/vR44K2h.jpg|Parse the document? Nah man, I'm gonna wait for this script to download and execute.|true %} If the subresource is an external script tag, the parser stops, waits until it the script has downloaded, executes the entire script, and then moves on. 4. As soon as the parser stops and has to wait for an external script to download, it sends ahead something called a *preloader*. The preloader *may* notice and begin downloading resources *if* it understands how to (hint: a very popular Javascript pattern prevents this). Thanks to these little wrinkles, web page loads often have new connections opening *very* late in a page load - right before the end even! Ideally, the browser would open all of those connections like in our first scenario - immediately after the document is downloaded. We want to maximize network utilization across the life of the webpage load process. There's four ways to accomplish this: * **Don't stop the parser.** * **Get out of the browser preloader's way**. * **Utilize HTTP caching - but not *too* much**. * **Use the Resource Hint API**. ## Glossary I'm going to use a couple of terms here and I want to make sure we're all on the same page. * Connection - A "connection" is one TCP connection between a client (your browser) and a server. These connections can be re-used across multiple requests through things like [keep-alive](https://en.wikipedia.org/wiki/HTTP_persistent_connection). * Request - A browser "requests" resources via HTTP. 99% of the time when we're talking about requesting resources, we're talking about an HTTP GET. Each request needs to use a TCP connection, though not necessarily a unique or new one (see [keep-alive](https://en.wikipedia.org/wiki/HTTP_persistent_connection)). * Subresource - In browser parlance, a subresource is generally any resource required to completely load the main resource (in this case, the document). Examples of subresources include external Javascript (that is, `script` tags with a `src` attribute), external CSS stylesheets, images, favicons, and more. * Parser - When a browser tries to load your webpage, it uses a parser to read the document and decide what sub resources need to be fetched and to construct the DOM. The parser is responsible for getting the document to one of the first important events during a page load, DOMContentLoaded. ## Letting the Preloader do it's Job Sometimes the parser has to stop and wait for an external resource to download - 99% of the time, this is an external script. When this happens, the browser starts something called a preloader. The preloader is a bit like a "parser-lite", but rather than construct the DOM, the preloader is more like a giant regex that searches for sub resources to download. If it finds a subresource (say an external script at the end of the document), it will start downloading it *before* the parser gets to it. You may be thinking this is rather ridiculous - why should a browser stop completely when it sees an external script tag? Well, thanks to The Power of Javascript, that external script tag *could* potentially wreak havoc on the document if it wanted. Heck, it could completely erase the entire document and start over with `document.write()`. The browser just doesn't know. So rather than keep moving, it has to wait, download, and execute. {% sidenote 5 "All in the HTML spec." %} Browser preloaders were a huge innovation in web performance when they arrived on the scene. Completely unoptimized webpages could speed up by 20% or more just thanks to the preloader fetching resources! That said, there are ways to help the preloader and there are ways to hinder it. We want to help the preloader as much as possible, and sometimes we want to stay the hell out of it's way. ### Stop inserting scripts with "async" script-injection {% marginnote_lazy https://i.imgur.com/G3DhZwf.gif|It's just one more script tag!|true" %} The marketing department says you need to integrate your site with SomeBozoAdService. They said it's really easy - you just have to "add five lines of code!". You go to SomeBozoAdService's developer section, and find that they tell you to insert this into your document somewhere: ```javascript var t = document.createElement('script'); t.src = "//somebozoadservice.com/ad-tracker.js"; document.getElementsByTagName('head')[0].appendChild(script); ``` There are other problems with this pattern (it blocks page rendering until it's done, for one), but here's one really important one - browser preloaders can't work with this. Preload scanners are *very* simple - they're simple so that they can be fast. And when they see one of these async-injected scripts, they just give up and move on. So your browser can't download the resource until the main parser thread gets to it. Bummer! It's far better to use `async` and `defer` attributes on your script tags instead, to get this: ```html ``` Kaboom! There are some other advantages to `async` that I get into in [this other post here](/blog/hacking-head-tags-for-speed-and-profit/), but be aware that one of them is that the browser preloader can get started downloading this script before the parser even gets there. Here's a list of other things that generally don't work with browser preloaders: * IFrames. Sometimes there's no way around using an iframe, but if you have the option - try not to. The content of the frame can't be loaded until the parser gets there. * @import. I'm not sure of anyone that uses @import in their production CSS, but don't. Preloaders can't start fetching `@import`ed stylesheets for you. * {% marginnote_lazy https://i.imgur.com/oL7MkI0.jpg|Design department: \"But we need these 90 fonts to spice up the visual interest of the page!\" %} Webfonts. Here's an interesting one. I could write a whole article on webfont speed (I should/will!), but they usually aren't preloaded. This is fixable with resource hints (we'll get to that in a second). * HTML5 audio/video. This is also fixable with resource hints. I've heard that in the past, preloaders wouldn't scan the body tag when blocked in the head. If that was ever true, it is no longer true in Webkit based browsers. In addition, modern preloaders are smart enough not to request resources that are already cached. Speaking of HTTP caching... ## HTTP caching The fastest HTTP request is the one that is never made. That's really all HTTP caching is for - preventing unnecessary requests. Cache control headers are really for telling clients "Hey - this resource, it's not going to change very quickly. Don't ask me again for this resource until..." That's awesome. We should do that everywhere possible. [Yet, the size of the resource cache is smaller than you might think.](https://web.archive.org/web/20160201000000/http://www.guypo.com/mobile-browser-cache-sizes-round-2/) Here's the default disk cache size in modern browsers: {% marginnote_lazy https://i.imgur.com/E0yJ6HR.jpg||true %} | Browser | Cache Size (default) | | -------- | -------- | | Internet Explorer 9+ | ~250MB | | Chrome | 200MB | | Firefox | 540MB | | Mobile Safari | 0 | | Android (all) | ~25-80 MB | Not as large as you might imagine. And you read that right - Mobile Safari does not have a persistent, on-disk cache. Most browser resource caches work on an LRU basis - last recently used. So if something doesn't get used in the cache, it's the first thing to be evicted if the cache fills up. A pattern I've often seen is to use 3rd-party, CDN-hosted copies of popular libraries in an attempt to leverage HTTP caching. The idea is to use Google's copy of JQuery (or what have you), and a prospective user to your site will already have it downloaded before coming to yours. The browser will notice it's already in their cache, and not make a new request. There's some other benefits, but I want to pick on this one. This sounds good in theory, but given the tiny size of caches, I'm not sure if it really works in practice. Consider how few sites actually use Google-hosted (or Cloudflare-hosted, or whatever) JQuery. Even if they did - how often is your cached copy pushed *out* of the cache by other resources? Do you know? Consider the alternative - bundling JQuery into your application's concatenated "application.js" file (Rails' default behavior). In the best case, the user already has the 3rd-party CDN-hosted JQuery downloaded and cached. The request to go and get your application.js doesn't take *quite* as long because it's ~20kb smaller now that it doesn't include JQuery. But remember what we said above - bandwidth is hardly the issue for most connections (saving 20kb is really saving less than 100ms, even on a 2MB/s DSL connection). But consider the worst case scenario - the user doesn't have our 3rd-party JS downloaded already. Now, compared to the "stock" application.js scenario, you have to make an additional new connection to a new domain, likely requiring SSL/TLS negotiation. Without even downloading the script, you've been hit with 1-300ms of network latency. Bummer. Consider how much worse this gets when you're including more than 1 library from an external CDN. God forbid that the script tags aren't `async`, or your user will be sitting there for a while. In conclusion, 3rd-party hosted Javascript, while a good idea and, strictly speaking, faster in the best-case scenario, is likely to impose a huge performance penalty to users that don't have every single one of your 3rd-party scripts cached already. Far preferable is to bundle it into a single "application.js" file, served from your own domain. That way, we can re-use the already warm connection (as long you allowed the browser to "keep-alive" the connection it used to download the document) to download all of your external Javascript in one go. ## Resource hints There's another way we can maximize network utilization - through something called *resource hints*. There are couple of different kinds of resource hints. In general, most of them are telling the browser to *prepare some connection or resource in advance* of the parser getting to the actual point where it needs the connection. This prevents the parser from blocking on the network. * **DNS Prefetch** - Pretty simple - tell the browser to resolve the DNS of a given hostname (`example.com`) as soon as possible. * **Preconnect** - Tells the browser to open a connection as soon as possible to a given hostname. Not only will this resolve DNS, it will start a TCP handshake and perform TLS negotiation if the connection is SSL. * **Prefetch** - Tells to browser to download an entire resource (or subresource) that may be required later on. This resource can be an entire HTML document (for example, the next page of search results), or it can be a script, stylesheet, or other subresource. The resource is only downloaded - it isn't parsed (if script) or rendered (if HTML). * **Prerender** - One of these things is not like the other, and prerender is it. Marking an `` tag with `prerender` will actually cause the browser to get the linked `href` page and *render it before the user even clicks the anchor!* This is the technology behind Google's Instant Pages and Facebook's Instant Articles. It's important to note that all of these are *hints*. The browser may or may not act upon them. Most of the time, though, they will - and we can use this to our advantage. **Browser support**: I've detailed which browsers support which resource hints (as of November 2015) below. However, any user agent that doesn't understand a particular hint will just skip past it, so there's no harm in including them. Most resource hints enjoy >50% worldwide support (according to to [caniuse.com](http://www.caniuse.com)) so I think they're definitely worth including on any page. Let's talk about each of these items in turn, and when or why you might use each of them: ## DNS Prefetch ```html ``` In case you're brand new to networking, here's a review - computers don't network in terms of domain names. Instead, they use IP addresses (like `192.168.1.1`, etc). They *resolve* a hostname, like `example.com`, into an IP address. To do this, they have to go to a DNS server (for example, Google's server at `8.8.8.8`) and ask: "Hey, what's the IP address of `some-host.com`?" This connection takes time - usually somewhere between 50-100ms, although it can take much longer on mobile networks or in developing countries (500-750ms). **When to Use It:** {% marginnote_lazy https://i.imgur.com/iTlcW8x.gif|\"Stop trying to make dns-prefetch a thing!\"|true %} But you may be asking - why would I ever want to resolve the DNS for a hostname and *not actually connect to that hostname*? Exactly. So forget about `dns-prefetch`, because it's cousin, `preconnect`, does exactly that. **Browser Support**: Everything except IE 9 and below. ## Preconnect ```html ``` A preconnect resource hint will hint the browser to do the following: * Resolve the DNS, if not done already (1 round-trip) * Open a TCP connection ([1.5 round-trips](https://blog.packet-foo.com/2014/07/determining-tcp-initial-round-trip-time/)) * Complete a TLS handshake if the connection is HTTPS ([2-3 round-trips](https://zoompf.com/blog/2014/12/optimizing-tls-handshake)) The only thing it won't do is actually download the (sub)resource - the browser won't start loading the resource until either the parser or preloader tries to download the resource. This can eliminate up to 5 round-trips across the network! That can save us a heck of a lot of time in most environments, even fast home Wifi connections. **When to Use It:** Here's an example from [Rubygems.org](https://rubygems.org). Taking a look at how Rubygems.org loads in [webpagetest.org](https://www.webpagetest.org), we notice a few things. What we're looking for is network utilization after the document is downloaded - once the main "/" document loads, we should see a bunch of network requests fire at once. Ideally, they'd all fire off at this point. In a perfect world, network utilization would look like a flat line at 100%, which then stops as soon as the page loads completely. Preconnect helps us to do that by allowing us to move some network tasks earlier in the page load process. Notice these these two resources, closer to the end of the page load: Two are related to gaug.es, an analytics tracking service, and the other is a GIF from a Typekit domain. The green bar here is time-to-first-byte - time spent waiting for a server response. But note how the analytics tracking service and the Typekit GIF have teal, orange, and purple bars as well - these bars represent time spent resolving DNS, opening a connection, and negotiating SSL, respectively. By adding a preconnect tag to the head of the document, we can move this work to the beginning of the page load, so that when the browser needs to download these resource it has a pre-warmed connection. That loads each resource ~200ms faster in this case. You may be wondering - why hasn't the preloader started loading these resources earlier? In the case of the gang.es script, it was loaded with an "async" script-injection tag. This is why that method is a bit of a stinker. For more about why script-injection isn't a great idea, see Ilya Grigorik's post on the topic. So in this case, rather than adding a `preconnect` tag, I'll simply change the gaug.es script to a regular script tag with an `async` attribute. That way, the browser preloader will pick it up and download it as soon as possible. In the case of that Typekit gif, it was also script-injected into the bottom of the document. A `preconnect` tag would speed up this connection. However, `p.gif` is [actually a tracking beacon for Adobe](https://www.leaseweb.com/labs/2015/03/ghostery-blocks-adobe-typekit-hosted-fonts/), so I don't think that speeding that up will provide any performance benefit to the user. In general, `preconnect` works best with sub resources that are script-injected, because the browser preloader cannot download these resources. Use webpagetest.org to seek out sub resources that load late and trigger the DNS/TCP/TLS setup cost. In addition, it works very well for script-injected resources with dynamic URLs. You can set up a connection to the domain, and then later use that connection to download a dynamic resource (like the Typekit example above). See the W3C spec: > The full resource URL may not be known until the page is being constructed by the user agent - e.g. conditional loading logic, UA adaptation, etc. However, the origin from which one or more of these resources will be fetched is often known ahead of time by the developer or the server generating the response. In such cases, a preconnect hint can be used to initiate an early connection handshake such that when the resource URL is determined, the user agent can dispatch the request without first blocking on connection negotiation. **Browser Support**: Unfortunately, preconnect is probably the least-supported resource hint. It only works in *very* modern Chrome and Firefox versions, and is coming to Opera soon. Safari and IE don't support it. ## Prefetch ```html ``` {% marginnote_lazy https://i.imgur.com/gQq7Lru.gif|Go get the resource, Chrome! Go get it, boy!|true %} A prefetch resource hint will hint the browser to do the following: * Everything that we did to set up a connection in the `preconnect` hint (DNS/TCP/TLS). * But in addition, the browser will also *actually download the resource*. * However, `prefetch` only works for resources required by *the next navigation*, not for the *current page*. **When to Use It:** Consider using `prefetch` in any case where you have a good idea what the user might do next. For example, if we were implementing an image gallery with Javascript, where each image was loaded with an AJAX request, we might insert the following prefetch tag to load the next image in the gallery: ```html ``` You can even prefetch entire pages. Consider a paginated search result: ```html ``` **Browser Support**: IE 11 and up, Firefox, Chrome, and Opera all support `prefetch`. Safari and iOS Safari don't. ## Prerender Prerender is prefetch on steroids - instead of just downloading the linked document, it will actually pre-render the entire page! Obviously, this means that pre rendering only works for HTML documents, not scripts or other subresources. This is a great way to implement something like Google's Instant Pages or Facebook's Instant Articles. Of course, you have to be careful and considerate when using prefetch and prerender. If you're prefetching something on your own server, you're effectively adding another request to your server load for every prefetch directive. A prerender directive can be even more load-intensive because the browser will also fetch all sub resources (CSS/JS/images, etc), which may also come from your servers. It's important to only use prerender and prefetch where you can be pretty certain a user will actually use those resources on the next navigation. There's another caveat to prerender - like all resource hints, prerenders are given much lower priority by the browser and aren't always executed. [Here's straight from the spec](http://www.w3.org/TR/resource-hints/#speculative-resource-prefetching-prefetch): "The user agent may: * Allocate fewer CPU, GPU, or memory resources to pre rendered content. * Delay some requests until the requested HTML resource is made visible - e.g. media downloads, plugin content, and so on. * Prevent pre rendering from being initiated when there are limited resources available." **Browser Support**: IE 11 and up, Chrome, and Opera. Firefox, Safari and iOS Safari don't get this one. ## Conclusion We have a long way to go with performance on the web. I scraped together a little script to check the Alexa Top 10000 sites and look for resource hints - here's a quick table of what I found. | Resource Hint | Prevalence | | --------------|------------| | `dns-prefetch` | 5.0% | | `preconnect` | 0.4% | | `prefetch` | 0.4% | | `prerender` | 0.1% | So many sites could benefit from liberal use of some or all of these resource hints, but so few do. Most sites that do use them are just using `dns-prefetch`, which is practically useless when compared to the superior `preconnect` (how often do you really want to know the DNS resolution of a host and then *not* connect to it?). {% marginnote_lazy https://i.imgur.com/cchNrOn.gif||true %} I'd like to back off from the flamebait-y title off this article *just* slightly. Now that I've explained all of the different things you can do to increase network utilization during a webpage load, know that 100% utilization isn't always possible. Resource hints and the other techniques in this article *help* complex pages load faster, but thanks to many different constraints you may not be able to apply them in all situations. Page weight *does* matter - a 5MB page will be more difficult to optimize than a 500 KB one. What I'm really trying to say is that page weight *only sorta* matters. I hope I've demonstrated to you that page weight - while certainly *correlated* with webpage load speed, is not the final answer. You shouldn't feel like your page is doomed to slowness because The Marketing People need you to include 8 different external ad tracking services (although you should consider quitting your job if that's the case). **TL;DR:** * Don't inject scripts. * Reduce the number of connections required *before* reducing page size. * HTTP caching is great, but don't *rely* on any particular resource being cached. * Use resource hints - especially `preconnect` and `prefetch`. --- ## Hacking Your Webpage's Head Tags for Speed and Profit URL: https://www.speedshop.co/blog/hacking-head-tags-for-speed-and-profit/ {% marginnote_lazy https://i.imgur.com/2K3eIZI.gif|\"What's that? The site takes 15 seconds to load on mobile?
Sorry, but Marketing says I gotta put Mixpanel in here first.\"|true %} Most of us developers settle for page load times somewhere between 3 and 7 seconds. We open up the graph in NewRelic or [webpagetest.org](http://webpagetest.org), sigh, and then go back to implementing that new feature that the marketing people *absolutely must have deployed yesterday*. Little do we realize, perceived front-end load times closer to half a second are possible for most (if not all) websites with very little effort. Most webpages have slow frontend load times not because they're heavy (north of 1MB), or because they need 200kb of Javascript just to render a "Hello World!" (*cough Ember cough*). It isn't because the pipes are too small either - bandwidth is really more than sufficient for the Web today. **HTML, TCP and latency are the problems, not bandwidth**. Page weight, while important, is a false idol. A 1MB webpage, with all of it's scripts and CSS inlined, will load faster than 1 MB webpage with 100 different asset requests spread across 10 domains. Each of these asset requests requires a TCP connection, and setting up those connections takes longer when there's more network latency. This is really TCP's fault - it was designed for long, streaming downloads, not the machine-gun fire of 3rd-party Javascript and assets that most websites today require. God forbid you're in a high-latency environment too, like a mobile connection or a developing country. When latency starts to shoot north of 100 milliseconds, webpages grind to a halt trying to set up dozens of [three-way handshakes](https://support.microsoft.com/en-us/kb/172983) to download all of the cat gifs your social media intern said would *totally blow up* this blog post on Reddit. In addition, some quirks in how HTML works means that certain subresources {% sidenote 1 "Sub-resource is a fancy word for another the HTML document needs - images, stylesheets, fonts, scripts, video and audio are all subresources." %} *must* block page rendering - leaving the browser idling, waiting for things to download and execute. Preventing (and dealing with) the various types of blocking that can happen during a webpage load presents a major performance opportunity. The problem of webpage loading is generally not a problem of resources, it's a problem of using those resources efficiently so that they don't block each other's execution. Thankfully, **humans are squishy, and perceived load times are not the same as window load times**. We can hack our user's perceptions to make them *think* the webpage loaded faster than it did. `window.load`, while a good starter metric for measuring page load speed, is not a realistic interpretation of how users look at webpages. Humans (unlike computers) can begin to understand the webpage before it's even finished completely loading. This means that *time to paint*, not *time to load* is important. In addition, *time to paint the page's usable content* is of course the most important thing. Gmail quickly paints a loading bar, sure, but you didn't come to Gmail to see the loading bar. You came to see the application. Likewise, if our news website paints some divs to the page but doesn't actually show any text until 2 seconds later because the web fonts took forever to load, then the site wasn't really usable until that text was painted. Thankfully, it's easier to decrease *perceived* load times than it is to decrease *total* load time (as measured by `window.load`). {% marginnote_lazy https://i.imgur.com/I8Zmwht.jpg %} Amazon, for example, paints a nearly complete page just 1.5 seconds after a request is sent, but `window.load` doesn't fire until 3.5 seconds later. We can leverage human perception to disproportionately affect perceived load times with minimal effort. And the place these opportunities can be exploited is in a site's `head` tag. The `head` tag is probably the most important part of any webpage from a performance standpoint. It can truly make or break a speedy page - two identical head tags with different element ordering can have speed differences on an order of magnitude, especially in poor network conditions (like mobile or the developing world). But sometimes optimizing head tags can be confusing - there's a lot to understand and browser technology changes rapidly, meaning yesterday's advice can be out of date. In this article, I'll attempt to show what the optimal head tag looks like - what elements in contains, in what order, and with what special attributes (such as `async` and `defer`) that will lead to zippy-quick load times. First, some definitions. What *exactly* are we going to optimize for? When thinking about page load optimization, there are usually three important times for the end user: * **First paint** - When does the page first start painting to the screen? This doesn't have to be *all* the content - frequently it looks like just a few colored `div` blocks with no text in them (waiting for the fonts to load). Images are usually not loaded yet. Heck, we may not even have downloaded the CSS for anything below the fold yet (the initial viewport - more on that later). But this time is still important - it's when a user first sees a reaction to their input. Decreasing time-to-first-paint can be a critical optimization in improving user *perception* of page loads. This is why [Facebook hacks the JPEG algorithm to send a blurred, 200 byte version of cover photos on mobile](https://code.facebook.com/posts/991252547593574). Creating a *perception* of the page loading is just as important as the page *actually* loading. * **First paint of text content** - Webpages are text-delivery mechanisms. The Web is typography. When does the page start painting text to the screen? As soon as a page's critical text has been painted - before the images have been downloaded or even any decorative elements rendered - the user can begin processing the information on the screen. And not all text content is equal here - painting "Loading..." to the screen doesn't count.{% marginnote_lazy https://i.imgur.com/yYIJraq.gif|Typical user reaction to loading screens.|true %} A user cannot begin to *do what they came to your website to do* until the text on that page has painted to the screen, making the moment that text appears one of the most important of your website's loading process. This time can often be substantially different than time to first paint, for reasons I'll get into later on. This is a pet theory of mine, and I am not a designer or information architect by trade, so take this all with a grain of salt. * **The `load` event** - The `load` event is the last major event the browser fires during a webpage load. It signals that the browser has loaded *all* images, stylesheets, and scripts. Usually (though not necessarily) the page is stable by this point and doesn't change. We can say that when `load` has executed, the page is done loading. However, in reality, the two times above are much more important for a user's perception of page loads. [Above-the-fold render time is so Web 2.0](https://web.archive.org/web/20260209215205/https://www.stevesouders.com/blog/2013/05/13/moving-beyond-window-onload/). Our optimal `head` tag will try to optimize *all* of these times. It's important to note that often you'll be presented with a tradeoff - you can decrease time to first paint by increasing time to load, and vice versa. I'm going to point out these tradeoffs, but generally I'm going to prefer to decrease time to first text paint. ## Encoding {% marginnote_lazy http://i.imgur.com/kWudACZ.gif|\"You get used to it. I don't even see the code anymore. All I see is cat gif, BuzzFeed listicle, Facebook status...\"|true" %} Here's an easy optimization to start us off. When a browser downloads your page off the network, it's just a stream of bits and bytes, and the browser doesn't really know what character encoding you used. Before it can read the data, it needs to decide on a character encoding to use to read the document. 99.9% of the time on the web, we do this with UTF-8, but that isn't guaranteed. The browser has to decide what *character encoding to use*. There's a couple of ways it can do this (fastest first): * **The `Content-Type` HTTP header** By putting the document's character encoding right in the response headers, you're ensuring that the browser sets the right character encoding before it even tries to parse the document. This is perfect. * **`meta` tag** This is probably the most common option. For example, [Bootstrap's example page does this](http://getbootstrap.com/getting-started/#template). If you do this, it's important that it's the very first element in the `head`. If the browser starts reading the document with a different encoding (old IE will sometimes use some weird Windows encoding), it has to go back to the beginning and restart. * **Guessing** If there's no `meta` tag, and no HTTP header, the browser will try to guess, using things like byte ordering characters. Of course, there are obvious compatibility issues there (and only God knows what old IE will guess), but it's also probably the slowest of all the options. `X-UA-Compatible` is very similar to character encoding - we want as high up in the document as possible because if you specify a value that's different than what the browser is already using to parse the document, you'll restart the rendering process. If you have to specify a X-UA-Compatible value, here's some tips: * If you're specifying `X-UA-Compatible` and the value is just "IE=edge", [that may be unnecessary](http://stackoverflow.com/questions/26346917/why-use-x-ua-compatible-ie-edge-anymore). Remove it unless a) you think your site will be used on an intranet b) you're not a top-10000 site that might get added to [Microsoft's compatibility list](https://learn.microsoft.com/en-us/deployedge/edge-ie-mode-site-list-manager). {% marginnote_lazy https://i.imgur.com/XfG2mTw.gif|Internet Explorer's reaction to IE=edge|true" %} * If you can, specify `X-UA-Compatible` in an HTTP header, not in the document itself. This is faster for the same reasons as it is for character encoding, above. * If it has to be in the document, put `X-UA-Compatible` as high up as you can, specifically within the first 4KB of the response. IE10 and above will [speculatively prescan the first 4KB of the document](https://web.archive.org/web/20161220140918/https://blogs.msdn.microsoft.com/ieinternals/2011/07/18/best-practice-get-your-head-in-order/) looking for an `X-UA-Compatible` tag. Putting it lower on the page will cause page rendering to *stop and restart*. Ouch. ## Viewports Here's another one. If you're going to specify a `viewport` size, do it at the very top of the `head`. Why? Browsers translate this: ```html ``` ...into this: ```html ``` While [the spec for how this works is still unfinished](http://www.w3.org/TR/css-device-adapt/#translation-into-viewport-properties), you can bet that most browsers already implement it this way. There's a problem with this - if you put the `viewport` meta tag *after* your stylesheets, you will cause [a layout reflow](https://developers.google.com/speed/articles/reflow?hl=en) for the entire document, slowing down rendering. Don't do that. Keep your viewport tags at the top, right after your character encoding. In addition, putting a viewport tag at the bottom of the head will almost certainly cause a "flash of unstyled content" as the CSS is first loaded in the default viewport, then re-rendered in your specified viewport. ## Concatenation of Assets TCP isn't really designed for short bursts. It's got a load of overhead, and needs a lot of back-and-forth just to set up a connection. {% marginnote_lazy https://i.imgur.com/CrC5D2x.gif||true %} Despite this, the top 1000 websites in the world *on average* require 31-40 TCP connections. I'm sure all of them are important, and aren't [advertisements](https://www.google.com/adwords/), [creepy 3rd-party trackers](http://www.mediamath.com/), or [bloatware](https://jquery.com/)! Surely, all of those requests are for absolutely necessary subresources and not a single one could be eliminated. Alright, jokes aside, here's the scoop. Opening a new TCP connection is slow - it's especially slow if you're asking for content from a different domain (you might need to resolve DNS, negotiate TLS, and more). Minimize new connections where you can. One of the easiest places to do this is by concatenating your assets. Although the Rails asset pipeline has been a constant source of headache for beginner Rails developers, it is absolutely one of the best performance optimizations that the framework provides. Concatenate all of your site's stylesheets and scripts into one file each. It's 2015. There's no excuse. {% sidenote 2 "Yes, I know all of this will change when HTTP2 becomes widespread. But it isn't yet, and might not be for at least another year or two. If you're living a magical fairy land where you already get to use HTTP2 in production, go read someone else's guide on that." %} If you've got a lot of images, it may be time to start thinking about image sprites or an icon font. All of this can be benchmarked in the wonderful Chrome Network tab - try different configurations and watch the results. ## Async Defer I'm a Ruby guy, but I hear those Javascript people talking about "async" stuff a lot. It seems like the cool thing these days - everything is "asynchronous" and "non-blocking"! But I live in Ruby land, and most things in our applications are synchronous and blocking. Gee, thanks GIL. Ordinarily, script tags with an external `src` attribute (that is, not inlined) are synchronous and blocking too. ```html ``` When this tag is in the head, the browser *cannot proceed with rendering the page* until it has *downloaded* and *executed* the script. This can be very slow, and even if it isn't, if you do it 6-12 times on one page it will be slow anyway (thanks TCP!). [Here's an example you can test in your own browser](https://web.archive.org/web/20151101000000/http://stevesouders.com/cuzillion/?c0=hc1hfff2_0_f&c1=hj1hfff2_0_f&c2=bi1hfff2_0_f&c3=bi1hfff2_0_f&c4=bi1hfff2_0_f&t=1445441057). Ouch, right? {% sidenote 3 "While the browser cannot proceed with rendering the page (and therefore painting anything to the screen) until it's finished executing the script, it CAN download other resources further on in the document. This is accomplished with the browser preloader, something I'll get in to next week." %} You may be thinking this is rather ridiculous - why should a browser stop completely when it sees an external script tag? Well, thanks to The Power of Javascript, that external script tag *could* potentially wreak havoc on the document if it wanted. Heck, it could completely erase the entire document and start over with `document.write()`. The browser just doesn't know. So rather than keep moving, it has to wait, download, and execute. {% sidenote 4 "All in the HTML spec." %} However, in the world of front-end performance, I'm not so restricted! This is not the only way! There's an `async` attribute that can be added to any `script` tag, like so: ```html ``` And *bam!* instantly that entire Javascript file is made _**magically asynchronous**_ right? Well, no. The `async` tag just tells the browser that this particular script *isn't required to render the page*. This is perfect for most 3rd-party marketing scripts, like Google Analytics or Gaug.es. In addition, if you're really good (and you're not a Javascript single-page-app), you may be able to make every single external script on your page `async`. `async` downloads the script file without stoppping parsing of the document - the script tag is no longer *synchronous* with the There's also this `defer` attribute, which has slightly different effects. What you need to know is that Internet Explorer 9 and below doesn't support `async`, but it does support `defer`, which provides a similar functionality. It never hurts to just add the `defer` attribute after `async`, like so: ```html ``` That way IE9 and below will use `defer`, and everyone who's using a browser from after the Cold War will use `async`. [Here's a great visual explanation of the differences between async and defer](http://www.growingwiththeweb.com/2014/02/async-vs-defer-attributes.html). So **add `async defer` to every script tag that isn't required for the page to render**. {% sidenote 5 "The caveat is that there's no guarantee as to the order that these scripts will be evaluated in when using async, or even when they'll be evaluated. Even defer, which is *supposed* to execute scripts in order, sometimes won't (bugs, yay). Async is hard." %} ## Stylesheets first You may have a few non-`async` script tags remaining at this point. Webfont loaders, like Typekit, are a common one - we need fonts to render the page. Some *really* intense marketing JS, like Optimizely, should probably be loaded before the page renders to avoid any flashes of unstyled content as well. **Put any CSS before these blocking script tags.** ```html ``` There's no `async` for stylesheets. This makes sense - we need stylesheets to render the page. But if we put CSS (external or inlined) after an external, blocking script, the browser can't use it to render the page until that external script has been downloaded and executed. This may cause flashes of unstyled content. The most common case is the one I gave above - web fonts. A great way to manage this is with CSS classes. While loading web fonts with Javascript, TypeKit (and many other font loaders) apply a CSS class to the body called `wf-loading`. When the fonts are done loading, it changes to `wf-active`. So with CSS rules like the below, we can hide the text on the page until we've finished loading fonts: ```css .wf-loading p { visibility: hidden } ``` While text is the most important part of a webpage, it's better to show some of the page (content blocks, images, background styles) than none of it (which is what happens when your external scripts come before your CSS). ## Conclusion To wrap up my recommendations from this article: * Specify content encoding with HTTP headers were possible, otherwise do it with meta tags at the *very top* of the document. * If using `X-UA-Compatible`, put that as far up in the document as possible. * `` tags should go right below any encoding tags. * Concatenate your assets. * `async defer` all the script tags. * Stylesheets before blocking (non-`async`) scripts. Next week, I'll be covering even more ways to speed up page loads by optimizing your head tag. We'll cover browser preloaders, HTTP caching, resource hints, streaming responses, and < 4KB headers. --- ## How to Measure Ruby App Performance with New Relic URL: https://www.speedshop.co/blog/ruby-app-performance-with-new-relic/ It's 12pm on a Monday. Your boss walks by: "The site feels...slow. I don't know, it just does." Hmmm. You riposte with the classic developer reply: "Well, it's fast on my local machine." Boom! Boss averted! {% marginnote_lazy https://i.imgur.com/D20c1vk.gif||true %} Unfortunately, you were a little dishonest. You know better than to think that speed in the local environment has anything to do with speed in production. You know that, right? Wait, we're not on the same page here? Several factors can cause Ruby applications to have performance discrepancies between production and development: * **Application settings, like code reloading** Rails and most other Ruby web frameworks reload (almost) all of your application code on every request to pick up on changes you've made to files. That's a pretty slow process. In addition, there a lot of subtle differences between apps in development and production modes, especially surrounding asset pipelines. Simpler frameworks may not have these behaviors, but don't kid yourself - if anything in your app is changing in response to "RACK_ENV", you could be introducing performance problems that you can't catch in development. * **Caching behavior** Rails disables caching in development mode by default. Obviously, turning that on has a big performance impact in production. In addition, caches in development work differently than caches in production, mostly due to the introduction of network latency. Even 10ms of network latency between your application server and your cache store can cripple pages that make many cache calls. * **Differences in data** {% marginnote_lazy https://i.imgur.com/l1rvh6w.jpg %} This is an insidious one, and the usual cause for an app that seems slow in production but fast in development. Sure, that query you run locally (User.all, for example) only returns 100 rows in development using your seed data. But in production, that query could return 10,000 or 100,000 rows! In addition, consider what happens when those 10,000 rows you return need to be attached to 10,000 other records because you used `includes` to pre-load them. Get ready to wait around. * **Network latency** It takes time for a TCP packet to go from one place to another. And while the speed of light is fast, it does add up. As a rule of thumb, figure 10ms for the same city, 20ms for a state or two away, 100ms across the US (from NY to CA), and up to 300ms to get to the other side of the world. These numbers can quadruple on mobile networks. This has a major impact when a page makes many asset requests or has blocking external JavaScript resources. * **JavaScript and devices** JavaScript takes time and CPU to execute. Most people don't have fancy new MacBooks like we developers do - and on mobile devices the story is certainly even worse. Consider that even a low-end desktop processor sports twice the computing power of top-end mobile CPUs and you can see how complex Javascript that "feels fine on my machine" can grind a mobile device to a halt. * **System configuration and resources** Unless you're using containers, system configuration will always differ between environments. This can even be as subtle as utilities being compiled with different compiler flags! Of course, even containers will run on different physical hardware, which can have severe performance consequences, especially regarding threading and concurrency. * **Virtualization** Most people deploy to shared, virtualized environments nowadays. Unfortunately, that means a physical server will share resources with up to half-a-dozen or so virtual servers, which can negatively and unpredictably impact performance when one virtualized server is hogging up the resources available. So what's a developer to do? Why, install a performance monitoring solution in production! NewRelic is the tool I reach for. Not only is it free to start with, the tools included are extensive, even at the free level. In this post, I'm going to give you a tour of each of NewRelic's features and how they can help you to diagnose performance hotspots in a Rails app. **Full disclosure** - I don't work work New Relic, and no one from New Relic paid for or even talked to me about this post. I haven't used [Skylight](https://www.skylight.io/), New Relic's biggest competitor in this space, so I can't give you a good comparison of the two or their features. I hope to someday do a post on Skylight, but I'll need a production app I can use it on first. ## Pareto and Zipf Before we get into any code or fancy graphs, though, I want to talk about principles. Actually, I want to tell you about an American linguist and an Italian economist. {% marginnote_lazy https://i.imgur.com/RbKfDx7.jpg %}George Kingsley Zipf was an American philologist that studied languages using a new and interesting field at his time - statistics. Zipf's novel idea to apply statistics to the study of language landed him an astonishing insight: in nearly every language, some words are used *a lot*, but most (nearly all) words are used hardly at all. That is to say, if you took every English word ever written and plotted the frequency of words used as a histogram, you'd end up with a graph that looked something like what you see to the right. It's a power law. The [Brown Corpus](https://en.wikipedia.org/wiki/Brown_Corpus) is 500 samples of English-language text comprising 1 million words. But just 135 unique words are needed to account for 50% of those million. That's insane. If you take Zipf's probability distribution and make it continuous instead of discrete, you get the **Pareto distribution**. Many of you probably see where I'm going with this by now. Stay with me. {% marginnote_lazy https://upload.wikimedia.org/wikipedia/commons/thumb/1/11/Probability_density_function_of_Pareto_distribution.svg/500px-Probability_density_function_of_Pareto_distribution.svg.png %}The Pareto distribution, pictured at right, has been found to hold for a *scary* number of completely different and unrelated fields in the sciences. For example, here are some natural phenomena that exhibit a Pareto (power law) distribution: * Wealth inequality * Sizes of rocks on a beach * Hard disk drive error rates (!) * File size distribution of Internet traffic (!!!) We tend to think of the natural world as *random* or *chaotic*. But often, it is anything but. Many probability distributions, in the wild, support the Pareto Principle: > 80% of the output will come from 20% of the input While you may have heard this before, what I'm trying to get across to you is that isn't made up. The Pareto distribution is the real deal - utilized in hundreds of otherwise completely unrelated scientific fields - and we can use it's ubiquity to our advantage. Allow me to reformulate and apply this to web application performance: > **80% of an application's work occurs in 20% of it's code.** {% marginnote_lazy https://i.imgur.com/fliqI9N.gif|I pity the fool that prematurely optimizes their application! %}This is why premature optimization is so bad and why performance monitoring, profiling and benchmarking are so important. What the Pareto Principle reveals to us is that optimizing any random line of code in our application is in fact *unlikely* to speed up our application at all! 80% of the "slowness" in any given app will be hidden away in a minority of the code. So instead of optimizing blindly, applying principles at random we read from blog posts or engaging in Hacker-News-Driven-Development by using the latest and "most performant" web technologies, we need to *measure* where the bottlenecks and problem areas are in our application. Repeat after me: I will not optimize anything in my application until my metrics tell me so. ## Getting an Overview Let's walk through the process I use when I look at a Ruby app on NewRelic. When I first open up a New Relic dashboard, I'm trying to establish the broad picture: How big is this application? Where does most of its time go? Are there are any "alarm bells" going off just on the main dashboard? ### A Glossary New Relic uses a couple of terms that we'll need to define: * **Transactions** This is New Relic's cross-platform way of saying "response". In Rails, a single "transaction" would be a single response from a controller action. Transactions from a Rails app in NewRelic look like "WelcomeController#index" and so on. * **Real-User Monitoring (also RUM and Browser monitoring)** {% marginnote_lazy https://i.imgur.com/UfGgA6g.jpg|Too much rum, though, and you start wearing eyeshadow. %} If you enable it, New Relic will automatically insert some Javascript for you on every page. This Javascript hooks into the [NavigationTimingAPI](https://developer.mozilla.org/en-US/docs/Web/API/Navigation_timing_API) of the browser and sends several important metrics back to NewRelic. Events set include domContentLoaded, domComplete, requestStart and responseEnd. Any time you see NewRelic refer to "real-user monitoring" or "browser metrics", they're referring to this. ### Response time - where does it go? The web transaction response time graph is one of the most important on NewRelic, and forms the broadest possible picture of the backend performance of your app. NewRelic defaults to 30 minutes as the the timeframe, but I immediately change this to the longest interval available - preferably about a month, although 7 days will do. The first thing I'll look at here is the app server and browser response averages. Here are some rules of thumb for what you should expect these numbers to be in an average Rails application: | App server avg response time | Status | | -------- | -------- | | < 100ms | Fast! | | < 300ms | Average | | > 300ms | Slow! | Of course, those numbers are just rules of thumb for Rails applications that serve up HTML - your typical "Basecamp-style" application. For simple API servers that serve JSON only, I might divide by 2, for example. | Browser avg load time | Status | | -------- | -------- | | < 3 sec | Fast! | | < 6 sec | Average | | > 6 sec | Slow! | {% marginnote_lazy https://i.imgur.com/vmp1svR.gif||true %}I can hear the keyboards clattering already furiously emailing me: "That's so slow! Rails sucks! Blah blah..." I'm just sharing what I've seen in the wild in my own experience. Remember - Github, Basecamp and Shopify are all *enormous* WebScale™ Ruby shops that average 50-100ms responses, which is pretty good by anyone's measure. Based on what I'm seeing with these numbers, I know where to pay attention later on. For example, if I notice a fast or average backend but slow browser (real-user monitoring) numbers, I'll go look at the browser numbers next rather than delving deeper into the backend numbers. Note that most browser load times are 1-3 seconds, while most application server response times are 1-300 milliseconds. Application server responses, *on average*, are just 10% of the end-users total page loading experience. This means front-end performance optimization is actually far more important that most Rails developers will give it credit for. Back-end optimization remains important for scaling (lower response times mean more responses per second), but when thinking about the browser experience, they usually mean vanishingly little. Next, I'm considering the shape of the response time graph. Does the app seem to slow down at certain times of day or during deploys? The most important part of this graph, though, is to figure out how much time goes to what part of the stack. Here's a typical Ruby application - most of its time is spent in Ruby. If I see an app that spends a lot of time in the database, web external, or other processes, I know there's a problem. Most of your time should be spent in Ruby (running Ruby code is usually the slowest part of your app!). If, for example, I see a lot of time in web external, I know there's probably a controller or view that's waiting, synchronously, on an external API. That's almost never necessary and I'd work to remove that. A lot of time in request queueing means you need more servers, because requests are spending too much time waiting for an open application instance. #### Percentiles and Histograms {% marginnote_lazy https://i.imgur.com/KZhqgxR.png %}The histogram makes it easy to pick out what transactions are causing extra-long response times. Just click the histogram bars that are way far out to the right and pay attention to what controllers are usually causing these actions. Optimizing these transactions will have the biggest impact on 95% percentile response times. Most Ruby apps response time histograms look like an power curve. Remember what I said above about Pareto. So, conversely, be sure to check out what actions take the least amount of time (the histogram bar furthest to the left). Are they asset requests? Redirects? Errors? Is there any way we can *not* serve these requests (in the case of assets, for example, you should be using a CDN)? ### What realm of RPM are we playing in? {% marginnote_lazy https://i.imgur.com/cEKvA82.gif|What it looks like optimizing a high-scale app in production|true %} It's always helpful to check what "order of magnitude" we're at as far as scale. Here are my rules of thumb: | Requests per minute | Scale | | -------- | -------- | | < 10 | Tiny. Should only have 1 server or dyno. | | 10 - 1000 | Average | | > 1000 | High. "Just add more servers" may not work anymore. | Apps above 1000 RPM may start running into scaling issues *outside* of the application in external services, such as databases or cache stores. When I see scale like that, I know my job just got a lot harder because the surface area of potential problems just got bigger. ## Transactions {% marginnote_lazy https://i.imgur.com/SQItcqT.jpg|Note that the top 3 transactions account for 2/3 of time consumed %} Now that I've gotten the lay of the land, I'll start digging into the specifics. We know the averages, but what about the details? At this stage, I'm looking for my "top 5 worst offenders" - where does the app slow to a crawl? What's the 80/20 of time consumed in this application - in other words, in what actions does this application spend 80% of its time? Most Ruby applications will spend 80% of their time in just 20% of the application's controllers (or code). This is good for us performance tweakers - rather than trying to optimize across an entire codebase, we can concentrate on just the top 5 or 10 slowest transactions. For this reason, in the transactions tab, I almost always sort by *most time consuming*. If the top 5 actions in this tab consume 50% of the server's time (they almost always do), and we speed them up by 2x, we've effectively scaled the application up by 25%! That's free scale. Alternatively, if an application is on the lower end of the requests-per-minute scale, I might sort by slowest average response time instead. This sort also helps if you're concentrating on squashing 95th percentiles. ## Database {% marginnote_lazy https://i.imgur.com/Ipd76lV.png %} I'm carrying that "worst offender" mindset into the database. Now, if the previous steps have shown that the database isn't a problem, I may glaze over this section or just try and make sure it's not a single query that's taking up all of our database time. Again, "most time consuming" is probably the best sort here. Here's some symptoms you might see here: * **Lots of time in #find** If your top SQL queries are all model lookups, you've probably got a bad query somewhere. Pay attention to the "time consumption by caller" graph on the right - where is this query being called the most? Go check out those controllers and see if you're doing a WHERE on a column that hasn't been properly indexed, or if you've accidentally added an N+1 query. * **SQL - OTHER** You may see this one if you've got a Rails app. Rails periodically issues queries just to check if the database connection is active, and those queries show up under this "OTHER" label. Don't worry about them - there isn't really anything you can do about it. ## External Services {% marginnote_lazy https://i.imgur.com/2oiZrCk.png %} What I'm looking for here is to make sure that there aren't any external services being pinged during a request. Sometimes that's inevitable (payment processing) but usually it isn't necessary. Most Ruby applications will block on network requests. For example, if to render my cool page, my controller action tries to request something from the Twitter API (say I grab a list of tweets), the end user has to wait until the Twitter API responds before the application server even returns a response. This can delay page loading by 200-500ms *on average*, with 95th percentile times reaching 20 seconds or more, depending on what your timeouts are set at. For example, what I can tell from this graph is that Mailchimp (purple spikes in the graph to the right) seems to go down a lot. Wherever I can, I need to make sure that my calls to Mailchimp have an aggressive timeout (something like 5 seconds is reasonable). I may even consider coding up a [Circuit Breaker](http://martinfowler.com/bliki/CircuitBreaker.html). If my app tries to contact Mailchimp a certain number of times and times out, the circuit breaker will trip and stop any future requests before they've even started. ## GC stats and Reports To be honest, I don't find New Relic's statistics here very useful. You're better off with a tool like `rack-mini-profiler` and `memory_profiler`. I don't find New Relic's "average memory usage per instance" graph very accurate for threaded or multi-process setups either. If you're having issues with garbage collection, I recommend debugging that in development rather than trying to use New Relic's tools to do it in production. [Here's an excellent article](https://blog.codeship.com/debugging-a-memory-leak-on-heroku/) by Heroku's Richard Schneeman about how to debug memory leaks in Ruby applications. In addition, I'm not going to cover the Reports, as they're part of New Relic's (rather expensive) paid plans. For what it's worth, they're pretty self-explanatory. ## Browser / Real user monitoring (RUM) {% marginnote_lazy https://i.imgur.com/UyLFqvw.png %} Remember how we applied an 80/20 mindset to the top offenders in the web transactions tab? We want to do the same thing here. Change the timescale on the main graph to the longest available. Instead of the percentile graph (which is the default view), change it to the "Browser page load time" graph that breaks average load time down by its components. * **Request queueing** Same as the web graph. Notice how little of an impact it usually has on a typical Ruby app - most queueing times are something like 10-20ms, which is just a minuscule part of the average 5 second page load. * **Web application** This is the entire time taken by your app to process a request. Also notice how little time this takes out of the entire stack required to render a webpage. * **Network** Latency. For most Ruby applications, average latency will be longer than the amount of time spent queuing and responding! This number includes the latency in both directions - to and from your server. * **DOM Processing** This is usually the bulk of the time in your graph. DOM Processing in New Relic-land is the time between your client receiving the full response and the [`DOMContentReady` event](https://developer.mozilla.org/en-US/docs/Web/Events/DOMContentLoaded) firing. Now, this is *just* the client having loaded and parsed the *document*, not the CSS and Javascript. *However*, this event is usually delayed while synchronous Javascript executes. WTF is synchronous Javascript? Pretty much anything without an `async` tag. [For more about getting rid of that, check out Google](https://developers.google.com/speed/docs/insights/BlockingJS). In addition, `DOMContentReady` usually also gets [slowed down by external CSS](https://developers.google.com/speed/docs/insights/OptimizeCSSDelivery). Note that, in most browsers, the page pretty much still looks like a blank white window at this point. * **Page Rendering** Page Rendering, according to NewRelic, is everything that happens between the `DOMContentReady` event and the [`load` event](https://developer.mozilla.org/en-US/docs/Web/Events/load). `load` won't fire until every image, script, and iframe is fully ready. So, the browser *may* have started displaying at least parts of the page before this is finished. Note also that `load` always fires *after* [`DOMContentLoaded`](https://developer.mozilla.org/en-US/docs/Web/Events/DOMContentLoaded), the event that you usually attach most of your Javascript to (JQuery's `$(document).ready` attaches functions to fire after `DOMContentLoaded`, for example). For a full guide to optimizing front-end performance issues you find here, see my extensive guide on the topic. It's important to note that while most users won't see *anything* of your site until at least DOM Processing has finished, they probably will start seeing *parts* of it during Page Rendering. It's impossible to know just how much of it they see. If your site has a ton of images, for example, Page Rendering might take *ages* as it downloads all of the images on the page. Note also that Turbolinks and single-page Javascript apps pretty much break real-user-monitoring, because all of these events (DOMContentLoaded, DOMContentReady, load) will only fire *once*, when the page is initially loaded. New Relic *does* give you additional information on AJAX calls, such as throughput and response time, if you pay for the Pro version of the Browser product. ## Conclusion NewRelic, and other production performance monitoring tools like it, is an invaluable tool for the performance-minded Rubyist. You simply cannot be serious about speed and not have a production profiling solution installed. As a takeaway, I hope you've learned how to apply an 80/20 mindset to your Ruby application. This mindset can be applied at all levels of the stack, but don't forget - profiling that isn't based on what the end-user experience isn't based in reality. That's why, for a browser-based application, we should be paying attention first to our *browser* experience, not to our backend, even if that's sometimes easier to measure. --- ## Ludicrously Fast Page Loads - A Guide for Full-Stack Devs URL: https://www.speedshop.co/blog/frontend-performance-chrome-timeline/ Server response times, while easy to track and instrument, are ultimately a meaningless performance metric from an end-user perspective. {% marginnote_lazy https://i.imgur.com/u5soMkT.gif|Actual end-user response to the word 'microservices'|true %} End-users don't care how fast your super-turbocharged bare-metal Node.js server is - they care about the page being completely loaded as fast as possible. Your boss is breathing down your neck about the site being slow - but your Elixir-based microservices architecture has average server response times of 10 nanoseconds! What's going on? Well, what does constructing a webpage actually require? The server has to respond with the HTML (along with the network latency involved in the round-trip), the JS, CSS and HTML needs to be parsed, rendered, and painted, and all the Javascript tied to the page ready event needs to be executed. That's actually a lot of stuff. Usually, server response times make up only a small fraction of this total end-user experience, sometimes as little as 10%. In addition, it's very easy for any of these steps to get out of hand very quickly: * Server response times can easily balloon without proper use of caching, both at the application and HTTP layers. Bad SQL queries in certain parts of the application can send times skyrocketing. * JS and CSS assets must be concatenated, minified and placed in the right place in the document, or rendering may be blocked while the browser stops to load external resources (more on this later). In addition, these days when there's a JQuery plugin or CSS mixin for just about anything, most developers have completely lost track of just how much CSS and JS is being loaded on each page. Even if, gzipped and minified, your CSS and JS assets are <100kb, once they're un-gzipped, they *still* must be parsed and loaded to create the DOM and CSSOM (explained in more detail below). **While gzipped size is important when considering how long CSS or JS will take to come across the network, *uncompressed* size is important for figuring out how long it will take the client to parse these resources and construct the page.** * Web developers (especially non-JavaScripters, like Rails devs) have an awful habit of placing tons of code into `$(document).ready();` or otherwise tying Javascript to page load. This ends up causing *heaps* of unnecessary Javascript to be executed on every page, further delaying page loads. So what's a good, performance-minded full stack developer to do? How can we take our page loads from slow to ludicrous speed? {% marginnote_lazy https://i.imgur.com/F3y7xvo.gif||true %} But, rather than just *tell you* that XYZ technique is faster than another, I'm going to *show you* how and *why*. Rather than take my word for it, you can test different frontend optimizations for yourself. To do that, we're going to need a profiling tool. ## Enter Chrome Timeline My number one front-end performance tool is [Chrome Timeline](https://developer.chrome.com/devtools/docs/timeline). While I use New Relic's real user monitoring (RUM) to get a general idea of how my end-users are experiencing page load times, Chrome Timeline gives you a millisecond-by-millisecond breakdown of exactly what happens during any given web interaction. Although I'm going to show you how to use Chrome Timeline to analyze page loads, you can also use it to profile Javascript interactions once the page has loaded. Note that most of Google's documentation on Chrome Timeline is severely out of date and shows a "waterfall" view that no longer exists in Chrome as of October 2015 (Chrome 45). This post is up-to-date as of that time. Chrome Timeline *also* works really well for optimizing "60fps" JavaScript applications. I'm not going to get into that here. What I'm going discuss is how we can use Chrome Timeline to make our applications take as little time as possible between user input (clicking, pushing a button, hitting enter) and response (displaying data, moving us to a new page, etc), focusing on the initial page load. To open Chrome Timeline, open up Chrome Developer Tools (Cmd + Alt + I on Mac) and click on the Timeline tab. You'll see a blank timeline with millisecond markings. For now, uncheck the "causes", "paint" and "memory" checkboxes on the top, and disable the FPS counter by clicking the bar graph icon. {% marginnote_lazy https://i.imgur.com/VkvsEfY.png | What your settings should look like %} These tools are mostly useful for people profiling client-side JS apps, which I won't get into here. The Chrome Timeline records page interactions a lot like a VCR. You can click the little circular icon (the record button) at any time to turn on Timeline recording, and then click it again to stop recording. If the Timeline is open during a refresh, it will automatically record until the page has loaded. Let's try it on [https://github.com/nateberkopec/todomvc-turbolinks](https://github.com/nateberkopec/todomvc-turbolinks). This is a [TodoMVC](http://todomvc.com) implementation I did for a previous blog on Turbolinks. While the Timeline is open, you can trigger a full page load with CMD + Shift + R and Chrome will automatically record the page load for you in Timeline.{% sidenote 1 "Be sure you're doing a hard refresh here, otherwise you may not redownload any assets." %} **Note that browser extensions will show up on Chrome Timeline.** Any extension that alters the page may show up and make your timelines confusing. Do yourself a favor and disable all of your extensions while profiling with Chrome Timeline. We're going to start with a walkthrough of a typical HTML page load in Timeline, and then we're going to identify what this performance profile says about our application and how we can speed it up. Here's what my Timeline looked like: ![My timeline](https://i.imgur.com/hXsZNPt.png) 254 ms from refresh to done - not bad for an old Rails app, eh? ## Receiving the HTML The first thing you'll notice is that big chunk of idle time at the beginning. Almost nothing is happening until about 67ms after I hard-refreshed. {% marginnote_lazy https://i.imgur.com/cjQ5N38.png|"An idle browser is the devil's workshop." %} What's going on there? It's a combination of server response time (on this particular app, I know it hovers around 20ms), and network latency (depending on how far you are from the US East Coast, anywhere from 10-300ms). Even though we live in an age of mass cable and fiber optic internet, our HTTP requests still take a lot of time to go from place to place. Even at the theoretical maximum speed of an HTTP request (the speed of light), it would take a user in Singapore about 70ms to reach a server in the US. And HTTP doesn't travel at the speed of light - cable internet works about half that speed. In addition, they make as many as a dozen intermediate stops along the way along the Internet backbone. You can see these stops using `traceroute`. In addition, you can get the approximate network latency to a given server by simply using `ping` (that's what it was designed for!). For example, I live in New York City. Pinging a NIST time server in Oregon, I usually can see network latency times of about 100ms {% marginnote_lazy https://i.imgur.com/weVRDG9.png|Oregon? Well these packets Oregonna take a long time to get there! %}. That's a pretty substantial increase over the time we'd expect if the packets were traveling at the speed of light (~26ms). By comparison, my average network latency for a time server in Pennsylvania is just 20ms. And Indonesia? Packets take a whopping 364ms to make the round trip. For websites that are trying to keep page load times under 1 second, this highlights the importance of geographically distributed CDNs and mirrors. Let's zoom in on the first event on the timeline. It seems to happen in the middle of this big idle period. You can use the mouse wheel to zoom. The first event on the Timeline is "Receive Response". {% marginnote_lazy https://i.imgur.com/6aEHkq4.png %} A few milliseconds later, you'll see a (tiny) "Receive Data" event. You might see one or two more miscellaneous events related to page unloading, another "Receive Data" event, and finally a "Finish Loading" event. What's going on here? The server has started responding to your request when you see that first "Receive Response" event. You'll see several "Receive Data" events as bytes come down over the wire, completing with the "Finish Loading" event. This pattern of events will occur for any resource the page needs - images, CSS, JS, whatever. Once we've finished downloading the document, we can move on to parsing it. ### Parse HTML "Parsing HTML" sounds like a pretty simple process, but Chrome (and any browser) actually has a lot of work to do. The browser will read the bytes of HTML off the network (or disk, if you're viewing a page on your computer), and convert those bytes into UTF-8 or whatever document encoding you've specified. Then, the browser has to "tokenize" - basically taking the long text string of the HTML and picking out each tag, like `` and ``. Imagine that the browser converts the ~100kb string of HTML into an array of several strings. {% marginnote_lazy https://i.imgur.com/2ybDk0W.jpg|Me, waiting for The Verge to load %} Then it "lexes" these tokens (basically converts them into fancy objects) and finally constructs a DOM out of them. On complicated pages, these steps add up - on my machine, The Verge takes over 200ms just to *parse the HTML*. Yow. You may also see two "Send Request" events (they're really small) beneath the "Parse HTML" event. In case you haven't figured it out already, what we're looking at is called a "flamegraph". Events underneath other ones mean that the upper event "called" the lower one. The two "Send Request" events you see here are the browser requesting the Javascript and CSS files linked in the head. This is a Rails app, so there's only one of each. {% marginnote_lazy https://i.imgur.com/SFCvlgQ.png|The two teeny tiny blue lines there are the JS and CSS requests being sent. %} In addition, the Javascript file in this app is marked with an `async` attribute: ```html ``` Normally, when a browser sees a Javascript tag like this in the head, it *stops completely* until it has finished downloading and evaluating the script. If the script is remote, we have to wait while the script downloads. This can take *a lot* of time - even more than a whole second, when you include network latency and the time required to evaluate the script. The reason browsers do this is because Javascript can modify the DOM - any time there's a script tag, the browser has to execute it because it could change the DOM or layout. For more about Javascript blocking page rendering, [Google does a great explanation here](https://developers.google.com/speed/docs/insights/BlockingJS). Because this script tag was marked with the `async` attribute, this doesn't happen - the browser won't "stop the world" to download and evaluate the Javascript.{% marginnote_lazy https://i.imgur.com/kiAS3za.gif|Non-blocking async! WhoOOoOOAaaa!|true %} This can be a *huge* boost to speeding up time-to-first-paint for most websites. Browsers will *not* wait on external CSS before continuing past this step. If you think about it, this makes sense. CSS cannot modify the DOM, it can only style it and make it pretty. In order to even apply the CSS, we need to have the DOM constructed first. So the browser, smartly, simply sends the request for the CSS and moves on to the next step. Note that this "Parse HTML" step will reoccur every time the browser has to read new HTML - for example, from an AJAX request. ### Recalculate Styles The next major event you're going to see is the purple "Recalculate Styles". Unfortunately, this event covers a lot of things that actually happen during page construction. The first is the construction of the CSSOM. {% marginnote_lazy https://i.imgur.com/RNaLvZj.png %} As HTML is to the DOM, so CSS is to the CSSOM. Your CSS, after it's downloaded has to be converted -> tokenized -> lexed -> constructed just like the HTML was. This process is usually the cause of any "Recalculate Styles" bars you see at the beginning of the page load. "Recalculate Styles" can also mean a lot of other confusing things are happening with your CSS, like "recursive calculation of computed styles", or whatever that means. The gist is that if you're seeing a lot of time in "Recalculate Styles", your CSS is too complicated. Try to eliminate unused or unnecessary style rules. Why are we seeing Recalculate Styles events when the CSS hasn't even been downloaded yet? The browser is applying the browser's default CSS to the document, and it may also be applying any `style` attributes present in the HTML markup itself (`display: none` being a common one, present on this page). You will probably see more purple events (Recalculate Styles and its cousin, Layout) later on in the timeline. Again, your browser does not wait for CSS to finish downloading - it's already calculating styles and layouts based on just your HTML markup and the browser defaults right now. The rendering events you see later on occur once the CSS is finished downloading. ### Layout Slightly after your first Recalculate Styles event, you should see a purple "Layout" event. Basically, at this point, your browser has all of the DOM and CSSOM in memory and needs to turn it into pixels on the screen. The browser traverses the visible elements of the DOM (actually the render tree), and figures out each node's visibility, applicable CSS styles, and relative geometry (50% width of its parent and so on). Complicated CSS will obviously make this step longer, but so will complicated HTML. If you're seeing a lot of "layout" events during a page load, you may be experiencing something called **"layout thrashing"**. {% marginnote_lazy https://i.imgur.com/YGvW85u.gif|Actual layout thrashing in progress|true %} Any time you change the geometry of an element (its height, width, whatever), you trigger a layout event. And, unfortunately, browsers can't tell what part of the page they need to recalculate. Usually, they have to recalculate the layout for *the entire document*. This is especially slow with float-based layouts, though it's slightly faster with flex box layouts. Layout thrashing is usually going to be caused by Javascript messing with the DOM, though using multiple stylesheets will also cause it. [For more about layout thrashing, Google has an excellent page on the topic](https://developers.google.com/web/fundamentals/performance/rendering/avoid-large-complex-layouts-and-layout-thrashing?hl=en). In summary - in the "Layout" step, then, the browser is just calculating what's visible, what isn't, and where it should go on the page. ### DomContentLoaded It's generally at this point that you'll see the blue bar in Timeline - this is the [`DomContentLoaded`](https://developer.mozilla.org/en-US/docs/Web/Events/DOMContentLoaded) event. At this point, your browser is done parsing the HTML and running any blocking Javascript (that is, Javascript either embedded in the page or in a script tag that isn't marked `async`). Most browsers have not painted *anything* to the screen by this point. To speed up `DomContentLoaded`, you can do a few things: * Make script tags `async` where possible. Moving script tags to the end of the document doesn't help speed up `DomContentLoaded`, as the browser must still evaluate the Javascript before completing the construction of the DOM. All "async" means is that the only part of the script executed "synchronously" is the start of downloading of the script itself, its execution will be delayed until later. [Ilya Grigorik suggests that using `async` tags is generally cleaner and more effective than using so-called 'async' script injection](https://www.igvita.com/2014/05/20/script-injected-async-scripts-considered-harmful/). * Use less complex HTML markup. * Avoid layout thrash (see above). Don't use more than one stylesheet - concatenate your assets! * Inline styles in moderation. Inlining styles means that the browser may try to parse the stylesheet before moving on to the rest of the document. Google recommends inlining only styles required to display above-the-fold content. This will slow down DOMContentLoaded but will speed up the window's `load` event. This may be true, but you certainly don't want to inline *all* of your CSS. Also, figuring out what CSS rules you need for the above-the-fold content in this age of CSS frameworks and Bootstrap sounds like a *lot* of work to me. How much CSS do you need to render above-the-fold? *All of it*. As a rule of them, don't consider inlining *all* of your CSS unless you've got about 50kb or less of it. Once HTTP2 becomes more common and we can download CSS, HTML and JS over the same connection, this optimization will no longer be needed. ### Paint As we move along the timeline to the right, you should start seeing some green bars in the flamegraph. These are Paint related events. There's a *whole* lot that can go on in these events (and Chrome even provides profiling tools just for these painting events), but I'm not going to go too deep on them here. All you need to know is that paint events happen when the browser is done rendering (the purple bars - the process of turning your CSS and HTML into a layout) and needs to turn the layout into pixels on a screen. The green bar in the timeline is the first paint - the first time anything is rendered to screen. Optimizing first paint is largely a matter of optimizing DOMContentLoaded and getting the stylesheet to the client as fast as possible. Any stylesheet that doesn't specify a media query (like `print`) will block page rendering until we've downloaded it and parsed it. ### Parse Author Style Sheet Keep scrolling to the right on the Timeline. Wow - see how much longer it took to get to this part? {% marginnote_lazy https://i.imgur.com/wIqXezh.png %} In my case, it took almost 40 ms of just waiting around to download the whole stylesheet - and this app's stylesheet isn't even that big! To be exact, we sent the request for the stylesheet at about 65ms, and it didn't come back until 101ms. In reality, this actually extremely fast (in a real app, you would expect that to be more like 200-350ms at least), and we can't really optimize that much further. I'm in NYC and Heroku is in Virginia, so most of that time is network latency anyway. Once the stylesheet is downloaded, it's parsed. You'll see another cycle of purple events (as the CSSOM is re-calculated, we re-render the layout) and green events (now that the layout is updated, we render the result to the screen). The stylesheet for this app is extremely simple, and my app appears to be wasting about 30ms waiting for the CSS to download. It may be worth investigating the performance impact of inlining the entire stylesheet in the HEAD of this page. Most sites won't benefit from this optimization (see my bit about this above), but because this app is idling for about 20ms waiting for the styles to download, we may want to eliminate that network round-trip. ### Javascript Eventually, you'll notice the Javascript finish downloading (this is the "Finish Loading" event for your Javascript file). {% marginnote_lazy https://i.imgur.com/Z1GoQQh.png %} A millisecond or two after this occurs, you'll see the big yellow "Evaluate Script" bars start up. You'll notice the flamegraph start to get a lot deeper here. It's hard to tell on this site as to what's going on because the Javascript has been minified, but in development mode, pre-minified, you can learn a lot about why it takes so long for your Javascript to evaluate here. Note that this is a really, really simple application, but because of the sheer amount of Javascript involved, it takes 76ms for my machine just to parse and evaluate it all. Remember that this will happen on *every page load*, and *double* the amount of time on a mobile browser. This isn't even that much JavaScript in web terms - 37kb gzipped. Eventually, after a whole lot of script evaluation, you'll probably see a couple of Recalculate Style and Paint events. Your Javascript will probably do a few things to change the layout - that's what's happening here. Finally, you should see the `load` event fire off. There will be several Javascript functions attached to this event in almost every application. Once all of those callbacks attached to `load` have completed, you'll see the **red bar**, which signifies the end of the `load `. This is generally when the page is "ready" and finished loading. Finally! ## Using Chrome Timeline to Debug Browser Speed So, you've got a site that takes 5-10 seconds to get to the `load` event. How can you use Timeline to profile it and find the performance hotspots? 1. **Hard reload (ctrl-shift-r) and load the Timeline with fresh data** 2. **Look at the pie graph for the entire page load**. After hard reloading, Chrome will show the aggregate stats for the entire page load in the pie graph. You can see here that it took about 2.23 seconds from my refresh input to get to `load`. Get an idea of where you spend most of your time - is it in parsing (loading), scripting or rendering and painting? Is it idle time? * **Reduce Idle** Idling comes from slow server responses and asset requests. If you're idling a lot, make sure your server is still zippy-quick. If it is, you may have an unoptimized order of assets. See the "DomContentLoaded" section above. * **Reduce Loading** Recall that "loading" here refers to time spent parsing HTML and CSS. To decrease loading time, you don't have many options other than to decrease the amount of HTML and CSS you're sending to the client. * **Reduce Scripting** Time spent evaluating scripts is usually the largest chunk of page load time outside of waiting for the network. Most sites use quite a few different marketing-related JavaScript plugins, like Olark and Mixpanel. Where possible, I would try to add `async` tags to these scripts to get them off the rendering critical path, even if the vendor proudly claims the script is already "async!". Try to look at the call stacks and figure out where you're spending most of your time. * **Reduce Rendering and Painting** Sites can also have quite a few layout changes and re-renders due to tools like Optimize.ly, something we can see by checking the "First Layout Invalidation" property of some of the "Layout" events in the Timeline. This is a tough one. Optimize.ly's whole purpose is to essentially change the content of the page, so moving it to an `async` script tag may cause a "flash of unstyled content" where part of the page would look one way and then suddenly flash into a different styling. That isn't acceptable, so we're stuck with Optimize.ly's slow and painful re-layouts here. ### TL:DR; Do these things to make your pages load faster. * **You should have only one remote JS file and one remote CSS file**. If you're using Rails, this is already done for you. Remember that every little marketing tool - Olark, Optimize.ly, etc etc - will try to inject scripts and stylesheets into the page, slowing it down. Remember that the cost of these tools is not free. However, there's no excuse for serving multiple CSS or JS files from your own domain. Having just one JS file and one CSS file eliminates network roundtrips - a major gain for users in high-latency network environments (international and mobile come to mind). In addition, multiple stylesheets cause layout thrashing. * **Async all the things!** "Async" javascripts that download and inject their own scripts (like [Mixpanel's "async" script here](https://mixpanel.com/help/reference/javascript)) are not truly "asynchronous". Using the `async` attribute on script tags will *always* yield a performance benefit. Note that the attribute has no effect on inline Javascript tags (tags without a `src` attribute), so you may need to drop things like Mixpanel's script into a remote file you host yourself (in Rails, you might put it into `application.js` for example) and then make sure that remote script has an `async` attribute. Using `async` on external scripts takes them off the blocking render path, so the page will render without waiting for these scripts to finish evaluating. * **For god's sake, CSS goes before JavaScript**. If you *absolutely must* put external JS on your page and you can't use an `async` tag, external CSS must go first. External CSS doesn't block further processing of the page, unlike external JS. We want to send off all of our requests *before* we wait on remote JS to load. * **JavaScript is not free.** I don't care how small your JS is gzipped - any additional JS you add takes additional time for the browser to evaluate on *every page load*. While a browser may only need to *download* JavaScripts once, and can use a cached copy thereafter, it will need to *evaluate* all of that JavaScript on *every page load*. Don't believe me that this can slow your page down? Check out [The Verge](http://www.theverge.com) and look at how much time their pages spend executing JavaScript. Yowch. * **$(document).ready is not free**. Every time you're adding something to the document's being ready, you're adding script execution that delays the completion of page loads. Look at the Chrome Timeline's flamegraph when your `load` event fires - if it's long and deep, you need to investigate how you can tie fewer events to the document being ready. Can you attach your handlers to `DomContentLoaded` instead? --- ## Action Cable - Friend or Foe? URL: https://www.speedshop.co/blog/action-cable/ One of the marquee features of Rails 5 (likely releasing sometime Q1/Q2 2016) is Action Cable, Rails' new framework for dealing with WebSockets. Action Cable has generated a lot of interest, though perhaps for the wrong reasons. "WebSockets are those cool things the Node people get to use, right?" and "I heard WebSockets are The Future™" seem to be the prevailing attitudes, resulting in a lot of confusion and uncertainty about Action Cable's purpose and promise. It doesn't help that current online conversation around WebSockets is thick with overly fancy buzzwords like "realtime" and "full-duplex". {% marginnote_lazy https://i.imgur.com/U7vo0Hs.gif|Full-duplex? That's twice as good as half-duplex right? |true %} In addition, some claim that a WebSockets-based application is somehow more scalable than traditional implementations. What's a Rails application developer to make of all of this? This won't be a tutorial or a how-to article - instead, we're going to get into the *why* of Action Cable, not the *how*. Let's start with a review of how we got here - what problem is WebSockets trying to solve? How did we solve this problem in the past? ## Don't hit the refresh button! The Web is built around the HTTP request. In the good old days, you requested a page (GET) and received a response with the page you requested. We developed an extensive methodology (REST) to create a stateless Web based on requesting and modifying resources on the server. It's important to realize that an HTTP request is *stateless* - in order for us to know *who* is making the request, the request must tell us itself. Without reading the contents of the request, there's really no way of knowing what request belongs to which session. Usually, in Rails, we do this with a secure "signed" cookie {% sidenote 1 "A signed cookie means that a client can't tamper with it's value - important if you want to prevent session hijacking!" %} that carries a user ID. As the web grew richer, with video, audio and more replacing the simple text-only pages of yesteryear, we started to crave a constant, uninterrupted connection between server and client. There were places where we wanted the server to communicate back to the client (or vice versa) frequently: * **Clients needing to send rapidly to the server**. High-throughput environments, like online browser-based games, needed clients and servers to be able to exchange several messages *per second*. Imagine trying to implement an first person shooter's networking code with HTTP requests. Sometimes this is called a "full-duplex" or "bi-directional" communication. * **"Live" data**. Web pages started to have "live" elements - like a comments section that automatically updated when a new comment was added (without a page refresh), chat rooms, constant-updated stock tickers and the like. We wanted the page to update itself when the data changed on the server *without* user input. Sometimes this is called a "realtime" application, though I find that term buzzwordy and usually inaccurate. "Realtime" implies constant, nano-second resolution updating. The reality is that the comments section on your website probably doesn't change every nano-second. If you're lucky, it'll change once every minute or so. I prefer the term "Live" for this reason. We all know "live" broadcasts are every so slightly delayed by a few seconds, but we'll still call it "live!". * **Streaming**. HTTP proved unsuitable for streaming data. For many years, streaming video required third-party plugins (remember RealPlayer?). Even now, streaming data other than video remains a complex task without WebSockets (remote desktop connections, for example), and it remains nearly impossible to stream binary data to Javascript without Flash or Java applets (eek!). ## The Road to WebSockets Over the years, we've developed a lot of different solutions to these problems. Some of them haven't really stood the test of time - Flash XMLSocket relays, and `multipart/x-mixed-replace` come to mind. However, several techniques for solving the "realtime" problem(s) are still in use: ### Polling Polling involves the client asking the server, on a set interval (say, three seconds) if there is any new data. {% marginnote_lazy https://i.imgur.com/dKNsN7L.gif|Hey! Hey server! You got any new data? Server? SERVER!|true %} Returning to the "live comments" example, let's say we have a page with a comments section. To create this application with polling, we can write some Javascript to ask the server every three seconds for the latest comment data in JSON format. If there is new data, we can update the comment section. The advantage of polling is that it's rock-solid and extremely simple to set up. For these reasons, it's in wide use all over the Web. It's also very resistant to network outage and latency - if you miss 1 or 2 polls because the network went out, for example, no problem! You just keep polling until eventually it works again. Also, thanks to the stateless nature of HTTP, IP address changes (say, a mobile client with data roaming) won't break the application. However, you might already have alarm bells going off in your head here regarding scalability. You're adding considerable load to your servers by causing *every* client to hit your server *every* 3 seconds. There are ways to alleviate this - HTTP caching is a very good one - but the fact remains, your server will have to return a response to every client every 3 seconds, no matter what. Also, while polling is acceptable for "live" applications (most people won't notice a 3-second delay in your chat app or comments thread), it isn't appropriate for rapid back-and-forth (like games) or streaming data. ### Long-polling Long-polling is a bit like polling, but without a set interval between requests (or "polls"). The client sends a request to the server for new data - if the server has new data, then it sends a response back like normal. If there isn't any new data, though, it *holds the request open*, effectively creating a persistent connection, and then when it receives new data, completes the response. Exactly how this is accomplished varies. There are several "sub-techniques" of long-polling you may have heard of, like [BOSH](https://en.wikipedia.org/wiki/BOSH) and [Comet](https://en.wikipedia.org/wiki/Comet_(programming)). Suffice it so say, long-polling techniques are considerably more complicated than polling, and can often involve weird hacks like hidden iframes. Long-polling is great when data doesn't change very often. Let's say we connect to our live comments, and 45 seconds later a new comment is added. Instead of 15 polls to the server over 45 seconds from a single client, a server would open only 1 persistent connection. However, it quickly falls apart if data changes often. Instead of a live comments section, consider a stock ticker. A stock's price can changes at the millisecond interval (or faster!) during a trading day. That means any time the client asks for new data, the server will return a response immediately. This can get out of hand quickly, because as soon as the client gets back a response it will make a new request. This could result in 5-10 requests per second *per client*. You would be wise to implement some limits in your client! Then again, as soon as you've done that, your application isn't really RealTime™ anymore! ### Server-sent Events (SSEs) Server-sent Events are essentially a one-way connection from the server to the client. Clients can't use SSEs to send data back to the server. Server-sent Events got turned into a browser API back in 2006, and is currently supported by every major browser *except* any version of Internet Explorer.{% marginnote_lazy https://i.imgur.com/FHB2E1f.gif | |true %} Using server-side events is really quite simple from the (Javascript) client's side. You set up an `EventSource` object, define an `onmessage` callback describing what you'll do when you get a new message from the server, and you're off to the races. Server-sent event support was added to Rails in 4.0, through [ActionController::Live](http://tenderlovemaking.com/2012/07/30/is-it-live.html). Serving a client with SSEs requires a persistent connection. This means a few things: using Server-sent events won't work pretty much at all on Heroku, since they'll terminate any connections after 30 seconds. Unicorn will do the same thing, and WEBrick won't work at all. So your options are Passenger, Puma, or Thin, and you can't be on Heroku. Oh, and no one using your site can use Internet Explorer. You can see why ActionController::Live hasn't caught on. It's too bad - the API is really simple and for most implementations ("live" comments, for example) SSE's would work great. ## How WebSockets Work This is the part where I say: "WebSockets to the rescue!" right? Well, maybe. But first, let's investigate what makes them unique. ### Persistent, stateful connection Unlike HTTP requests, WebSocket connections are *stateful*. What does this mean? To use a metaphor - HTTP requests are like a mailbox. All requests come in to the same place, and you have to look at the request (e.g., the return address) to know who sent it to you. In contrast, WebSocket connections are like building a pipe between a server and the client. Instead of all the requests coming in through one place, they're coming in through hundreds of individual pipes. When a new request comes through a pipe, you know *who sent the request*, without even looking at the actual request. The fact that WebSockets are a *stateful* connection means that the connection between a particular client machine and server must remain constant, otherwise the connection will be broken. For example - a *stateless* protocol like HTTP can be served by any of a dozen or more of your Ruby application's servers, but a WebSocket connection must be maintained by a single instance for the duration of the connection. This is sometimes called "sticky sessions".{% sidenote 2 "As far as I can tell, Action Cable solves this problem using Redis. Basically, each Action Cable server instance listens to a Redis pubsub channel. When a new message is published, the Action Cable server rebroadcasts that message to all connected clients. Because all of the Action Cable servers are connected to the same Redis instance, everyone gets the message." %} It also makes load balancing a lot more difficult. However, in return, you don't need to use cookies or session IDs. ### No data frames To generalize - let's say that every message has *data* and *metadata*. The *data* is the actual thing we're trying to communicate, and *metadata* is data about the data. You might say a communication protocol is more *efficient* if it requires less *metadata* than another protocol. HTTP needs a decent amount of metadata to work. In HTTP, metadata is carried in the form of HTTP headers. Here are some sample headers from an HTTP response of a Rails server: ``` HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 Vary: Accept-Encoding X-Runtime: 0.121484 X-Powered-By: Phusion Passenger 5.0.14 X-Xss-Protection: 1; mode=block Set-Cookie: _session_id=f9087b681653d9daf948137f7ece14bf; path=/; secure; HttpOnly Server: nginx/1.8.0 + Phusion Passenger 5.0.14 Via: 1.1 vegur Cache-Control: max-age=0, private, must-revalidate Date: Wed, 23 Sep 2015 19:43:03 GMT X-Request-Id: effc7fe2-0ab8-4462-8b64-cb055f5d1b13 Strict-Transport-Security: max-age=31536000 Content-Length: 39095 Connection: close X-Content-Type-Options: nosniff Etag: W/"469b11fcecff716247571b85ff1fc7ae" Status: 200 OK X-Frame-Options: SAMEORIGIN ``` Yikes, that's 652 bytes before we even get to the data. And we haven't even gotten to the cookie data you sent with the request, which is probably another 2,000 bytes. You can see how inefficient this might be if our data is really small or if we're making a lot of requests. WebSockets gets rid of most of that. To open a WebSockets connection, the client makes a HTTP request to the server with a special `upgrade` header. The server makes an HTTP response that basically says "Cool, I understand WebSockets, open a WebSockets connection." The client then opens a WebSockets pipe. Once that WebSockets connection is open, data sent along the pipe requires *hardly any metadata at all*, usually less than about 6 bytes. Neat! What does all of this mean to us though? Not a whole lot. You could easily do some fancy math here to prove that, since you're eliminating about 2KB of data *per message*, at Google scale you could be saving petabytes of bandwidth. Honestly, I think the savings here are going to vary a lot from application to application, and unless you're at Top 10,000 on Alexa scale, any savings from this might amount to a few bucks on your AWS bill. ### Two-way communication {% marginnote_lazy https://www.reactiongifs.com/r/prs.gif|How many duplexes do YOU have, Red Ranger?|true %} One thing you hear a lot about WebSockets is that they're "full-duplex". What the hell does that mean? Well, clearly, *full* duplex is *better* than *half-duplex* right? That's double the duplexes! All that full-duplex really means is **simultaneous communication**. With HTTP, the client usually has to complete their request to the server before the server can respond. Not so with WebSockets - clients (and servers) can send messages across the pipe at any time. The benefits of this to application developers are, in my opinion, somewhat unclear. Polling can simulate full-duplex communication (at a ~3 second resolution, for example) fairly simply. It does reduce latency in certain situations - for example, instead of requiring a request to pass a message back to the client, the server can just send a message immediately, as soon as it's ready. But the applications where ~1-3 second of latency matters are few and far between - gaming being an obvious exception. Basecamp's chat app, Campfire, used 3-second polling for 10 years. ### Caniuseit? What browsers can you actually use WebSockets in? Pretty much all of them. This is one of WebSockets' biggest advantages over SSE, their nearest competitor. [caniuse.com puts WebSockets' global adoption rate at about 85%](http://caniuse.com/websockets), with the main laggards being Opera Mini and old versions of the Android browser. ## Enter Action Cable Action Cable was [announced at RailsConf 2015 in DHH's keynote](https://www.youtube.com/watch?v=KJVTM7mE1Cc#t=42m30s). He briefly touched on polling - Basecamp's chat application, Campfire, has used a 3-second polling interval for over 10 years. But then, David said: > "If you can make WebSockets even less work than polling, why wouldn't you do it?" That's a great mission statement for Action Cable, really. If WebSockets were as easy as polling, we'd all be using it. Continuous updates are just simply better than 3-second updates. If we can get continuous updates without paying any cost, then we should do that. So, that's our yardstick - is Action Cable as easy (or easier) to use than polling? ### API Overview Action Cable provides the following: * A "Cable" or "Connection", a single WebSocket connection from client to server. It's worthwhile to note that Action Cable assumes you will only have one WebSocket connection, and you'll send all the data from your application along different... * "Channels" - basically subdivisions of the "Cable". A single "Cable" connection has many "Channels". * A "Broadcaster" - Action Cable provides its own server. Yes, you're going to be running another server process now. Essentially, the Action Cable server just uses Redis' pubsub functions to keep track of what's been broadcasted on what cable and to whom. Action Cable essentially provides just one class, `Action Cable::Channel::Base`. You're expected to subclass it and make your own Cables, just like ActiveRecord models or ActionController. Here's a full-stack example, straight from the Action Cable source: ```ruby # app/channels/application_cable/connection.rb module ApplicationCable class Connection < Action Cable::Connection::Base # uniquely identify this connection identified_by :current_user # called when the client first connects def connect self.current_user = find_verified_user end protected def find_verified_user # session isn't accessible here if current_user = User.find(cookies.signed[:user_id]) current_user else # writes a log and raises an exception reject_unauthorized_connection end end end end class WebNotificationsChannel < ApplicationCable::Channel def subscribed # called every time a # client-side subscription is initiated stream_from "web_notifications_#{current_user.id}" end def like(data) comment = Comment.find(data['comment_id') comment.like(by: current_user) comment.save end end # Somewhere else in your app Action Cable.server.broadcast \ "web_notifications_1", { title: 'New things!', body: 'All shit fit for print' } # Client-side coffescript which assumes you've already requested the right to send web notifications @App = {} App.cable = Cable.createConsumer "ws://cable.example.com" App.cable.subscriptions.create "WebNotificationsChannel", received: (data) -> # Called every time we receive data new Notification data['title'], body: data['body'] connected: -> # Called every time we connect like: (data) -> @perform 'like', data ``` A couple of things to notice here: * Note that the channel name "WebNotificationsChannel" is implicit, based on the name of class. * We can call the public methods of our Channel from the client side code - I've given an example of "liking" a notification. * `stream_from` basically establishes a connection between the client and a named Redis pubsub queue. * `Action Cable.server.broadcast` adds a message in a Redis pubsub queue. * We have to write some new code for looking up the current_user. With polling, usually whatever code we already have written works just fine. Overall, I think the API is pretty slick. We have that very Rails-y feel of a Cable's class methods being exposed to the client automatically, the Cable's class name becoming the name of the channel, et cetera. Yet, this does feel like a lot of code to me. And, in addition, you're going to have to write more JavaScript than what you have above to connect everything together. Not to mention that now we've got a Redis dependency that we didn't have before. What I didn't show above is some things that Action Cable gives you for free, like a 3-second heartbeat on all connections. If a client can't be contacted, we automatically disconnect, calling the `unsubscribe` callback on our Channel class. In addition, [the code, as it stands right now](https://github.com/rails/actioncable), is a joy to read. Short, focused classes with well-named and terse methods. In addition, it's extremely well documented. DHH ain't no slouch. It's a fast read too, weighing in at about 850 lines of Ruby and 200 lines of CoffeeScript. ## Performance and Scaling Readers of my blog will know that my main focus is on performance and Ruby app speed. It's been vaguely claimed that WebSockets offers some sort of scaling or performance benefit to polling. That makes some intuitive sense - surely, large sites like Facebook can't make a 3-second polling interval work. But moving from polling to WebSockets involves a big trade-off. You're trading a high volume of HTTP requests for a high volume of *persistent connections*. And persistent connections, in a virtual machine like MRI that lacks true concurrency, sounds like trouble. Is it? ### Persistent connections > Also note that your server must provide at least the same number of database connections as you have workers. The default worker pool is set to 100, so that means you have to make at least that available. Action Cable's server uses EventMachine and Celluloid under the hood. However, while Action Cable uses a worker pool to send messages to clients, it's just a regular old Rack app and will need to be configured for concurrency in order to accept many incoming concurrent connections. What do I mean? Let's turn to `thor`, a WebSockets benchmarking tool. It's a bit like `siege` or `wrk` for WebSockets. We're going to open up 1500 connections to an Action Cable server running on Puma (in default mode, Puma will use up to 16 threads), with varying incoming concurrency: | Simultaneous WebSocket connections | Mean connection time | | -------- | -------- | | 3 | 17ms | | 30 | 196ms | | 300 | 1638ms | As you can see, Action Cable slows linearly in response to more concurrent connections. Allowing Puma to run in clustered mode, with 4 worker processes, improves results slightly: | Simultaneous WebSocket connections | Mean connection time | | -------- | -------- | | 3 | 9ms | | 30 | 89ms | | 300 | 855 ms | Interestingly, these numbers are slightly better than a [node.js application I found](https://github.com/websockets/ws/tree/master/examples), which seemed to completely crumple under higher load. Here are the results against this node.js chat app: | Simultaneous WebSocket connections | Mean connection time | | -------- | -------- | | 3 | 5ms | | 30 | 65ms | | 300 | 3600 ms | Unfortunately, I can't really come up with a great performance measure for *outbound* messaging. Really, we're going to have to wait to see what happens with Action Cable in the wild to know the full story behind whether or not it will scale. For now, the I/O performance looks at least comparable to Node. That's surprising to me - I honestly didn't expect Puma and Action Cable to deal with this all that well. I suspect it still may come crashing down in environments that are sending many large pieces of data back and forth quickly, but for ordinary apps I think it will scale well. In addition, the use of the Redis pubsub backend lets us scale horizontally the way we're used to. ## What other tools are available? That concludes our look at Action Cable. What alternatives exist for the Rails developer? ### Polling Let's take the example from above - basically pushing "notifications", like "new message!", out to a waiting client web browser. Instead of pushing, we'll have the client basically ask an endpoint for our notification partial every 5 seconds. ```javascript function webNotificationPoll(url) { $.ajax({ url : url, ifModified : true }).done(function(response) { $('#notifications').html(response); // maybe you call some fancy JS here to pop open the notification window, do some animation, whatever. }); } setInterval(webNotificationPoll($('#notifications').data('url'), 5000); ``` Note that we can use HTTP caching here (the ifModified option) to simplify our responses if there are no new notifications available for the user. Our show controller might be as simple as: ```ruby class WebNotificationsController < ApplicationController def show @notifications = current_user.notifications.unread.order(:updated_at) if stale?(last_modified: @notifications.last.updated_at.utc, etag: @notifications.last.cache_key) render :show end # note that if stale? returns false, this action # automatically returns a 304 not modified. end end ``` Seems pretty straightforward to me. Rather than reaching for Action Cable first, in most "live view" situations, I think I'll continue reaching for polling. ### MessageBus [MessageBus](https://github.com/SamSaffron/message_bus) is Sam Saffron's messaging gem. Not limited to server-client interaction, you can also use it for server to server communication. Here's an example from Sam's README: ```ruby message_id = MessageBus.publish "/channel", "message" MessageBus.subscribe "/channel" do |msg| # block called in a background thread when message is received end ``` ```javascript // in client JS MessageBus.start(); // call once at startup // how often do you want the callback to fire in ms MessageBus.callbackInterval = 5000; MessageBus.subscribe("/channel", function(data){ // data shipped from server }); ``` I like the simplicity of the API. On the client side, it doesn't look all that different from stock polling. However, being backed by Redis and allowing for server-to-server messaging means you're gaining a lot in reliability and flexibility. In a lot of ways, MessageBus feels like "Action Cable without the WebSockets". MessageBus does not require a separate server process. ### Sync [Sync](https://github.com/chrismccord/sync) is a gem for "real-time" partials in Rails. Under the hood, it uses WebSockets via Faye. In a lot of ways, I feel like Sync is the "application layer" to Action Cable's "transport layer". The API basically boils down to changing this: ```ruby <%= render partial: 'user_row', locals: {user: @user} %> ``` to this: ```ruby <%= sync partial: 'user_row', resource: @user %> ``` But, unfortunately, it isn't that simple. Sync requires that you sprinkle calls throughout your application any time the `@user` is changed. In the controller, this means adding a `sync_update(@user)` to the controller's update action, `sync_destroy(@user)` to the destroy action, etc. "Syncing" outside of controllers is even more of a nightmare. Sync seems to extend its fingers all through your application, which feels wrong for a feature that's really just an accident of the view layer. Why should my models and background jobs care that my views are updated over WebSockets? ### Others There are several other solutions available. * **ActionController::Live**. This might work if you're OK with never supporting Internet Explorer. * **Faye**. Working with Faye directly is probably more low-level than you'll ever actually need. * **websocket-rails**. While I'd love another alternative for the "WebSockets for Rails!" space, this gem hasn't been updated since the announcement of Action Cable (actually over a year now). ## What do we really want? Overall, I'm left with a question: I know *developers* want to use WebSockets, but what do our *applications* want? Sometimes the furor around WebSockets feels like it's putting the cart before the horse - are we reaching for the latest, coolest technology when polling is *good enough*? > "If you can make WebSockets easier than polling, then why wouldn't you want WebSockets?" I'm not sure if Action Cable is easier to use than polling (yet). I'll leave that as an exercise to the reader - after all, it's a subjective question. You can determine that for yourself. But I think providing Rails developers access to WebSockets is a little bit like showing up at a restaurant and, when you order a sandwich, being told to go make it yourself in the back. WebSockets are, fundamentally, a *transportation* layer, not an *application* in themselves. Let's return to the three use cases for WebSockets I cited above and see how Action Cable performs on each: * **Clients needing to send rapidly to the server.** Action Cable seems appropriate for this sort of use case. I'm not sure how many people are out there writing browser-based games with Rails, but the amount of access the developer is given to the transport mechanism seems wholly appropriate here. * **"Live" data** The "live comments" example. I predict this will be, by far, the most common use case for Action Cable. Here, Action Cable feels like overkill. I would have liked to see DHH and team double down on the "view-over-the-wire" strategy espoused by Turbolinks and make Action Cable something more like "live Rails partials over WebSockets". It would have greatly simplified the amount of work required to get a simple example working. I predict that, upon release, a number of gems that build upon Action Cable will be written to fill this gap. * **Streaming** Honestly, I don't think anyone with a Ruby web server is streaming binary data to their clients. I could be wrong. In addition, I'm not sure I buy into "WebSockets completely obviates the need for HTTP!" rhetoric. HTTP comes with a lot of goodies, and by moving away from HTTP we'll lose it all. Caching, routing, multiplexing, gzipping and lot more. You *could* reimplement all of these things in Action Cable, but why? So when *should* a Rails developer be reaching for Action Cable? At this point, I'm not sure. If you're really just trying to accomplish something like a "live view" or "live partial", I think you may either want to wait for someone to write the inevitable gem on top of Action Cable that makes this easier, or just write it yourself. However, for high-throughput situations, where the client is communicating several times per second back to the server, I think Action Cable could be a great fit. --- ## rack-mini-profiler - the Secret Weapon of Ruby and Rails Speed URL: https://www.speedshop.co/blog/rack-mini-profiler-the-secret-weapon/ `rack-mini-profiler` is a a performance tool for Rack applications, maintained by the talented [@samsaffron](https://twitter.com/samsaffron). [rack-mini-profiler](https://github.com/MiniProfiler/rack-mini-profiler) provides an entire suite of tools for measuring the performance of Rack-enabled web applications, including detailed drill downs on SQL queries, server response times (with a breakdown for each template and partial), incredibly detailed millisecond-by-millisecond breakdowns of execution times with the incredible `flamegraph` feature, and will even help you track down memory leaks with its excellent garbage collection features. **I wouldn't hesitate to say that `rack-mini-profiler` is my favorite and most important tool for developing fast Ruby webapps.** {% marginnote_lazy https://i.imgur.com/DgONqEH.gif||true %} The best part - `rack-mini-profiler` is designed to be run in production. Yeah! You can accurately profile production performance (say that three times fast) with `rack-mini-profiler`. Of course, it also works fine in development. But your development environment is usually a lot different than production - hardware, virtualization environments, and system configuration can all be different and play a huge part in performance. Not to mention Rails' development mode settings, like reloading classes on every request! In this post, I'm going to take a deep dive on `rack-mini-profiler` and show you how to use each of its powerful features to maximize the performance of your Rails app. ## Installation For the purposes of this demo, I'm going to assume you're in a Rails app. The installation procedure is slightly different for a pure Rack app, [see the README for more](https://github.com/MiniProfiler/rack-mini-profiler). First, let's add the following gems to our Gemfile, below any database gems like 'pg' or 'mysql2'. ``` gem 'pg' # etc etc gem 'rack-mini-profiler' gem 'flamegraph' gem 'stackprof' # ruby 2.1+ only gem 'memory_profiler' ``` `rack-mini-profiler` is self explanatory, but what are the other gems doing here? `flamegraph` will give us the super-pretty flame graphs that we're going to use later on. `stackprof` is a stack profiler (imagine that), which will be important when we start building our flame graphs. This gem is Ruby 2.1+ only - don't include it otherwise (`rack-mini-profiler` will fallback to another gem, `fast_stack`). `memory_profiler` will let us use `rack-mini-profiler`'s GC features. Fire up a server in development mode and hit a page. You should see the new speed badge in the upper left. {% marginnote_lazy https://i.imgur.com/3euqzxD.png %} We'll get to what that does in a second. To see a full list of rack-mini-profiler's features and info on how to trigger them, add `?pp=help` to the end of any URL {% sidenote 3 "In more recent versions of rack-mini-profiler, there's also a 'help' button on the speed badge" %} - this prints the help screen and lists the various commands available (all used by adding to the URL query string){% marginnote_lazy https://i.imgur.com/p3zgkM5.png %}. We're going to go through all of these options - but first, we need to make our app run in production mode on our local machine. `rack-mini-profiler` is designed to be used in production. In Rails, your application probably behaves very differently in production mode than in development mode - in fact, most Rails apps are 5-10x slower in development than they are in production, thanks to all the code reloading and asset recompilation that happens per request. So when profiling for speed, run your server in production mode, even when just checking up on things locally. Be careful, of course - change your database.yml file so that it doesn't point towards your *actual* production database (not necessary for Heroku-deployed apps). `rack-mini-profiler` runs in the development environment by default in Rails apps. We're going to enable it in production, and hide it behind a URL parameter. You can also do things like make it visible only to admin users, etc. ```ruby # in your application_controller before_filter :check_rack_mini_profiler def check_rack_mini_profiler # for example - if current_user.admin? if params[:rmp] Rack::MiniProfiler.authorize_request end end ``` Also, I prefer not to use `rack-mini-profiler`'s default storage settings in production. By default, it uses the filesystem to store data. This is slow to begin with, and especially slow if you're on Heroku (which doesn't have a real filesystem). ``` # in an initializer Rack::MiniProfiler.config.storage = Rack::MiniProfiler::MemoryStore ``` If you're forcing SSL in production, you're going to want to turn that off for now. ``` config.force_ssl = false ``` Finally, I need to get the app running in production mode.{% sidenote 4 "Also, if you're having trouble getting the speed badge to show up in production mode and you're using Rack::Deflater or any other gzipping middleware, [you need to do some other stuff](https://github.com/MiniProfiler/rack-mini-profiler#custom-middleware-ordering-required-if-using-rackdeflate-with-rails) to make sure rack-mini-profiler isn't trying to insert HTML into a gzipped response." %} In my case (a Rails 4.2 app), I just have to run the database setup tasks in production mode, compile assets, and add a secret key base to my rails server command: ``` RAILS_ENV=production rake db:reset # CAREFUL! RAILS_ENV=production rake assets:precompile RAILS_ENV=production SECRET_KEY_BASE=test rails s ``` ## The Speed Badge So, you've got the speed badge. In my example app, starting the rails server in development mode and then hitting the root url actually causes two speed badges to show up. `rack-mini-profiler` will create a speed badge for each request made to your app, including some asset requests. In my case, I also got a speed badge for the favicon request. When you click on the speed badge, you can see that `rack-mini-profiler` breaks down the time your page took to render on a per-template basis. It breaks out execution time spent in the layout, for example, and then break out each partial that was rendered as well. Here's an example readout from a different app I work on: ![https://i.imgur.com/e0G29PD.png](https://i.imgur.com/e0G29PD.png) I think this view is pretty self explanatory so far. You're looking at exactly where your time goes on each request in a brief overview. When I look at this view for any given request, here's what I look for: * *How many SQL queries am I generating*? This view generates a total of 9 SQL queries. That strikes me as lot, especially since this is just the homepage for a non-logged-in user. Usually, for simple pages, you wouldn't want to see more than 1 to 3 queries, and almost always you'd like just oneJav query per ActiveRecord model class. * *What's my total request time?* This view is a little slow - 85ms. For a mostly-static and highly visited page like this (like I said, it's the homepage) I'd like to see it be completed in under 50ms. * *What % of time am I spending in SQL?* This view is doing fairly well as far as time spent in SQL goes. I always test my applications with a copy of the production database - this makes sure that my query results match production results as much as possible. Too often, simplistic development databases return 1000 results where a production database would return 100,000. * *How long until DOMContentLoaded fires?* This view took about 250ms between receiving a response and finishing loading all the content. That's pretty good for a simple page like this. Decreasing this time requires frontend optimization - something I can't get into in this post, but doing things like reducing the number of event handlers and frontend JavaScript, and optimizing the order of external resources being loaded onto the page. * *Are any of the parts of the page taking up an extreme amount of time compared to others?* Sometimes, just a single partial is taking up the majority of the page load time. If that's true, that's where I start digging for more information. In this case, the page's load time looks fairly evenly distributed. It looks like one of the post partials here is generating some SQL - a prime candidate for caching (or just getting rid of the query in the first place. There are some other features here in the speed badge. Click any of the SQL links and you'll see the exact query being executed. Here are two as an example: ![https://i.imgur.com/C6XnlTu.png](https://i.imgur.com/C6XnlTu.png) The number on the top left (39.20 ms) is the total time spent between rendering this partial and the next one - notice that this is slightly different than the number to the right, the amount of time actually spent rendering the partial (16.75ms). Whenever I see "lost time" like this, I dig in with the flamegraph tool to see exactly where the time went. We'll get into that in the next section. Notice that `rack-mini-profiler` calls out the exact line in our view that triggered the query. These queries look like the view was probably looking up the `current_user` (or some other user), and that `current_user` `has_one` `Profile`. I probably need to: * Find a way to either eliminate this query or cache the result in the view * Add an `includes` call to the original query so that the profile is loaded along with the User, reducing my query count by 1. I follow this process for every query on the page - see if I can remove it or cache the result. [For my full guide on Rails caching, check this post out](/blog/the-complete-guide-to-rails-caching/). ## The Flamegraph This is one of my favorite parts of `rack-mini-profiler`, and as far as I know, not duplicated anywhere else. If I add `?pp=flamegraph` to my query string, I can get this incredible flamegraph of the same request I outlined above: ![https://i.imgur.com/nr2aojD.png](https://i.imgur.com/nr2aojD.png) The height of the "flame" indicates how deep we are in the stack. Think of the Y axis as stack level, and the X axis as time. You can zoom in and out with your mouse scroll wheel. ![https://i.imgur.com/d9hPsKR.png](https://i.imgur.com/d9hPsKR.png) At the bottom of the page, you'll see a legend, denoting what all the colors refer to. Note that the percentage displayed next to each part is the *percentage of the time the request spent inside that stack frame*. For example, this app is called SomeApp. It looks like we spent 76.42% of our time in the app itself. The other time was taken up by rack middleware (like `lograge`, `airbrake` and `hirefire-resource`) and Rails. Looking at this legend and poking around the flamegraph reveals an interesting tidbit - Paperclip appeared in 28.3% of all stack frames! Yikes. That's way too many for a simple homepage. For this page, I'd look into ways of avoiding calls to Paperclip. It looks like most of the time is spent generating a Paperclip::Attachment's URL. I may experiment with ways to cache or otherwise avoid recalculating that value. ## GC Profiling Here's another awesome part of `rack-mini-profiler` that I haven't seen anywhere else - a set of tools for debugging memory issues *live* on *production!* Even better, it incurs no performance penalty for requests where `rack-mini-profiler` is not enabled! ### profile-gc So let's add `pp=profile-gc` to our query string and see what we get (the output is usually enormous and takes a while to generate): ``` Overview ------------------------------------ Initial state: object count - 331594 , memory allocated outside heap (bytes) 75806422 GC Stats: count : 39, heap_allocated_pages : 1792, heap_sorted_length : 2124, heap_allocatable_pages : 353, heap_available_slots : 730429, heap_live_slots : 386538, heap_free_slots : 343891, heap_final_slots : 0, heap_marked_slots : 386536, heap_swept_slots : 343899, heap_eden_pages : 1439, heap_tomb_pages : 353, total_allocated_pages : 1852, total_freed_pages : 60, total_allocated_objects : 4219050, total_freed_objects : 3832512, malloc_increase_bytes : 960, malloc_increase_bytes_limit : 26868266, minor_gc_count : 27, major_gc_count : 12, remembered_wb_unprotected_objects : 9779, remembered_wb_unprotected_objects_limit : 19558, old_objects : 366156, old_objects_limit : 732312, oldmalloc_increase_bytes : 1344, oldmalloc_increase_bytes_limit : 22319354 New bytes allocated outside of Ruby heaps: 1909904 New objects: 17029 ``` Here's the first section. If that output looks familiar to you, it is - it's the output of `GC.stat`. [GC is a module from the stdlib](http://ruby-doc.org/core-2.2.0/GC.html) that has a whole lot of convenience methods for working with the garbage collector. `stat` gives us that output above. For a full explanation about what each of those values mean, read Sam's post [on how Ruby's GC works](https://web.archive.org/web/20131210200517/http://samsaffron.com/archive/2013/11/22/demystifying-the-ruby-gc). At the bottom, you'll see the new bytes allocated outside of Ruby heaps, along with a count of new objects. Pay attention to any requests that generate abnormally high values here (10+ MB allocated per request, for example). Here's the next section: ``` ObjectSpace delta caused by request: -------------------------------------------- String : 9285 Array : 3641 Hash : 1421 Regexp : 375 MatchData : 349 RubyVM::Env : 214 Proc : 204 Time : 173 Psych::Nodes::Scalar : 168 ... ``` This section shows us the change (that's what delta means) in the total objects in the ObjectSpace that the request caused. For example, after the request, we have 9285 more Strings than before. [ObjectSpace](http://ruby-doc.org/core-2.2.0/ObjectSpace.html) is an incredibly powerful module - for example, with `ObjectSpace.each_object` you can iterate through *every single currently allocated object in the Ruby VM*. That's insane! I don't find this section very useful on its own - though a huge number of app-specific objects (for example, let's say 2,000 `Paperclip::Attachment`s) may be a red flag. ``` ObjectSpace stats: ----------------- String : 175071 Array : 49440 RubyVM::InstructionSequence : 32724 ActiveSupport::Multibyte::Unicode::Codepoint : 27269 Hash : 12748 RubyVM::Env : 8102 Proc : 7806 MIME::Types::Container : 3816 Class : 3371 Regexp : 2739 MIME::Type : 1907 ... ``` Here's the *total* number of Objects, by Class, alive in the VM. This one is considerably more interesting for my application. What's with all of those `MIME::Type`s and `MIME::Types::Container` objects? I suspect it might have something to do with Paperclip, but then again, nearly every gem uses MIME types somehow. In fact, it's such a notorious memory hog that [Richard Schneeman recently saved roughly 50,000 objects from being created with just a single change](https://github.com/mime-types/ruby-mime-types/commit/3aad2228f907e21d8fac302c3f6334231baf2315)! ``` String stats: ------------ 444 : 352 : : 218 : / 129 : :s3_path_url 117 : 116 : 108 : a 106 : href 96 : << 78 : [&"'><] 78 : index 73 : # Amazon S3 Credentials ... ``` Here's the final bit of output - a count on the number of times a certain string was allocated. For example, the string "index" has been allocated 78 times. This output is useful to determine if a string should be extracted to a constant and frozen. For example, [this is what Rack does here with the string "chunked"](https://github.com/rack/rack/blob/master/lib/rack/response.rb#L24). Why would we do this? If, for example, Rack was allocating the string "chunked" 1000 times in a single request, we can reduce that to 1 time by only referring to a constant value. [In fact, that's exactly why this was done](https://github.com/rack/rack/commit/dc53a8c26dc55d21240233b3d83d36efdef6e924). If all of this memory stuff is going over your head, don't worry. I recommend watching [John Crepezzi's talk On Memory](https://www.youtube.com/watch?v=yxhrYiqatdA) for an intro to how memory works in Ruby. ### profile-gc-ruby-head The `pp=profile-gc-ruby-head` {% sidenote 5 "The name of this feature is changing in a future release to profile-memory" %} parameter uses the excellent memory_profiler gem (which you should use on its own to benchmark other code). It's like a hopped-version of `profile-gc` from earlier. Instead of just telling us *what* Strings were allocated during a request, `profile-cg-ruby-head` tells us exactly *what line of code allocated that String*. This is *extremely powerful*. Here's some example output: ``` Total allocated 16986 Total retained 1208 allocated memory by gem ----------------------------------- 769864 paperclip-4.3.0 382958 activesupport-4.2.3 324621 actionpack-4.2.3 274792 activerecord-4.2.3 246966 2.2.2/lib 234562 actionview-4.2.3 118650 newrelic_rpm-3.9.9.275 72424 rack-1.6.4 69359 nokogiri-1.6.6.2 43845 SomeApp/app ..... allocated memory by file ----------------------------------- 689672 ~/gems/paperclip-4.3.0/lib/paperclip/interpolations.rb 224356 ~/gems/activesupport-4.2.3/lib/active_support/core_ext/string/output_safety.rb 136744 ~/gems/actionpack-4.2.3/lib/action_dispatch/routing/route_set.rb 104800 ~/.rbenv/versions/2.2.2/lib/ruby/2.2.0/erb.rb 84291 ~/gems/actionview-4.2.3/lib/action_view/helpers/tag_helper.rb 76272 ~/gems/actionpack-4.2.3/lib/action_dispatch/journey/formatter.rb 53964 ~/gems/activerecord-4.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb 52145 ~/gems/rack-1.6.4/lib/rack/response.rb 43824 ~/.rbenv/versions/2.2.2/lib/ruby/2.2.0/psych/scalar_scanner.rb ..... allocated objects by gem ----------------------------------- 4321 paperclip-4.3.0 2322 activerecord-4.2.3 2300 actionpack-4.2.3 2082 actionview-4.2.3 1726 activesupport-4.2.3 1538 2.2.2/lib 981 newrelic_rpm-3.9.9.275 ``` There's Paperclip again! Note that this output of the first section (allocated memory) is in bytes, which means Paperclip is allocating about 1 MB of objects for this request. That's a lot, but I'm not quite worried *yet*. But this view in general is a good way of finding memory hogs. {% sidenote 6 "The actual RAM cost will always be slightly higher than what is reported here. MRI heaps are not squashed to size." %} Oh - and what does "allocated" mean, exactly? `memory_profiler` differentiates between an "allocated" and a "retained" object. A "retained" object will live on beyond this request, probably at *least* until the next garbage collection. It may or may not be garbage collected at that time. An allocated object may or may not be retained. If it isn't retained, it's just a temporary variable that Ruby knows to throw away when it's done with. Retained objects are ones we should really worry about though - which is contained later on in the report. Keep scrolling down and you'll see the same output, but for "retained" objects only. Pay attention in this area - all of these objects will stick around after this request is over. If you're looking for a memory leak, it's in there somewhere. ### analyze-memory `pp=analyze-memory`, new with `rack-mini-profiler` version 0.9.6, performs some basic heap analysis and lists the 100 largest strings in the heap. Usually, the largest one is your response. I haven't found a lot of use for this view either, but if you're tracking down String allocations, you may find it useful. ## Exception Tracing Did you know that raising an Exception in Ruby is very slow? [Well, it is. Up to 32x slower](http://simonecarletti.com/blog/2010/01/how-slow-are-ruby-exceptions/). And unfortunately, *some people* and *certain gems* use exceptions as a form of flow control. For example, the `stripe` gem for Ruby raises an Exception when a credit card transaction is denied. Your app should not raise Exceptions anywhere during normal operation. Your libraries may be doing this (and of course, catching them) without your knowledge. If you suspect you've got a problem with exceptions being raised and caught in your stack, give `pp=trace-exceptions` a try. ## Conclusion That wraps up our tour of `rack-mini-profiler`. I hope you've enjoyed this in-depth tour of the Swiss army knife of Rack/Ruby performance. Here's my condensed version of my tips from above: * Run `rack-mini-profiler` in production, and set up your local environment to run in production mode when you need seriously accurate performance results. * Pay attention to how many SQL queries a request generates using the speed badge. * Time until DOMContentLoaded is important for front end performance. Keep an eye on it in the speed badge and try not to let it get over 1000ms. * Cache last, not first. Eliminate SQL queries and unnecessary code wherever possible, then cache what you can't get rid of. * Tracking down a memory leak? Use the various GC tools available to track object allocations *in production*. * Exceptions are slow. Use `trace-exceptions` to make sure you aren't silently raising and catching any exceptions. --- ## Scaling Ruby Apps to 1000 Requests per Minute - A Beginner's Guide URL: https://www.speedshop.co/blog/scaling-ruby-apps-to-1000-rpm/ Scaling is an intimidating topic. Most blog posts and internet resources around scaling Ruby apps are about scaling Ruby to *tens of thousands of requests per minute*. That's Twitter and Shopify scale. These are interesting - it's good to know the ceiling, how much Ruby can achieve - but not very useful for the majority of us out there that have apps bigger than 1 server but less than 100 servers. Where's the "beginner's guide" to scaling? {% sidenote 1 "I think the problem is that most people aren't comfortable writing about how big they are until they're huge." %} Thus, most scaling resources for Ruby application developers are completely inappropriate for their needs. The techniques Twitter used to scale from 10 requests/second to 600 requests/second {% marginnote_lazy https://i.imgur.com/x1MVcq7.gif|Me, after reading how a 1000 req/sec app scaled and my app only gets 10 requests/minute|true" %} are not going to be appropriate for getting your app from 10 requests/minute to 1000 requests/minute. Mega-scale has its own unique set of problems - database I/O especially becomes an issue, as your app tends to scale horizontally (across processes and machines) while your database scales vertically (adding CPU and RAM). All of this combines to make scaling a tough topic for most Rails application developers. When do I scale up? When do I scale down? Since I'm limiting this discussion to 1000 rpm or less, here's what I won't discuss: scaling the DB or other datastores like Memcache or Redis, using a high-performance message queue like RabbitMQ or Kafka, or distributing objects. Also, I'm not going to *tell* you how to get faster response times in this post, although doing so will help you scale. Also, I won't cover devops or anything beyond your application server (Unicorn, Puma, etc.) First, although it seems shocking to admit, I've spent my entire professional career deploying applications to the Heroku platform.{% sidenote 2 "I work for small startups with less than 1000 requests/minute scale. Most of the time, you're the sole developer or one of a handful. For small teams at small scales like this, I think Heroku's payoff is immense. Yes, you can pay perhaps even 50% more on your server bill, but the developer hours it saves screwing with Chef/Ansible/Docker/DevOps Flavor Of The Week pays off big time." %} I just don't have the experiences to share on scaling custom setups (Docker, Chef, what-have-you) on non-Heroku platforms. Second, when you're running less than 1000 requests/minute, your devops workflow doesn't really need to be specialized all that much. All of the material in this post should apply to all Ruby apps, regardless of devops setup. As a consultant, I've gotten to see quite a few Rails applications. And most of them are *over-scaled* and *wasting money*. Heroku’s dyno sliders and the many services of AWS make scaling simple, but they also make it easy to scale even when you don’t need to. Many Rails developers think that scaling dynos or upping their instance size will make their application faster {% sidenote 3 "Yes, scaling dynos on Heroku will NEVER make your application faster *unless* your app has requests queued and waiting most of the time (explained below). Even PX dynos will only make performance more *consistent*, not *faster*. Changing instance *types* on AWS though (for example, T2 to M4) may change performance characteristics of app instances." %}. When they see that their application is slow, their first reflex is to scale dynos or up their instance sizes (indeed - Heroku support will usually encourage them to do just this! Spend more money, that will solve the problem!). Most of the time though, it doesn't help their problem. Their site is still slow. As a glossary for this post: *host* refers to a single host machine, virtualized or physical. On Heroku, this is a Dyno. Sometimes people will call this a *server*, but for this post, I want to differentiate between your *host machine* and the *application server* that runs on that machine. A single *host* may run many *app servers*, like Unicorn or Puma. On Heroku, a single host runs a single app server. An *app server* has many *app instances*, which may be separate "worker" processes (like Unicorn) or threads (Puma when running on JRuby in multithreaded). For the purposes of this post, a multi-threaded web server with a single app instance on MRI (like Puma) is not an *app instance* because threads cannot be executed at the same time. Thus, a typical Heroku setup might have 1 host/dyno, with 1 app server (1 Puma master process) with 3-4 app instances (Puma clustered workers). **Scaling increases throughput, not speed**. Scaling hosts only speeds up response times if requests are spending time waiting to be served by your application. If there are no requests waiting to be served, scaling only wastes money. In order to learn about how to scale Ruby apps correctly from 1 to 1000 requests/minute, we're going to need to learn a considerable amount about how your application server and HTTP routing actually works. **I'm going to use Heroku as an example, but many custom devops setups work quite similarly.** Ever wondered exactly what the "routing mesh" was or where requests get queued before being routed to your server? Well, you're about to find out. ## How requests get routed to app servers One of the most important decisions you can make when scaling a Ruby web application is what application server you choose. Most Ruby scaling posts are thus out of date, because the Ruby application server world has changed dramatically in the last 5 years, and most of that whirlwind of change has happened only in the last year. However, to understand the advantages and disadvantages of each application server choice, we're going to have to learn how requests even get routed to your application server in the first place. Understandably, a lot of developers don't understand how, exactly, requests are routed and queued. It isn't simple. Here's the gist of what most Rails devs already understand about Heroku does it {% marginnote_lazy https://i.imgur.com/zy0XzzZ.gif|So the router load balances the Unicorns? Or the Pumas?|true" %}: * "I think routing changed between Bamboo and Cedar stacks." * "Didn't RapGenius got pretty screwed over back in the day? I think it was because request queueing was being incorrectly reported." * "I should use Unicorn. Or, wait, I guess Heroku says I should use Puma now. I don't know why." * "There's a request queue somewhere. I don't really know where." Heroku's documentation on HTTP routing is a good start, but it doesn't quite explain the whole picture. For example, it's not immediately obvious *why* Heroku recommends Unicorn or Puma as your application server. It also doesn't really lay out where, exactly, requests get "queued" and which queues are the most important. So let's follow a request from start to finish! ### The life of a request {% marginnote_lazy https://i.imgur.com/aawbrN5.png %}When a request comes in to yourapp.herokuapp.com, the first place it stops is a load balancer. These load balancers' job is to make sure the load between Heroku's routers is evenly distributed - so they don't do much other than decide to which router the request should go. The load balancer passes off your request to whichever router it thinks is best (Heroku hasn't publicly discussed how their load balancers work or how the load balancers make this decision). Now we're at the Heroku router. There are an undisclosed number of Heroku routers, but we can safely assume that the number is pretty large (100+?). The router's job is to *find your application's dynos* and *pass on the request to a dyno*. So after spending about 1-5ms locating your dynos, the router will attempt to connect to a *random dyno* in your app. Yes, a random one. This is where RapGenius got tripped up a few years ago (back then, Heroku was at best unclear and at worst misleading about how the router chose which dyno to route to). Once Heroku has chosen a random dyno, it will then wait *up to five seconds* for that dyno to accept the request and open a connection. While this request is waiting, it is placed in the router's request queue. However, *each router* has *its own* request queue, and since Heroku hasn't told us how many routers it has, there could be a *huge* number of router queues at any given time for your application. Heroku *will* start throwing away requests from the request queue if it gets too large, and it will also try to quarantine dynos that are not responding (but again, it only does this on an individual router basis, so *every router* on Heroku has to individually quarantine bad dynos). {% sidenote 4 "All of this is *basically* how most custom setups utilize nginx. See this DigitalOcean tutorial. Sometimes nginx plays the role of both load balancer and reverse-proxy in these setups. All of this behavior can be duplicated using custom nginx setups, though you may want to choose more aggressive settings. Nginx can actually actively send health-check requests to upstream application servers to check if they're alive. Custom nginx setups tend not to have their own request queues, however." %} There are two critical details here for Heroku users: the router will *wait up to 5 seconds for a successful connection to your dyno* and *while it's waiting, other requests will wait in the router request queue*. ### Connecting to your server - the importance of server choice The router {% sidenote 5 "Custom setup people - when I say router, you say 'nginx' or 'Apache'." %} attempting to connect to the server is *the most critical* stage for you to understand, and what happens differs *greatly* depending on your choice of web server. Here's what happens next, depending on your server choice: #### **Webrick (Rails default)** Webrick is a single-thread, single-process web server. It will keep the router's connection open until it has downloaded the entirety of the request from the router. The router will then move on to the next request. Your Webrick server will then take the request, run your application code, and then send back the response to the router. During all of this time, your host is busy and will not accept connections from other routers. If a router attempts to connect to this host while the request is being processed, the router will wait (up to 5 seconds, on Heroku) until the host is ready. The router will not attempt to open other connections to other dynos while it waits. The problems with Webrick are exaggerated with slow requests and uploads. If someone is trying to upload a 4K HD video of their cat over a 56k modem, you're out of luck - Webrick is going to sit there and wait while that request downloads, and will not do anything in the meantime. Got a mobile user on a 3G phone? Too bad - Webrick is going to sit there and not accept any other requests while it waits for that user's request to slowly and painfully complete. Webrick can't deal well with slow client requests or slow application responses. #### **Thin** Thin is an event-driven, single-process web server. {% sidenote 7 "There's a way to run multiple Thins on a single host - however, they must all listen on different sockets, rather than a single socket like Unicorn. This makes the setup Heroku-incompatible." %} Thin uses EventMachine under the hood (this process is sometimes called *Evented I/O*. It works not unlike Node.js.), which gives you several benefits, in theory. Thin opens a connection with the router and starts accepting parts of the request. Here's the catch though - if suddenly that request slows down or data stops coming in through the socket, Thin will go off and do something else. This provides Thin some protection from *slow clients*, because no matter how slow a client is, Thin can go off and receive other connections from other routers in the meantime. Only when a request is fully downloaded will Thin pass on your request to your application. In fact, Thin will even write very large requests (like uploads) to a temporary file on the disk. Thin is multi-threaded, not multi-process, and threads only run one at a time on MRI. So while actually running your application, your host becomes unavailable (with all the negative consequences outlined under the Webrick section above). Unless you get very fancy with your use of EventMachine, too, Thin cannot accept other requests while waiting for I/O in the application code to finish. For example - if your application code POSTs to a payments service for credit card authorization, Thin cannot accept new requests while waiting for that I/O operation to complete *by default*. Essentially you'd need to modify your application code to send *events* back to Thin's EventMachine reactor loop to tell Thin "Hey, I'm waiting for I/O, go do something else". [Here's more about how that works.](http://www.bigfastblog.com/rubys-eventmachine-part-3-thin) Thin can deal with slow client requests, but it can't deal with slow application responses or application I/O without a whole lot of custom coding. #### Unicorn Unicorn is a single-threaded, multi-process web server. Unicorn spawns up a number of "worker processes" (app instances), and those processes all sit and listen on a single Unix socket, coordinated by the "master process". When a connection request comes in from a host, it does *not* go to the master process, but instead directly to the Unicorn socket where all of the worker processes are waiting and listening. This is Unicorn's special sauce - no other Ruby web servers (that I know of) use a Unix domain socket as a sort of "worker pool" with no "master process" interference. A worker process (which is only listening on the socket because it isn't processing a request) accepts the request from the socket. It waits on the socket until the request is fully downloaded (setting off alarm bells yet?) and then stops listening on the socket to go process the request. After it's done processing the request and sending a response, it listens on the socket again. Unicorn is vulnerable to slow clients {% sidenote 8 "You can use nginx in a custom setup to buffer requests to Unicorn, eliminating the slow-client issue. This is exactly what Passenger does, below." %} in the same way Webrick is - while downloading the request off the socket, Unicorn workers cannot accept any new connections, and that worker becomes unavailable. Essentially, you can only serve as many slow requests as you have Unicorn workers. If you have 3 Unicorn workers and 4 slow requests that take 1000ms to download, the fourth request will have to sit and wait while the other requests are processed. This method is sometimes called *multi-process blocking I/O*. In this way, Unicorn can deal with slow application responses (because free workers can still accept connections while another worker process is off working) but not (very many) slow client requests. Notice that Unicorn's socket-based model is a form of *intelligent routing*, because only available application instances will accept requests from the socket. #### Phusion Passenger 5 Passenger uses a hybrid model of I/O - it uses a multi-process, worker-based structure like Unicorn, however it also includes a buffering reverse proxy. This is important - it's a bit like running nginx in front of your application's workers. In addition, if you pay for Passenger Enterprise, you can run multiple app threads on each worker (like Puma, below). To see why Phusion Passenger 5's built-in reverse proxy (a customized nginx instance written in C++, *not* Ruby) is important, let's walk through a request to Passenger. Instead of a socket, Heroku's router connects to `nginx` directly and passes off a request to it. This `nginx` is a specially optimized build, with a whole lot of fancy techniques that make it extremely efficient at serving Ruby web applications. It will download the *entire request* before forwarding it on to the next step - protecting your workers from slow uploads and other slow clients. Once it has completed downloading the request, `nginx` forwards the request on to a HelperAgent process, which determines which worker process should handle the request. Passenger 5 can deal with slow application responses (because its HelperAgent will route requests to unused worker processes) *and* slow clients (because it runs its own instance of `nginx`, which will buffer them). #### Puma (threaded only) Puma, in its default mode of operation, is a multi-threaded, single-process server. When an application connects to your host, it connects to an EventMachine-like Reactor thread, which takes care of downloading the request, and can asynchronously wait for slow clients to send their entire request (again, just like Thin). When the request is downloaded, the Reactor *spawns a new Thread* that communicates with your application code, and that thread processes your request. You can specify the maximum number of application Threads running at any given time. Again, in this configuration, Puma is multi-threaded, not multi-process, and threads only run one at a time on MRI Ruby. What's special about Puma, however, is that unlike Thin, you don't have to modify your application code to gain the benefits of threading. Puma automatically yields control back to the process when an application thread waits on I/O. If, for example, your application is waiting for an HTTP response from a payments provider, Puma can still accept requests in the Reactor thread or even complete other requests in different application threads. So while Puma can deliver a big performance increase while waiting on I/O operations (like databases and network requests) while actually running your application, your host becomes unavailable during processing, with all the negative consequences outlined under the Webrick section above. Puma (in threaded-only mode) can deal with slow client requests, but it can't deal with slow, CPU-bound application responses. #### Puma (clustered) Puma has a "clustered" mode, where it combines its multi-threaded model with Unicorn's multi-process model. In clustered mode, Heroku's routers connect to Puma's "master process", which is essentially just the Reactor part of the Puma example above. The master process' Reactor downloads and buffers incoming requests, then passes them to any available Puma worker sitting on a Unix socket (similar to Unicorn). In clustered mode, then, Puma can deal with slow requests (thanks to a separate master process whose responsibility it is to download requests and pass them on) and slow application responses (thanks to spawning multiple workers). ### But what does it all mean? So, if you've been paying attention so far, you've realized that a scalable Ruby web application needs **slow client protection** in the form of request buffering, and **slow response protection** in the form of some kind of concurrency - either multithreading or multiprocess/forking (preferably both). That only leaves **Puma in clustered mode** and **Phusion Passenger 5** as scalable solutions for Ruby applications on Heroku running MRI/C Ruby. If you're running your own setup, Unicorn with nginx becomes a viable option. Each of these web servers make varying claims about their "speed" - I wouldn't get too caught up on it. All of these web servers can handle 1000s of requests per minute, meaning that it takes them less than 1ms to actually handle a request. If Puma is 0.001ms faster than Unicorn, then that's great, but it really doesn't help you very much if your Rails application takes 100ms on average to turn around a request. The biggest difference between Ruby application servers is not their speed, but their varying I/O models and characteristics. As I've discussed above, I think that Puma in clustered mode and Phusion Passenger 5 are really the only serious choices for scaling Ruby application because their I/O models deal well with slow clients and slow applications. They have many other differences in features, and Phusion offers enterprise support for Passenger, so to really know which one is right for you, you'll have to do a full feature comparison for yourself. ### "Queue time" - what does it mean? As we've seen through the above explanation, there isn't really a single "request queue". In fact, your application may be interacting with hundreds of "request queues". Here are all the places a request might "queue": * At the load balancer, Unlikely, as load balancers are tuned to be very fast. (~10 load balancer queues?) * At any of the 100+ Heroku routers. Remember that each router queue is separate (100+ router queues). * If using a multiprocess server like Unicorn, Puma or Phusion Passenger, queueing at the "master process" or otherwise inside the host (1 queue per host). So how in the heck does New Relic know how to report queue times? Well, this is how RapGenius got burned. In 2013, RapGenius got burned hard when they discovered that Heroku's "intelligent routing" was not intelligent at all - in fact, it was completely random. Essentially, when Heroku was transitioning from Bamboo to Cedar stacks, they *also* changed the load balancer/router infrastructure for *everyone* - Bamboo and Cedar stacks both! So Bamboo stack apps, like RapGenius, were suddenly getting random routing instead of intelligent routing {% sidenote 9 "By intelligent routing, we just mean something better than random. Usually intelligent routing involves actively pinging the upstream application servers to see if they're available to accept a new request. This decreases wait time at the router." %} Even worse, Heroku's infrastructure *still reported stats* as if it had intelligent routing (with a *single* request queue, not one-queue-per-router). Heroku would report queue time back to New Relic (in the form of a HTTP header), which New Relic displayed as the "total queue time". However, that header was only reporting the time that particular request spent *in the router queue*, which, if there are 100s of routers, could be extremely low, regardless of load at the host! {% sidenote 10 "Imagine - Heroku connects to Unicorn's master socket, and passes a request onto the socket. Now that request spends 500ms on the socket waiting for an application worker to pick it up. Previously, that 500ms would be unnoticed because only router queue time was reported." %} Nowadays, New Relic reports queue times based on an HTTP header reported by Heroku called `REQUEST_START`. This header marks the time when Heroku accepted the request at the load balancer. New Relic just subtracts the time that your application worker started processing the request from `REQUEST_START` to get the queue time. So if `REQUEST_START` is exactly 12:00:00 PM, and your application doesn't start processing the request until 12:00:00.010, New Relic reports that as 10ms of queue time. What's nice about this is that it takes into account the time spent at all levels: time at the load balancer, time at the Heroku routers, and time spent queueing on your host (whether in Puma's master process, Unicorn's worker socket, or otherwise).{% sidenote 11 "Of course, by setting the correct headers on your own nginx/apache instance, you can get accurate request queueing times with your custom setup." %} ## When do I scale app instances? **Don’t scale your application based on response times alone.** Your application may be slowing down due to increased time in the request queue, or it may not. If your request queue is empty and you’re scaling hosts, you’re just wasting money. Check the time spent in the request queue before scaling. The same applies to worker hosts. Scale them based on the depth of your job queue. If there aren’t any jobs waiting to be processed, scaling your worker hosts is pointless. In effect, your worker dynos and web dynos are exactly the same - they both have incoming jobs (requests) that they need to process, and should be scaled based on the number of jobs that are waiting for processing. NewRelic provides time spent in the request queue, although there are gems that will help you to measure it yourself. If you’re not spending a lot of time (>5-10ms of your average server response time) in the request queue, the benefits to scaling are extremely marginal. ### Dyno counts must obey Little’s Law. I usually see applications over-scaled when a developer doesn't understand how many requests their server can process per second. They don't have a sense of "how many requests/minute equals how many dynos?" I already explained a practical way to determine this - measuring and responding to changes in request queueing time. But there's also a theoretical tool we can use - [Little’s Law](https://en.wikipedia.org/wiki/Little%27s_law). The Wikipedia explanation is a bit obtuse, so here’s my formulation, adapted slightly: ![Minimum application instances required = average web request arrival rate (req/sec) * average response time (in seconds)](https://i.imgur.com/ch59HBx.png) First off, some definitions - as mentioned above, the application instance is the atomic unit of your setup. Its job is to process a single request independently and send it back to the client. When using Webrick, your application instance is the entire Webrick process. When using Puma in threaded mode, I will define the *entire Puma process* as your application instance when using MRI, and when using JRuby, *each thread* counts as an application instance. When using Unicorn, Puma (clustered) or Passenger, your application instance is *each "worker" process*. {% sidenote 10 "Really, a multithreaded Puma process on MRI should count as 1.5 app instances, since it can do work while waiting on I/O. For simplicity, let's say it is one." %} Let’s do the math for a typical Rails app, with the prototypical setup - Unicorn. Let's say each Unicorn process forks 3 Unicorn workers. So our single-server app actually has 3 application instances. If this app is getting 1 request per second, and its average server response time is 300ms, it only needs 1 * 0.3 = 0.3 app instances to service its load. So we're only using 10% of our available server capacity here! What's our application's theoretical maximum capacity? Just change the unknowns: ![Theoretical maximum throughput = App Instances / Average Response Time](https://i.imgur.com/6jetB5M.gif) So for our example app, our theoretical maximum throughput is 3 / 0.3, or 10 requests per second! That's pretty impressive. But theory is never reality. Unfortunately, Little's Law is only true *in the long run*, meaning that things like a wide, varying distribution of server response times (some requests take 0.1 seconds to process, others 1 second) or a wide distribution of arrival times can make the equation inaccurate. But it's a good "rule of thumb" to think about whether or not you might be over-scaled. {% sidenote 11 "In addition, think about what these caveats mean for scaling. You can only maximize your actual throughput if requests are as close to the median as possible. An app with a predictable response time is a scalable app. In fact, you may obtain more accurate results from Little's Law if, instead of using *average* server response time, you use your *95th percentile* response time. You're only as good as your slowest responses if your server response times are variable and unpredictable. How do you decrease 95th percentile response times? Aggressively push work into background processes, like Sidekiq or DelayedJob." %} Recall again that scaling hosts doesn’t directly increase server response times, it can only increase the number of servers available to work on our request queue. If the average number of requests waiting in the queue is less than 1, our servers are not working at 100% capacity and the benefits to scaling hosts are marginal (i.e., not 100%). The maximum benefit is obtained when there is always at least 1 request in the queue. There are probably good reasons to scale *before* that point is reached, especially if you have slow server response times. But you should be aware of the rapidly decreasing marginal returns. So when setting your host counts, try doing the math with Little’s Law. If you’re scaling hosts when, according to Little’s Law, you're only at 25% or less of your maximum capacity, then you might be scaling prematurely. Alternatively, as mentioned above, spending a large amount of time per-request in the request queue as measured on NewRelic is a good indication that it’s time to scale hosts. #### Checking the math In [April 2007, a presentation was given at SDForum Silicon Valley](http://www.slideshare.net/Blaine/scaling-twitter) by a Twitter engineer on how they were scaling Twitter. At the time, Twitter was still fully a Rails app. In that presentation, the engineer gave the following numbers: * 600 requests/second * 180 application instances (mongrel) * About 300ms average server response time So Twitter's theoretical instances required, in 2007, was 600 * 0.3, or 180! And it appeared that's what they were running. Twitter running at 100% maximum utilization seems like a recipe for disaster - and Twitter did have a lot of scaling issues at the time. It may have been that they were unable to scale to more application instances because they were still stuck with a single database server (yup) and had bottlenecks elsewhere in the system that wouldn't be solved by more instances. As a more recent example, in 2013 at Big Ruby Shopify engineer John Duff gave a presentation on [How Shopify Scales Rails](http://www.slideshare.net/jduff/how-shopify-scales-rails-20443485) ([YouTube](https://www.youtube.com/watch?v=j347oSSuNHA)). In that presentation{% sidenote 12 "[Shopify's Scaling Rails presentation presents a form of Little's Law](https://www.youtube.com/watch?v=j347oSSuNHA#t=7m44s)."%}, he claimed: * Shopify receives 833 requests/second. * They average a 72ms response time * They run 53 application servers with a total of 1172 application instances (!!!) with Nginx and Unicorn. So, Shopify's theoretical required instance count is 833 * 0.072 just ~60 application instances. So why are they using 1172 and wasting (theoretically) 95% of their capacity? If application instances block each other in *any way*, like when reading data off a socket to receive a request, Little's Law will fail to hold. This is why I don't count Puma threads as an application instance on MRI. Another cause can be CPU or memory utilization - if an application server is maxing out its CPU or memory, its workers cannot all work at full capacity. This blocking of application instances (anything that stops all 1172 application instances from operating at the same time) can cause major deviations from Little's Law.{% sidenote 13 "[There is a distributional form of Little's Law](http://web.mit.edu/dbertsim/www/papers/Queuing%20Theory/The%20distributional%20Little's%20law%20and%20its%20applications.pdf) that can help with some of these inaccuracies, but unless you're a math PhD, it's probably out of your reach." %} Finally, [Envato posted in 2013 about how Rails scales for them](http://webuild.envato.com/blog/rails-still-scaling-at-envato/). Here's some numbers from them: * Envato receives 115 requests per second * They run an average of 147ms response time * [They run 45 app instances](http://www.slideshare.net/johnpviner/bank-west-10-deploys-a-day-at-envato-published). So the math is 115 * 0.147, which means Envato theoretically requires ~17 app instances to serve their load. They're running at 37% of their theoretical maximum, which is a good ratio. ## The Checklist: 5 Steps to Scaling Ruby Apps to 1000 RPM Hopefully this post has given you the tools you need to scale to 1000 requests-per-minute. As a reminder, here's what you need to remember: * Choose a multi-process web server with slow client protection and smart routing/pooling. Currently, your only choices are Puma (in clustered mode), Unicorn with an nginx frontend, or Phusion Passenger 5. * Scaling dynos increases throughput, not application speed. If your app is slow, scaling should not be your first reflex. * Host/dyno counts must obey Little's Law. * Queue times are important - if queue times are low (<10ms), scaling hosts is pointless. * Realize you have three levers - increasing application instances, decreasing response times, and decreasing response time variability. A scalable application that requires fewer instances will have fast response times and low response time variability. --- ## Make your Ruby or Rails App Faster on Heroku URL: https://www.speedshop.co/blog/secrets-to-speedy-ruby-apps-on-heroku/ I've seen a lot of slow Ruby web apps. Sometimes, it feels like my entire consulting career has been a slow accumulation of downward-sloping New Relic graphs. Why is the case? If you read [that bastion of intellectual thought, Hacker News](https://twitter.com/shit_hn_says), you'd think it was because Go rocks, Ruby sucks, and Rails is crappy old-news bloatware. Also, something about how concurrency is the future, and dynamic typing is for fake programmers that can't code. {% marginnote_lazy https://i.imgur.com/qQvbbt9.png %} And yet, top-1000 websites like [Basecamp](https://www.youtube.com/watch?v=yhseQP52yIY#t=50m30s), [Shopify](https://docs.shopify.com/partners/partner-resources/for-clients/why-shopify) and [Github](https://status.github.com) consistently achieve server response times of less than 100 milliseconds with Rails. That's pretty good for a dynamic, garbage-collected language, if you ask me. Most of my clients deploy on Heroku nowadays, since it's so easy and the payoff for teams without dedicated devops is obvious. Why spend hours of developer time (worth at least $100/hr in most cases) setting up and maintaining a home-brewed devops setup, when with Heroku you can set it up in minutes?{% marginnote_lazy https://i.imgur.com/6MnUrju.png|Actual client graph. Slopes for the slope throne! %} However, Heroku sometimes makes things a little *too* easy. Ruby apps on Heroku are often slow, with bloated memory requirements and poor webserver choices, leading to hundreds of dollars per month in wasted server costs. In addition, the combination of restricted introspection ability (you can't ssh into a dyne while it's running) and reduced devops skill requirements means that most developers that deploy on Heroku have no idea how to solve the performance problems that they've created. **This article will give you a solid grasp of how to diagnose and speed up slow Rails apps on the Heroku platform**. Some (or even most) of the points here are applicable to non-Heroku deployments, but I've tailored my terminology here to the Heroku environment. ## Memory - Swap is Your Worst Enemy The number one enemy of Ruby applications on Heroku? Memory. Most Unix systems use something called swap space when they run out of RAM. This is essentially the operating system using the file system as RAM. However, the filesystem is a lot slower than RAM - 10-50x slower, in fact. {% marginnote_lazy https://i.imgur.com/amwiTJI.jpg|Heroku's metrics dashboard. Red is swap memory. Red bad, purple good. %}If we run out of memory on Heroku, we’ll start using swap memory instead of regular, fast RAM memory. This can slow your app to a crawl. If you’re using swap memory on Heroku, you’re Doing It Wrong and need to reduce your memory usage through any means available. ### Memory bloat and swap usage Heroku dynos are small. The base 1x dyno carries just 512MB of memory, the 2X 1024MB. While Heroku (correctly) recommends using a worker-based multi-process web server like Puma or Unicorn, far too many Ruby developers don’t know how much memory just 1 worker uses to run their application. This makes it impossible to tune how many server workers are running on each dyno. Instead, developers turn to solutions like `puma-auto-tune`, which are extremely inaccurate and tend to over-estimate how many processes you can run on a dyno. I can't honestly recommend these "automatic" performance tuning solutions (worker killers and 'auto tuners' both) - I've just seen too many cases where the inaccuracy of their measurements causes the dyno to go deep into swap memory, leaving the entire application lurching along at a quarter of its usual speed. Thankfully, it's trivial to solve this problem ourselves. It’s simple math. The maximum number of processes (unicorn workers, puma workers) you can run per dyno is governed by the following formula: ![(Dyno RAM size in MB - memory used by the master worker process) / Memory per process](https://i.imgur.com/s4nDSs2.png) What's the master process? Puma (and Unicorn) use "master processes" to coordinate their subordinate worker processes{% sidenote 1 "What the master process actually does is very different in Puma and Unicorn. In Unicorn, it primarily serves the role of sending signals to child processes and forking new ones if old ones die. In Puma, it actually receives the request in an EventMachine-like Reactor pattern. Phusion Passenger 5 uses *several* additional processes, including it's own instance of nginx!" %}. Here's the output from `ps aux | grep puma` when I run Puma with 3 workers: ``` PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND 47835 0.0 2.8 2576900 117316 s000 S+ 11:33AM 0:08.55 puma 2.11.1 (tcp://0.0.0.0:5000) 47841 0.0 3.4 2646960 142412 s000 S+ 11:33AM 0:03.14 puma: cluster worker 2: 47835 47840 0.0 3.7 2657200 156400 s000 S+ 11:33AM 0:03.09 puma: cluster worker 1: 47835 47839 0.0 3.7 2647508 154096 s000 S+ 11:33AM 0:02.80 puma: cluster worker 0: 47835 ``` The master process usually consumes about ~128 MB of RAM all by itself, but you should test this for your application locally. Passenger 5 uses a separate request server and app helper process, which will also have its own memory needs that you should account for. The process is the same - run your server locally in production mode and use `ps` to check the RSS output. `thin` and `webrick` only use a single process in most Heroku configurations, so none of the above applies to use those servers (setting WEB_CONCURRENCY does nothing). However, using single-process web servers on Heroku can cause major issues if you experience moderate request volume (>60 requests/minute). The reasons why are a topic for another day, but suffice it to say - stick with multi-process web servers on Heroku like Unicorn, Puma and Passenger. Heroku recommends setting the number of worker processes per dyno based on an environment variable called `WEB_CONCURRENCY`. However, they also suggest that most applications will probably have `WEB_CONCURRENCY` set to 3 or 4. This just hasn’t been my experience - most Ruby applications would be comfortable at `WEB_CONCURRENCY=2` or even `WEB_CONCURRENCY=1` for 1X dynos. For example, for a typical mature Rails application, the app will use about ~250 MB in RAM once it’s warmed up. This is a big number (I’ll go into ways to measure it and make it smaller later), but this seems to be the usual size. To measure your own, start your Ruby app in production mode on your local machine (this is important - class loading behavior is very different in production), hit the server with a dozen or so requests, click around the site for awhile, and check memory usage with `ps`. A 1X dyno only has 512MB of RAM available, and the master process of a typical Puma server will use about 128MB of RAM itself. So with `WEB_CONCURRENCY` set to 1, a typical mature Rails application is already using 375MB of RAM! Scaling `WEB_CONCURRENCY` to 2 will use 625MB, sending us sailing by the memory limit of the dyno and causing us to use ultra-slow swap memory.{% sidenote 2 "Which is better - a 1x dyno with two worker processes or a 2x dyno with four worker processes? For scaling and request queueing reasons, the answer is the latter. I'll get into why in a future post." %} So the problem here is twofold - most Ruby applications use way too much memory per process, and most developers don’t set `WEB_CONCURRENCY` correctly based on their application’s RAM usage. Why do most Rails apps use *so much* memory per process? A lot of it is Gemfile cruft. Don’t forget - every single gem you add into your Gemfile increases the amount of memory your Rails server needs per process. Yes, every single line of Ruby code `require`d to run your application increases your memory usage, and decreases the number of servers you can run per dyno. This isn’t the *only* component of your Rails server’s memory usage, but it’s a big part. Use tools like [derailed_benchmarks](https://github.com/schneems/derailed_benchmarks) to measure how much memory each gem adds to your application. Just because you didn’t write a lot of code doesn’t mean a lot of code isn’t being run. Gem files hide a lot of complexity. When you drop in Devise to do simple authentication instead of rolling your own with Rails’ built-in `has_secure_password`, you’re adding thousands of lines of Ruby{% sidenote 3 "3038 lines, as of Devise 3.5. I'm picking on Devise here, and there are plenty of good use cases for Devise, but there are a lot of gems out there that people just drop in their Gemfile instead of writing the 20 lines of Ruby required for user/password auth." %} and ~20mb in RAM usage when you could have done it yourself for ~20 lines of Ruby and a negligible RAM impact. Sometimes you need the “big guns”, but usually you don’t. Be aware of the cost, in terms of Ruby lines added, and RAM usage added, of gems you add to your project. #### In case of leak, break glass So you know how I said I don't like worker-killer gems? Well, there's one special case. If you’ve got a memory leak you can’t track down (more on this in a future post), you need to employ a solution that will restart your workers when they start to use swap memory. There are a lot of ways to do this. Several gems, like puma-worker-killer, will do it for you. {% marginnote_lazy https://i.stack.imgur.com/nlwy8.png|What a leak looks like. Note the steep slope of the graph, which crashes back down to low numbers when the dyno restarts. This graph never really levels off. %} Remember, **you only need to employ a worker killer if your application is leaking memory - not if it’s just bloated**. How do you know the difference between bloat and leaks? Try running your application with just 1 process per dyno (e.g. `WEB_CONCURRENCY=1`) on a 2X dyno. You should have a lot of headroom now to watch your memory usage. Ruby applications memory usage curves, over time, look like logarithmic functions. This is mostly because, as users visit different sections of your site, caches are being warmed, files are being `require`d, and constants are being defined for the first time. Over time (this depends on your request load), these activities have already been performed, so our memory usage starts to level off. If, after a few hours of processing requests, your application is still increasing in memory usage unbounded, you’ve got a leak. If it levels off at some point, you’ve just got bloat. Many developers mistake bloats for leaks because they're not waiting long enough for memory usage to level off. You really need to let the server run for about 24 hours (with incoming requests) to be sure that your memory usage doesn't eventually level off. Remember: memory bloat looks like a logarithm, memory leaks look like linear functions. Worker-killers should only kill workers every hour or so, at maximum. If the worker killer is restarting workers more often than that, you may have your `WEB_CONCURRENCY` set too high. Remember that Ruby apps *always* grow in memory usage, gradually (sometimes not approaching their "level-off" point until 6 hours after restart), and you want your worker killer to only kill workers in extraordinary circumstances - not just because the server is still being warmed up! ## Slow Site, Fast Metrics I've often seen New Relic dashboards that seemed to describe an extra-speedy application. Wow, this app's median response time is less than 100ms! Wow, their request volume is really high too! But once you actually click around the site for a while, you realize those metrics can't be right. The entire site feels sluggish and slow to load. This can be a symptom of two different issues: **inaccurate measurement** and **poor frontend performance**. #### Inaccurate performance metrics Do you use NewRelic? Great! If you’re serving your own assets rather than uploading them to S3 (and this is true of your application if you use the `rails_12factor` gem as recommended by Heroku), NewRelic and the default Heroku metrics page on heroku.com are measuring those asset requests and adding them into your average server response times. Asset responses of most Ruby servers are *fast*. Like, 10-15ms per request fast out-of-the-box. And they’re usually very plentiful - you could have 5-10 asset requests per actual web request. See where I’m going here? If your actual HTML response takes 1000ms (unacceptably slow), but the page also makes 10 asset requests for, say, images and CSS, NewRelic averages all of those requests together and will report your overall server response time as just 110ms! Yikes! That’s going to hide the fact that our site is actually quite slow! You *must* exclude the assets directory from NewRelic’s tracking to get accurate average server response metrics - you can do this in its provided YAML configuration file. Unfortunately, you cannot exclude asset requests from Heroku’s metrics page. Thankfully, if you already use a CDN, like Cloudfront, then each asset is only requested from your server once before it is cached, making asset requests' effect on your metrics quite small. #### Poor frontend performance If your server response times look good, but New Relic's Real User Monitoring (RUM, also sometimes called 'Browser' or 'End-User' timings) is slow, then too much front-end Javascript is being executed, usually attached to the DOMContentLoaded event. Obviously, front-end slowness won’t show up in NewRelic’s server response time metrics. However, it can have a huge impact on page performance. Ruby developers seem to have a habit of dropping the kitchen sink into $(document).ready. To determine if this is the issue, check out Chrome Timeline to see how much time you’re spending executing Javascript on each page. I've got a post on Chrome Timeline coming for another day, but for now, their documentation isn't too bad. An aside: if you're not using a front-end Javascript framework, I highly recommend investigating "view-over-the-wire" technologies like PJAX and Turbolinks for speeding up your frontend. [Here's my exhaustive post on the Turbolinks and PJAX](/blog/100-ms-to-glass-with-rails-and-turbolinks/). ### Poor use of ActiveRecord A simple way to figure out if you’ve got an N+1 query or not is to check how often an SQL query runs per web transaction on NewRelic. If a Transaction Trace shows something like User#find with a count of 30, you know you’ve got a N+1 query. Ideally, there should only be 1 SQL query *per model* used on the page. Any more than a dozen SQL queries per page and you’ve likely got a serious N+1 issue. No matter how many blog posts hound developers about it, Ruby developers still seem to drop N+1 queries into their sites constantly. If you're using ActiveRecord, there’s just not a great excuse for that in 2015 - go read about the `includes` method, and know when to use it and its friends: `joins`, `eager_load`, and `preload`. [Here's some of the relevant documentation](http://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#method-i-includes). Tools like [bullet](https://github.com/flyerhzm/bullet) are somewhat useful, but only marginally. They fall apart when apps have complex stack traces, like when using a Rails Engine, and will often encourage `includes` where it isn't necessary. In addition, `bullet` isn't smart enough to realize when you're eager-loading too much data and instead should be paginating. Instead, I do two things: I make sure my development database is seeded with a large, complex dataset (not the simplistic, small seeds you usually see in projects) that closely mirrors my production data. If my production database has 20000 users, I make sure my seed.rb creates 20000 users. Secondly, I simply pay attention to the number of SQL queries occurring on a page. Watch the logs. Use tools like [rack-mini-profiler](https://github.com/MiniProfiler/rack-mini-profiler). If you see a lot of the same query over and over, you've probably got an N+1. ## Caching. DO IT. {% marginnote_lazy https://i.imgur.com/Y82ScT5.gif|You can do it. Make your response time dreams come true|true %} If your server response times are still greater than 250ms after you’ve knocked down the usual suspects of N+1 queries and memory usage, you need to start caching. If you’re already caching, cache more than you do already. Rails apps can be fast - Shopify, Github, and Basecamp all achieve less than 100ms server response times with millions more requests per hour than you have. You can do it - cache more! Most Ruby developers ignore the cache and then complain about how slow their site is. Ruby is a beautiful language, but it isn’t a fast one. To have a fast site, you need to minimize the amount of Ruby you run on each request and never do the same work twice. The only way to accomplish that is with smart caching. Huge Rails sites like Shopify, Github and Basecamp achieve less than 100ms average response times through smart use of caches. You can too! By default, Rails uses the filesystem for your cache store. That’s super slow on Heroku. Instead, use a networked cache store like Memcache or Redis. I prefer Redis - it’s under more active development and performs better on benchmarks than Memcache. [Here's my exhaustive guide on application caching in Rails including benchmarks.](/blog/the-complete-guide-to-rails-caching/) ## Pay attention to performance in development. Far too many Ruby developers use overly simplistic data in development, usually generated by rake db:seed. Where security concerns permit, use a copy of the production database in development. Production databases are nearly always larger and more complicated than anything in our database seeds, which makes it easier to identify N+1 queries and slow SQL. Queries that return 1,000,000 rows in production should return 1,000,000 rows in development. Use gems like rack-mini-profiler to constantly monitor the speed of your controller actions. ## Use a CDN like Cloudfront. If you’re serving your assets, instead of uploading them somewhere else like Amazon S3, you should be using a CDN between your end user and the application server. This will greatly reduce the load of asset requests on your server, as each asset will only be requested once, and then the cached version will be served from Cloudfront’s servers. Rails’ asset pipeline (via asset digests) will ensure that each time you change your assets, the cache on Cloudfront is expired and the new version will be cached anew. For what it's worth, the performance gained by moving assets entirely over to Amazon S3 has rarely been worth the hassle in my experience. Serving assets from your application server is just fine, especially if you've set up a CDN and each asset is only requested once before being cached on the CDN. You *may* still need to use S3 if you have thousands of assets (images, for example) that make your Heroku app slug too large. ## Be wary of huge requests/responses Before Heroku's routing mesh hands off a request to your dyno, it buffers the request body in a 1024 byte buffer. That's not very large. This means that tasks such as file uploads cannot be fully buffered before being handed off to the dyno, which means that the dyno (if it isn't prepared to deal with so-called 'slow clients') will be locked up while it downloads the request. Whether or not your application is vulnerable to these slow uploads (or other large requests - uploads are just the most common case) is dependent on your choice of web server. In short, it depends on how that web server handles I/O. I'll be getting more into web server choice on Heroku in a future post, but here's the gist of it: **Vulnerable to slow clients/slow uploads on Heroku**: * Unicorn * Thin (unless JRuby) * Goliath (unless JRuby) * Webrick **Not vulnerable to slow clients**: * Puma (protection limited to slow requests, responses are not buffered) * Phusion Passenger 5 (unsure about earlier versions) ## 11 Takeaways - The Checklist for Fast Ruby Apps on Heroku * **Use a performance monitoring solution.** I use NewRelic, but only because it’s the easiest to use on Heroku and I haven’t used it’s main competitor in the Ruby app space, Skylight. Pay attention to NewRelic’s Appdex scores in particular, because they take into account the inherent variance of site response time over time. In addition, pay particular attention to time spent in the request queue for the reasons mentioned above - it’s your most important scaling metric. * **Spend time debugging your top 5 slowest web transactions on a weekly basis.** Another enemy of a well-scaled web-app is performance variance. Server response times that are unpredictable or unevenly distributed require more servers to scale, even when average response times are unaffected. On a weekly basis, check in on your 5 slowest controller actions. NewRelic provides this metric for you. Treat each of those 5 slow transactions as a bug and try to close it out before the end of the week. * **Decide on a maximum acceptable server response time and treat anything more than that as a bug.** One of the reasons Rails developers don’t cache enough is because they don’t know how “slow” a slow average response time is. Decide on one for your application. Most Ruby applications should be averaging less than 250ms. Less than 100ms is a great goal for a performance-focused site or a site that requires extra fast response times or has a high number of requests, like a social media site. Any action that averages more than your maximum acceptable time should be treated as a bug. * **Pay attention to swap usage.** A little bit (less than 25mb) is fine. But a lot is a problem. Debug it ASAP! * **Make sure you're excluding assets directories in your performance monitoring tools.** * **Don't forget about frontend performance**. $(document).ready is not a kitchen sink. Attaching event handlers takes time. [Investigate Turbolinks and PJAX](/blog/100-ms-to-glass-with-rails-and-turbolinks/). * **Eliminate N+1's**. But don't forget to watch how much time it takes to build a complicated query. `includes` and friends are not free. Always be benchmarking. * **Develop with production-like data**. Development databases should not be simplistic, with just a few rows. Dev databases should either be populated by a big seed file or should be copies of production data (if security/privacy concerns permit). A query that returns 10k rows in production should return 10k rows in development. * **Cache all the things**. Cache it. [Read my guide on caching if you haven't already.](/blog/the-complete-guide-to-rails-caching/) * **Deliver assets over a CDN**. * **Use a slow-client protected webserver with multi-process I/O**. You need to be protected from slow requests and you need multiple worker processes per dyno. Currently, if you're on MRI Ruby and on Heroku, your options are Puma and Phusion Passenger 5. --- ## The Complete Guide to Rails Caching URL: https://www.speedshop.co/blog/the-complete-guide-to-rails-caching/ Caching in a Rails app is a little bit like that one friend you sometimes have around for dinner, but should really have around more often. Nearly every Rails app that's serious about performance could use more caching, but most Rails apps eschew it entirely! And yet, intelligent use of caching is usually the only path to achieving fast server response times in Rails - easily speeding up ~250ms response times to 50-100ms. A quick note on definitions - this post will only cover "application"-layer caching. I'm leaving HTTP caching (which is a whole nother beast, and not even necessary implemented *in* your application) for another day. ### Why don't we cache as much as we should? Developers, by our nature, are very different from end-users. We understand a lot about what happens behind the scenes in software and web applications. We know that when a typical webpage loads, a lot of code is run, database queries executed, and sometimes services pinged over HTTP. That takes time. We're used to the idea that when you interact with a computer, it takes a little while for the computer to come back with an answer. End-users are completely different. Your web application is a magical box. End-users have no idea what happens inside of that box.{% marginnote_lazy https://i.imgur.com/X17puIB.gif | Developer perception of end-users.|true %} Especially these days, **end-users expect near-instantaneous response from our magical boxes**. Most end-users wanted whatever they're trying to get out of your web-app *yesterday*. This rings of a truism. Yet, we never set hard performance requirements in our user stories and product specifications. Even though server response time is easy to measure and target, and we know users want fast webpages, we fail to ever say for a particular site or feature: "This page should return a response within 100ms." As a result, performance often gets thrown to the wayside in favor of the next user story, the next great big feature. Performance debt, like technical debt, mounts quickly. **Performance never really becomes a priority until the app is basically in flames** every time someone makes a new request. In addition, caching isn't always easy. **Cache *expiration* especially can be a confusing topic**. Bugs in caching behavior tend to happen at the integration layer, usually the least-tested layer of your application. This makes caching bugs insidious and difficult to find and reproduce. To make matters worse, **caching best practices seem to be frequently changing** in the Rails world. Key-based what? Russian mall caching? Or was it doll? ### Benefits of Caching So why cache? The answer is simple. Speed. With Ruby, we don't get speed for free because our language isn't very fast to begin with {% marginnote_lazy https://i.imgur.com/UDkHBEc.png|Ruby performance in the Benchmarks Game vs Javascript. %} . We have to get speed from *executing less Ruby on each request*. The easiest way to do that is with caching. Do the work once, cache the result, serve the cached result in the future. But how fast do we need to be, really? [Guidelines for human-computer interaction have been known since computers were first developed in the 1960s](https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two). The response-time threshold for a user to feel as if they are *freely navigating* your site, without waiting for the site to load, is 1 second or less. That's not a 1-second *response time*, but 1 second *"to glass"* - 1 second from the instant the user clicked or interacted with the site until that interaction is complete (the DOM finishes painting). 1 second "to-glass" is not a very long time. First, figure about 50 milliseconds for network latency (this is on desktop, latency on mobile is a whole other discussion). Then, budget another 150ms for loading your JS and CSS resources, building the render tree and painting. Finally, figure *at least* 250 ms for the execution of all the Javascript you've downloaded, and potentially much more than that if your Javascript has a lot of functions tied to the DOM being ready. So before we're even ready to consider how long the server has to respond, we're already about ~500ms in the hole. **In order to consistently achieve a 1 second to glass webpage, server responses should be kept below 300ms.** For a 100-ms-to-glass webpage, [as covered in another post of mine](/blog/100-ms-to-glass-with-rails-and-turbolinks/), server responses must be kept at around 25-30ms. 300ms per request is not impossible to achieve without caching on a Rails app, especially if you've been diligent with your SQL queries and use of ActiveRecord. But it's a heck of a lot of easier if you do use caching. Most Rails apps I've seen have at least a half dozen pages in the app that consistently take north of 300ms to respond, and could benefit from some caching. In addition, using heavy frameworks in addition to Rails, like Spree, the popular e-commerce framework, can slow down responses significantly due to all the extra Ruby execution they add to each request. Even popular heavyweight gems, like Devise or ActiveAdmin, add thousands of lines of Ruby to each request cycle. Of course, there will always be areas in your app where caching can't help - your POST endpoints, for example. If whatever your app does in response to a POST or PUT is extremely complicated, caching probably won't help you. But if that's the case, consider moving the work into a background worker instead (a blog post for another day). ### Getting started First, [Rails' official guide on caching](http://guides.rubyonrails.org/caching_with_rails.html) is excellent regarding the technical details of Rails' various caching APIs. If you haven't yet, give that page a full read-through. Later on in the article, I'm going to discuss the different caching backends available to you as a Rails developer. Each has their advantages and disadvantages - some are slow but offer sharing between hosts and servers, some are fast but can't share the cache at all, not even with other processes. Everyone's needs are different. In short, the default cache store, ```ActiveSupport::Cache::FileStore``` is OK, but if you you're going to follow the techniques used in this guide (especially key-based cache expiration), you need to switch to a different cache store eventually. As a tip to newcomers to caching, my advice is to **ignore action caching and page caching**. The situations where these two techniques can be used is so narrow that these features were removed from Rails as of 4.0. I recommend instead getting very comfortable with fragment caching - which I'll cover in detail now. ## Profiling Performance ### Reading the Logs Alright, you've got your cache store set up and you're ready to go. But what to cache? This is where profiling comes in. Rather than trying to guess "in the dark" what areas of your application are performance hotspots, we're going to fire up a profiling tool to tell us exactly what parts of the page are slow. My preferred tool for this task is the incredible [rack-mini-profiler](https://github.com/MiniProfiler/rack-mini-profiler). `rack-mini-profiler` provides an excellent line-by-line breakdown of where *exactly* all the time goes during a particular server response. However, we don't even have to use `rack-mini-profiler` or even any other profiling tools if we're too lazy and don't want to - Rails provides a total time for page generation out of the box in the logs {% marginnote_lazy https://i.imgur.com/wTHHYbr.png %} . It'll look something like this: ``` Completed 200 OK in 110ms (Views: 65.6ms | ActiveRecord: 19.7ms) ``` The total time (110ms in this case) is the important one. The amount of time spent in Views is a total of the time spent in your template files (index.html.erb for example). But this can be a little misleading, thanks to how ActiveRecord::Relations lazily loads your data. If you're defining an instance variable with an ActiveRecord::Relation, such as `@users = User.all`, in the controller, but don't do anything with that variable until you start using it's results in the view (e.g. `@users.each do ...`), then that query (and reification into ActiveRecord objects), will be counted in the Views number. ActiveRecord::Relations are *lazily loaded*, meaning the database query isn't executed until the results are actually accessed (usually in your view). The ActiveRecord number here is also misleading - as far as I can tell from reading the Rails source, this is *not* the amount of time spent executing Ruby in ActiveRecord (building the query, executing the query, and turning the query results into ActiveRecord objects), but only the time spent querying the database (so the actual time spent in DB). Sometimes, especially with very complicated queries that use a lot of eager loading, turning the query result into ActiveRecord objects takes a *lot* of time, and that may not be reflected in the ActiveRecord number here. And where'd the rest of the time go? Rack middleware and controller code mostly. But to get a millisecond-by-millisecond breakdown of *exactly* where your time goes during a request, you'll need `rack-mini-profiler` and the `flamegraph` extension {% marginnote_lazy https://i.imgur.com/h3ZvWGm.png|What the flamegraph looks like in rack-mini-profiler %}. Using that tool, you'll be able to see exactly where every millisecond of your time goes during a request on a line-by-line basis. I'm working on a guide for using `rack-mini-profiler` - if you'd like to hear about that guide when it comes out, be sure to sign up for my newsletter (bottom right). ### Production Mode Whenever I profile Rails apps for performance, **I always do it in production mode**. Not *on* production, of course, but with `RAILS_ENV=production`. Running in production mode ensures that my local environment is close to what the end-user will experience, and also disables code reloading and asset compilation, two things which will massively slow down any Rails request in development mode. Even better if you can use Docker to perfectly mimic the configuration of your production environment. For instance, if you're on Heroku, Heroku recently released some Docker images to help you - but usually virtualization is a mostly unnecessary step in achieving production-like behavior. Mostly, we just need to make sure we're running the Rails server in production mode. As a quick refresher, here's what you usually have to do to get a Rails app running in production mode on your local machine: ``` export RAILS_ENV=production rake db:reset rake assets:precompile SECRET_KEY_BASE=test rails s ``` In addition, **where security and privacy concerns permit, I always test with a copy of production data**. All too often, database queries in development (like User.all) return just 100 or so sample rows, but in production, trigger massive 100,000 row results that can bring a site crashing to its knees. Either use production data or make your seed data as realistic as possible. This is *especially* important when you're making extensive use of `includes` and Rails' eager loading facilities. ### Setting a Goal Finally, I suggest **setting a maximum acceptable average response time, or MAART, for your site**. The great thing about performance is that it's usually quite measurable - and what gets measured, gets managed! You may need two MAART numbers - one that is achievable in development, with your developer hardware, and one that you use in production, with production hardware. Unless you have an extremely 1-to-1 production/development setup, using virtualization to control cpu and memory access, you simply will not be able to duplicate performance results across those two environments (though you can come close). That's OK - don't get tripped up by the details. You just need to be sure that your page performance is in the right ballpark. As an example, let's say we want to build a 100ms-to-glass web app [like in my previous post](/blog/100-ms-to-glass-with-rails-and-turbolinks/). That requires server response times of 25-50ms. So I'd set my MAART in development to be 25ms, and in production, I'd slacken that to about 50ms. My development machine is a little faster than a Heroku dyne (my typical deployment environment), so I give it a little extra time on production. I'm not aware of any tools yet to do automated testing against your maximum acceptable average response time. We have to do that (for now) manually using benchmarking tools. ### Apache Bench So, how do we decide what our site's actual average response time is in development? I've only described to you how to read response times from the logs - so is the best way to hit "refresh" in your browser a few times and take your best guess at the average result? Nope. This is where benchmarking tools like `wrk` and `Apache Bench` come in. `Apache Bench`, or `ab`, is my favorite, so I'll quickly describe how to use it. You can install it on Homebrew with `brew install ab`.{% sidenote 1 "I've been told you may need to 'brew tap homebrew/apache' first for this to work." %} Start your server in production mode, as described earlier. Then fire up Apache Bench with the following settings: ``` ab -t 10 http://localhost:3000/ ``` Obviously, you'll need to change that URL out as appropriate. The -t option controls how long we're going to benchmark for (in seconds). Here's some example output from Apache Bench, abridged for clarity: ``` ... Requests per second: 161.04 [#/sec] (mean) Time per request: 12.419 [ms] (mean) Time per request: 6.210 [ms] (mean, across all concurrent requests) ... Percentage of the requests served within a certain time (ms) 50% 12 66% 13 75% 13 80% 13 90% 14 95% 15 98% 17 99% 18 100% 21 (longest request) ``` The "time per request" would be the number we compare against our MAART. If you also have a 95th percentile goal (95 percent of requests must be faster than X), you can get the comparable time from the chart at the end, next to "95%". Neat, huh? For a full listing of things you can do with Apache Bench, check out the man page. Notable other options include SSL support, KeepAlive, and POST/PUT support. Of course, the great thing about this tool is that you can also use it against your production server! If you want to benchmark heavy loads though, it's probably best to run it against your staging environment instead, so that your customers aren't affected! From here, the workflow is simple - **I don't cache anything unless I'm not meeting my MAART**. If my page is slower than my set MAART, I dig in with `rack-mini-profiler` to see exactly which parts of the page are slow.{% marginnote_lazy https://imgur.com/gtMaUPI.png|Breakdown in rack-mini-profiler %} In particular, I look for areas where a lot of SQL is being executed unnecessarily on every request, or where a lot of code is executed repeatedly. ## Caching techniques ### Key-based cache expiration Writing and reading from the cache is pretty easy - again, if you don't know the basics of it, [check out the Rails Guide on this topic](http://guides.rubyonrails.org/caching_with_rails.html). **The complicated part of caching is knowing when to expire caches**. In the old days, Rails developers used to do a lot of manual cache expiration, with Observers and Sweepers. Nowadays, we try to avoid these entirely, and instead use something called *key-based expiration*. Recall that a cache is simply a collection of keys and values, just like a Hash. In fact, we use hashes as caches all the time in Ruby. Key-based expiration is a cache expiration strategy that expires entries in the cache by making the *cache key* contain information about the *value being cached*, such that when the object changes (in a way that we care about), the cache key for the object also changes. We then leave it to the cache store to expire the (now unused) previous cache key. We never expire entries in the cache manually. In the case of an ActiveRecord object, we know that every time we change an attribute and save the object to the database, that object's `updated_at` attribute changes. So we can use `updated_at` in our cache keys when caching ActiveRecord objects - each time the ActiveRecord object changes, it's updated_at changes, busting our cache. Thankfully, Rails knows this and makes it very easy for us. For example, let's say I have a Todo item. I can cache it like this: ``` <% todo = Todo.first %> <% cache(todo) do %> ... a whole lot of work here ... <% end %> ``` When you give an ActiveRecord object to `cache`, Rails realizes this and generates a cache key that looks a lot like this: ``` views/todos/123-20120806214154/7a1156131a6928cb0026877f8b749ac9 ``` The `views` bit is self-explanatory. The `todos` part is based on the Class of the ActiveRecord object. The next bit is a combination of the `id` of the object (123 in this case) and the `updated_at` value (some time in 2012). The final bit is what's called the template tree digest. This is just an md5 hash of the template that this cache key was called in. When the template changes (e.g., you change a line in your template and then push that change to production), your cache busts and regenerates a new cache value. This is super convenient, otherwise we'd have to expire all of our caches by hand when we changed anything in our templates! Note here that changing anything in the cache key expires the cache. So if any of the following items change for a given Todo item, the cache will expire and new content will be generated: * The class of the object (unlikely) * The object's id (also unlikely, since that's the object's primary key) * The object's `updated_at` attribute (very likely, because that changes every time the object is saved) * Our template changes (possible between deploys) Note that this technique doesn't *actually* expire any cache keys - it just leaves them unused. Instead of manually expiring entries from the cache, we let the cache itself push out unused values when it begins to run out of space. Or, the cache might use a time-based expiration strategy that expires our old entries after a period of time. You can give an Array to `cache` and your cache key will be based on a concatenated version of everything in the Array. This is useful for different caches that use the same ActiveRecord objects. Maybe there's a todo item view that depends on the current_user: ``` <% todo = Todo.first %> <% cache([current_user, todo]) do %> ... a whole lot of work here ... <% end %> ``` Now if the current_user gets updated *or* if our todo changes, this cache key will expire and be replaced. ### Russian Doll Caching Don't be afraid of the fancy name - the DHH-named caching technique isn't very complicated at all. We all know what Russian dolls look like - one doll contained inside the other. Russian doll caching is just like that - we're going to stack cache fragments inside each other. Let's say we have a list of Todo elements: ``` <% cache('todo_list') do %>
    <% @todos.each do |todo| %> <% cache(todo) do %>
  • <%= todo.description %>
    • <% end %> <% end %>
<% end %> ``` But there's a problem with my above example code - let's say I change an existing todo's description from "walk the dog" to "feed the cat". When I reload the page, my todo list will still show "walk the dog" because, although the inner cache has changed, the outer cache (the one that caches the entire todo list) has not! That's not good. We want to re-use the inner fragment caches, but we also want to bust the outer cache at the same time. Russian doll caching is simply using key-based cache expiration to solve this problem. When the 'inner' cache expires, we also want the outer cache to expire. If the outer cache expires, though, we *don't* want to expire the inner caches. Let's see what that would like in our todo_list example above: ``` <% cache(["todo_list", @todos.map(&:id), @todos.maximum(:updated_at)]) do %>
    <% @todos.each do |todo| %> <% cache(todo) do %>
  • <%= todo.description %>
    • <% end %> <% end %>
<% end %> ``` Now, if *any* of the @todos change (which will change @todos.maximum(:updated_at)) or an Todo is deleted or added to @todos (changing @todos.map(&:id)), our outer cache will be busted. However, any Todo items which have not changed will still have the same cache keys in the inner cache, so those cached values will be re-used. Neat, right? That's all there is to it! In addition, you may have seen the use of the `touch` option on ActiveRecord associations. Calling the `touch` method on an ActiveRecord object updates' the record's `updated_at` value in the database. Using this looks like: ``` class Corporation < ActiveRecord::Base has_many :cars end class Car < ActiveRecord::Base belongs_to :corporation, touch: true end class Brake < ActiveRecord::Base belongs_to :car, touch: true end @brake = Brake.first # calls the touch method on @brake, @brake.car, and @brake.car.corporation. # @brake.updated_at, @brake.car.updated_at and @brake.car.corporation.updated_at # will all be equal. @brake.touch # changes updated_at on @brake and saves as usual. # @brake.car and @brake.car.corporation get "touch"ed just like above. @brake.save @brake.car.touch # @brake is not touched. @brake.car.corporation is touched. ``` We can use the above behavior to elegantly expire our Russian Doll caches: ``` <% cache @brake.car.corporation %> Corporation: <%= @brake.car.corporation.name %> <% cache @brake.car %> Car: <%= @brake.car.name %> <% cache @brake %> Brake system: <%= @brake.name %> <% end %> <% end %> <% end %> ``` With this cache structure (and the `touch` relationships configured as above), if we call `@brake.car.save`, our two outer caches will expire (because their `updated_at` values changed) but the inner cache (for `@brake`) will be untouched and reused. ## Which cache backend should I use? There are a few options available to Rails developers when choosing a cache backend: * **ActiveSupport::FileStore** This is the default. With this cache store, all values in the cache are stored on the filesystem. * **ActiveSupport::MemoryStore** This cache store puts all of the cache values in, essentially, a big thread-safe Hash, effectively storing them in RAM. * **Memcache and dalli** `dalli` is the most popular client for Memcache cache stores. Memcache was developed for LiveJournal in 2003, and is explicitly designed for web applications. * **Redis and redis-store** `redis-store` is the most popular client for using Redis as a cache. * **LRURedux** is a memory-based cache store, like ActiveSupport::MemoryStore, but it was explicitly engineered for performance by Sam Saffron, co-founder of Discourse. Let's dive into each one one-by-one, comparing some of the advantages and disadvantages of each. At the end, I've prepared some performance benchmarks to give you an idea of some of the performance tradeoffs associated with each cache store. ### ActiveSupport::FileStore FileStore is the default cache implementation for all Rails applications for as far back as I can tell. If you have not explicitly set `config.cache_store` in production.rb (or whatever environment), you are using FileStore. FileStore simply stores all of your cache in a series of files and folders - in `tmp/cache` by default. #### Advantages **FileStore works across processes**. For example, if I have a single Heroku dyne running a Rails app with Unicorn and I have 3 Unicorn workers, each of those 3 Unicorn workers can share the same cache. So if worker 1 calculates and stores my todolist cache from an earlier example, worker 2 can use that cached value. *However*, this does not work across hosts (since, of course, most hosts don't have access to the same filesystem). So, again, on Heroku, while all of the processes on each dyne can share the cache, they cannot share across dynos. **Disk space is cheaper than RAM**. Hosted Memcache servers aren't cheap. For example, a 30MB Memcache server will run you a few bucks a month. But a 5GB cache? That'll be $290/month, please. Ouch. But disk space is a heckuva lot cheaper than RAM, so if you access to a lot of disk space and have a huge cache, FileStore might work well for that. #### Disadvantages **Filesystems are slow(ish)**. Accessing the disk will always be slower than accessing RAM. However, it might be faster than accessing a cache over the network (which we'll get to in a minute). **Caches can't be shared across hosts**. Unfortunately, you can't share the cache with any Rails server that doesn't also share your filesystem (across Heroku dynes, for example). This makes FileStore inappropriate for large deployments. **Not an LRU cache**. This is FileStore's biggest flaw. FileStore expires entries from the cache based on the *time they were written to the cache*, not *the last time they were recently used/accessed*. This cripples FileStore when dealing with key-based cache expiration. Recall from our examples above that key-based expiration does not actually expire any cache keys manually. When using this technique with FileStore, the cache will simply grow to maximum size (1GB!) and then start expiring cache entries based on the time they were created. If, for example, your todo list was cached first, but is being accessed 10 times per second, FileStore will still expire that item first! Least-Recently-Used cache algorithms (LRU) work much better for key-based cache expiration because they'll expire the entries that haven't been used in a while *first*. **Crashes Heroku dynos** Another nail in FileStore's coffin is it's complete inadequacy for the ephemeral filesystem of Heroku. Accessing the filesystem is extremely slow on Heroku for this reason, and actually adds to your dynes' "swap memory". I've seen Rails apps slow to a total crawl due to huge FileStore caches on Heroku that take ages to access. In addition, Heroku restarts all dynes every 24 hours. When that happens, the filesystem is reset, wiping your cache! #### When should I use ActiveSupport::FileStore? Reach for FileStore if you have *low request load* (1 or 2 servers) and still need a *very large cache* (>100MB). Also, don't use it on Heroku. ### ActiveSupport::MemoryStore MemoryStore is the other main implementation provided for us by Rails. Instead of storing cached values on the filesystem, MemoryStore stores them directly in RAM in the form of a big Hash. ActiveSupport::MemoryStore, like all of the other cache stores on this list, is thread-safe. #### Advantages * **It's fast** One of the best-performing caches on my benchmarks (below). * **It's easy to set up** Simple change `config.cache_store` to `:memory_store`. Tada! #### Disadvantages * **Caches can't be shared across processes or hosts** Unfortunately, the cache cannot be shared across hosts (obviously), but it also can't even be shared across processes (for example, Unicorn workers or Puma clustered workers). * **Caches add to your total RAM usage** Obviously, storing data in memory adds to your RAM usage. This is tough on shared environments like Heroku where memory is highly restrained. #### When should I use ActiveSupport::MemoryStore? If you have one or two servers, with a few workers each, and you're storing very small amounts of cached data (<20MB), MemoryStore may be right for you. ### Memcache and dalli Memcache is probably the most frequently used and recommended external cache store for Rails apps. Memcache was developed for LiveJournal in 2003, and is used in production by sites like Wordpress.org, Wikipedia, and Youtube. While Memcache benefits from having some absolutely enormous production deployments, it is under a somewhat slower pace of development than other cache stores (because it's so old and well-used, if it ain't broke, don't fix it). #### Advantages * **Distributed, so all processes and hosts can share** Unlike FileStore and MemoryStore, *all* processes and dynos/hosts share the exact same instance of the cache. We can maximize the benefit of caching because each cache key is only written once across the entire system. #### Disadvantages * **Distributed caches are susceptible to network issues and latency** Of course, it's much, much slower to access a value across the network than it is to access that value in RAM or on the filesystem. Check my benchmarks below for how much of an impact this can have - in some cases, it's extremely substantial. * **Expensive** Running FileStore or MemoryStore on your own server is free. Usually, you're either going to have to pay to set up your own Memcache instance on AWS or via a service like Memcachier. * **Cache values are limited to 1MB**. In addition, cache keys are limited to 250 bytes. #### When should I use Memcache? If you're running more than 1-2 hosts, you should be using a distributed cache store. However, I think Redis is a slightly better option, for the reasons I'll outline below. ### Redis and redis-store Redis, like Memcache, is an in-memory, key-value data store. Redis was started in 2009 by Salvatore Sanfilippo, who remains the project lead and sole maintainer today. In addition to [redis-store](https://github.com/redis-store/redis-store), there's a new Redis cache gem on the block: [readthis](https://github.com/sorentwo/readthis). It's under active development and looks promising. #### Advantages * **Distributed, so all processes and hosts can share** Like Memcache, *all* processes and dynos/hosts share the exact same instance of the cache. We can maximize the benefit of caching because each cache key is only written once across the entire system. * **Allows different eviction policies beyond LRU** Redis allows you to select your own eviction policies, which gives you much more control over what to do when the cache store is full. For a full explanation of how to choose between these policies, check out the [excellent Redis documentation](http://redis.io/topics/lru-cache). * **Can persist to disk, allowing hot restarts** Redis can write to disk, unlike Memcache. This allows Redis to write the DB to disk, restart, and then come back up after reloading the persisted DB. No more empty caches after restarting your cache store! #### Disadvantages * **Distributed caches are susceptible to network issues and latency** Of course, it's much, much slower to access a value across the network than it is to access that value in RAM or on the filesystem. Check my benchmarks below for how much of an impact this can have - in some cases, it's extremely substantial. * **Expensive** Running FileStore or MemoryStore on your own server is free. Usually, you're either going to have to pay to set up your own Redis instance on AWS or via a service like Redis. * **While Redis supports several data types, redis-store only supports Strings** This is a failure of the `redis-store` gem rather than Redis itself. Redis supports several data types, like Lists, Sets, and Hashes. Memcache, by comparison, only can store Strings. It would be very interesting to be able to use the additional data types provided by Redis (which could cut down on a lot of marshaling/serialization). #### When should I use Redis? If you're running more than 2 servers or processes, I recommend using Redis as your cache store. ### LRURedux Developed by Sam Saffron of Discourse, LRURedux is essentially a highly optimized version of ActiveSupport::MemoryStore. Unfortunately, it does not yet provide an ActiveSupport-compatible interface, so you're stuck with using it on a low-level in your app, not as the default Rails cache store for now. #### Advantages * **Ridiculously fast** LRURedux is by far the best-performing cache in my benchmarks. #### Disadvantages * **Caches can't be shared across processes or hosts** Unfortunately, the cache cannot be shared across hosts (obviously), but it also can't even be shared across processes (for example, Unicorn workers or Puma clustered workers). * **Caches add to your total RAM usage** Obviously, storing data in memory adds to your RAM usage. This is tough on shared environments like Heroku where memory is highly restrained. * **Can't use it as a Rails cache store** Yet. #### When should I use LRURedux? Use LRURedux where algorithms require a performant (and large enough to the point where a Hash could grow too large) cache to function. ## Cache Benchmarks Who doesn't love a good benchmark? [All of the benchmark code is available here on GitHub](https://gist.github.com/nateberkopec/14d6a2fb7fe5da06a1f6). ### Fetch The most often-used method of all Rails cache stores is `fetch` - if this value exists in the cache, read the value. Otherwise, we write the value by executing the given block. Benchmarking this method tests both read and write performance. `i/s` stands for "iterations/second". ``` LruRedux::ThreadSafeCache: 337353.5 i/s ActiveSupport::Cache::MemoryStore: 52808.1 i/s - 6.39x slower ActiveSupport::Cache::FileStore: 12341.5 i/s - 27.33x slower ActiveSupport::Cache::DalliStore: 6629.1 i/s - 50.89x slower ActiveSupport::Cache::RedisStore: 6304.6 i/s - 53.51x slower ActiveSupport::Cache::DalliStore at pub-memcache-13640.us-east-1-1.2.ec2.garantiadata.com:13640: 26.9 i/s - 12545.27x slower ActiveSupport::Cache::RedisStore at pub-redis-11469.us-east-1-4.2.ec2.garantiadata.com: 25.8 i/s - 13062.87x slower ``` Wow - so here's what we can learn from those results: * LRURedux, MemoryStore, and FileStore are so fast as to be basically instantaneous. * Memcache and Redis are still very fast when the cache is on the same host. * When using a host far away across the network, Memcache and Redis suffer significantly, taking about ~50ms per cache read (under extremely heavy load). This means two things - when choosing a Memcache or Redis host, choose the one closest to where your servers are and benchmark its performance. Second, don't cache anything that takes less than ~10-20ms to generate by itself. ### Full-stack in a Rails app For this test, we're going to try caching some content on a webpage in a Rails app. This should give us an idea of how much time read/writing a cache fragment takes when we have to go through the entire request cycle as well. Essentially, all the app does is set `@cache_key` to a random number between 1 and 16, and then render the following view: ``` <% cache(@cache_key) do %>

<%= SecureRandom.base64(100_000) %>

<% end %> ``` #### Average response time in ms - less is better The below results were obtained with Apache Bench. The result is the average of 10,000 requests made to a local Rails server in production mode. * Redis/redis-store (remote) 47.763 * Memcache/Dalli (remote) 43.594 * With caching disabled 10.664 * Memcache/Dalli (localhost) 5.980 * Redis/redis-store (localhost) 5.004 * ActiveSupport::FileStore 4.952 * ActiveSupport::MemoryStore 4.648 Some interesting results here, for sure! Note that the difference between the fastest cache store (MemoryStore) and the uncached version is about 6 milliseconds. We can infer, then, that the amount of work being done by ```SecureRandom.base64(100_000)``` takes about 6 milliseconds. Accessing the remote cache, in this case, is actually slower than just doing the work! The lesson? **When using a remote, distributed cache, figure out how long it actually takes to read from the cache**. You can find this out via benchmarking, like I did, or you can even read it from your Rails logs. Make sure you're not caching anything that takes longer to read than it does to write! ## Conclusions Hopefully, this article has given you all you need to know to get out there and use caching more in your Rails apps. It really is the key to extremely performant Rails sites. --- ## How To Use Turbolinks to Make Fast Rails Apps URL: https://www.speedshop.co/blog/100-ms-to-glass-with-rails-and-turbolinks/ A perceived benefit of a client-side JS framework is the responsiveness of its interface - updates to the UI are instantaneous. A large amount of application logic (and, usually, state) lives on the client, instead of on the server. The client-side application can perform most tasks without running back to the server for a round-trip. As a result, in the post-V8 era, many developers think traditional server-side languages and frameworks (Ruby, Python, even Java) are simply too slow for modern web applications, which are now supposed to behave like native applications, with instantaneous responses. Is Rails dead? Can the old Ruby web framework no longer keep up in this age of "native-like" performance? Shopify (an e-commerce provider that lets you set up your own online shop) has [over 150,000 customers](http://www.sec.gov/Archives/edgar/data/1594805/000119312515129273/d863202df1.htm) and is a [Top 1000](http://www.alexa.com/siteinfo/shopify.com) site on Alexa.{% marginnote_lazy https://i.imgur.com/F49D9La.png %} In addition, Shopify hosts their customers' sites, with an average of 100ms response times for over 300 million monthly page views. Now that's Web Scale. And they did it all on Rails. They're not the only ones doing huge deployments with blazing fast response times on Rails. [DHH claims Basecamp's average server response time is 27ms](https://www.youtube.com/watch?v=yhseQP52yIY). [Github averages about 60ms](https://status.github.com/). But fast response times are only half of the equation. If your server is blazing fast, but you're spending 500-1000ms on each new page load rendering the page, setting up a new Javascript VM, and re-constructing the entire render tree, your application will be fast, but it won't be *instantaneous*. Enter [Turbolinks](http://github.com/rails/turbolinks). ## Turbolinks and other "view-over-the-wire" technologies Turbolinks received (and still receives) a huge amount of flak from Rails developers upon its release. Along with [pjax](https://github.com/defunkt/jquery-pjax), from which it evolved, Turbolinks represented a radical shift in the way Rails apps were intended to be architected. Suddenly, Rails apps had similar characteristics to the "Javascript single-page app" paradigm: no full page loads, pushState usage, and AJAX. But there was a critical difference between Turbolinks and their SPA brethren: instead of sending *data* over the wire, Turbolinks sent *fully rendered views*. Application logic was reclaimed from the client and kept on the server again. Which meant we got to write more Ruby! I'll call this approach "views-over-the-wire", becausing we're sending HTML, not data. "View-over-the-wire" technologies like turbolinks and pjax have laid mostly out of the limelight since their release in ~2012, despite their usage by such high-profile sites as [Shopify](https://www.shopify.com/technology/15646068-rebuilding-the-shopify-admin-improving-developer-productivity-by-deleting-28-000-lines-of-javascript) and Github. But with Rails 5, Turbolinks is getting a nice upgrade, with new features like partial replacement and a progress bar with a public API. So I wanted to answer for myself the question: how does building an application with Turbolinks feel? Can it be not just fast, but *instantaneous*? And just what is an instantaneous response? Thankfully, the guidelines for human-computer interaction speeds have remained constant since [they were first discovered in the late 60's](https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two): * **0.1 second** is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result. * **1.0 second** is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data. * **10 seconds** is about the limit for keeping the user's attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect. {% sidenote 1 "This is the Nielsen Norman group's interpretation of the linked paper. See the rest of their take on response times here." %} ## Can Turbolinks help us achieve sub-0.1 second interaction? In the non-Turbolinks world, Rails apps usually live in the 1.0 second realm. They return a response in 100-300ms, spend about 200ms loading the HTML and CSSOM, [a few hundred more ms rendering and painting](https://developers.google.com/web/fundamentals/performance/critical-rendering-path/render-tree-construction?hl=en), and then likely loads of JS scripting tied to the onload event. But in the Turbolinks/pjax world, we get to cut out a lot of the work that usually happens when accessing a new page. Consider: 1. When using Turbolinks, you don't throw away your entire Javascript runtime on every page. We don't have to attach a thousand event listeners to the DOM, nor throw out any JS variables between page loads. This requires you to rethink the way you write your Javascript, but the speed benefits are big. 1. When using Turbolinks partial replacement, we don't even throw away the entire DOM, instead changing only the parts we need to change. 1. We don't have to parse and tokenize the CSS and JS ever again - the CSS Object Model is maintained. All of this translates into eliminating 200-700ms on each new page. This lets us move out of the 1 second human-computer interaction realm, and start to flirt with the 100 ms realm of "instantaneous" interaction. As an experiment, I've constructed a TodoMVC app using Rails 5 (still under active development) and Turbolinks 3. You can find [the application here](https://github.com/nateberkopec/todomvc-turbolinks) and [the code here](https://github.com/nateberkopec/todomvc-turbolinks). It also utilizes partial replacement, a new feature in Turbolinks 3. Using your browsers favorite development tools, you can confirm that most interactions in the app take about 100-250ms, from the time the click event is registered until the response is painted to the screen. By comparison, the reference Backbone implementation for TodoMVC takes about 25-40ms. Consider also that our Backbone implementation isn't making any roundtrips to a server to update data - most TodoMVC implementations use LocalStorage. I can't find a live TodoMVC implementation that uses a javascript framework *and* a server backend, so the comparison will have to suffice. In any case, after removing network timing, Turbolinks takes about the same amount of time to update the page state and paint the new elements about as quickly as Backbone. And we didn't even have to write any new Javascript! Turbolinks also forces you to do a lot of things you should be doing already with your frontend Javascript - idempotent functions, and not treating your DOM ready hooks like a junk drawer. A lot of people griped about this when Turbolinks came out - but you shouldn't have been doing it anyway! Other than asking to re-evaluate the way you write your frontend JS, Turbolinks doesn't ask you to change a whole lot about the way you write Rails apps. You still get to use all the tools you're used to on the backend, because what you're doing is still The Web with a little spice thrown in, not [trying to build native applications in Javascript](http://www.quirksmode.org/blog/archives/2015/05/web_vs_native_l.html). ### load is dead, all hail load! Look in any Rails project, and for better or for worse, you're going to see a lot of this: ```javascript $(document).ready(function () { ... } ); ``` Rails developers are usually pretty lazy when it comes to Javascript (although, most *developers* are pretty lazy). [JQuery waits for DOMContentLoaded to fire](https://github.com/jquery/jquery/blob/master/src/core/ready.js#L81) before handing off execution to the function in `ready`. But Turbolinks takes DOMContentLoaded away from us, and [gives us a couple other events instead](https://github.com/rails/turbolinks#events). Try attaching events to these instead, or using JQuery's `.on` to attach event handlers to the document (as opposed to individual nodes). This removal of the `load` and `DOMContentLoaded` events can wreak havoc on existing Javascript that uses page ready listeners everywhere, and why I wouldn't recommend using Turbolinks on existing projects, and using it for greenfield only. ### Caching - still a Rails dev's best friend DHH has said it a hundred times: Rails is an extraction from Basecamp, and is best used when building Basecamp-like applications. Thus, DHH's [2013 talk on Basecamp's architecture](https://www.youtube.com/watch?v=yhseQP52yIY) is very valuable - most Rails apps should be architected this way, otherwise you're going to be spending most of your time fighting the framework rather than getting things done. Most successful large-scale Rails deployments make extensive use of caching. Ruby is a (comparatively) slow language - if you want to keep server response times below 300ms, you simply have to minimize the amount of Ruby you're running on every request and never calculate the same thing twice. Caching can be a double-edged sword in small apps, though. Sometimes, the amount of time it takes to read from the cache is more than it takes to just render something. When evaluating whether or not to cache something, *always* test your apps locally *in production mode*, with production-size datasets (hopefully just a copy of the production DB, if your company allows it). The only way to know for sure if caching is the right solution for a block of code is to measure, measure, measure. And how do we do that? ### rack-mini-profiler and the flamegraph [rack-mini-profiler](https://github.com/MiniProfiler/rack-mini-profiler) {% marginnote_lazy https://i.imgur.com/1J1hlPt.png %} has become an indispensable part of my Ruby workflow. It's written by the incredible Sam Saffron, who's doing absolutely vital work (along with others) on Ruby speed over at [RubyBench.org](https://rubybench.github.io). rack-mini-profiler puts a little white box at the upper left of a page, showing you exactly how long the last request took to process, along with a breakdown of how many SQL queries were executed. The amount of unnecessary SQL queries I've eliminated with this tool must number in the thousands. But that's not even rack-mini-profiler's killer feature. If you add in the `flamegraph` gem to your Gemfile, you get a killer flame graph showing exactly how long rendering each part of your page took. This is invaluable when tracking down exactly what parts of the page took the most time to render. ### Chrome Timeline - the sub-100ms developer's best friend When you're aiming for a sub-100ms-to-glass Turbolinks app, every ms counts. So allow me introduce you to my little friend: the Chrome Timeline.{% marginnote_lazy https://i.imgur.com/izy57wD.png %} This bad boy shows you, in flamegraph format, exactly where each of your 100ms goes. Read up on Google's documentation on exactly how to use this tool, and exactly what means what, but it'll give you a great idea of which parts of your Javascript are slowing down your page. ### Non-RESTful redirects 100ms-to-glass is *not* a lot of time. In most cases, you may not even have time to redirect. Consider this typical bit of Rails controller logic: ```ruby def create thing = Thing.new(params[:thing]) if thing.save redirect_to #... ``` Unfortunately, you've just doubled the number of round-trips to the server - one for the POST, and one for the GET when you get your response back from the redirect. I've found that this alone puts you beyond 100ms. With remote forms and Turbolinks, it seems to be far better to do non-RESTFUL responses here and [just re-render the (updated) index view](https://github.com/nateberkopec/todomvc-turbolinks/blob/master/app/controllers/todos_controller.rb#L8). ### Be wary of partials Partials in Rails have always been slow-ish. They're fast enough if you're aiming for 300ms responses, but in the 100ms-to-glass world, we can't really afford any less than a 50ms server response time. Be wary of using partials, cache them if you can, and always benchmark when adding a new partial. ### Response time goals and Apache Bench Another key tool for keeping your Turbolinks-enabled Rails app below 100ms-to-glass is to keep your server response times ridiculously fast - 50ms should be your goal. Apache Bench{% marginnote_lazy https://i.imgur.com/nsSgaBj.png %} is a great tool for doing this, but siege is another popular tool that does the same thing - slams your web server as fast as it can to get an idea of your max requests/second. Be sure to load up your rails server in production mode when benchmarking with these tools so that you don't have code reloading slowing down each request! In addition, be sure to test with production (or extremely production-like) data. If queries return 100 rows in development but return 1000 rows in production, you're going to see very different performance. We want our development environment to be as similar to production as possible. ### Common mistakes * **Be absolutely certain that a page load that you *think* is Turbolinks enabled, is actually Turbolinks enabled.** Click a link with the Developer console open - if the console says something like "Navigated to http://www.whatever.com/foo", that link wasn't Turbolinks-enabled. * **Don't render responses that do things like append items to the current page.** Instead, a Turbolinks-enabled action should return a full HTML page. Let Turbolinks do the work of swapping out the document, instead of writing your own, manual "$("#todo-list").append("<%= j(render(@todo)) %>");" calls. For an example, [check out my TodoMVC implementation](https://github.com/nateberkopec/todomvc-turbolinks/blob/master/app/views/todos/index.html.erb), which only uses an index template. Keep state (elements having certain classes, for example) in the template, rather than allowing too much DOM state to leak into your Javascript. It's just unnecessary work that Turbolinks frees us from doing. ### Limitations and caveats Turbolinks may not fare well in more complex UI interactions - the TodoMVC example is very simple. Caching *will* be required when scaling, which some people think is too complex. I think that with smart key-based expiration, and completely avoiding manual cache expiration or "sweepers", it isn't too bad. Turbolinks doesn't play great with client side JS frameworks, due to the transition cache and the lack of the `load` event. Be wary of multiple instances of your app being generated, and be careful of Turbolinks' transition cache. Integration testing is still a pain. Capybara and selenium-webdriver, though widely used, remain difficult to configure properly and, seemingly no matter what, are not deterministic and occasionally experience random failures. ### Conclusion: "View-over-the-wire" is better than it got credit for Overall, I quite enjoyed the Turbolinks development experience, and mostly, as a user, I'm extremely impressed with the user experience it produces. Getting serious about Rails performance and using a "view-over-the-wire" technology means that Rails apps will deliver top-shelf experiences on par with any clientside framework. UPdate