What’s that sound?
Have you just started streaming that highly anticipated live sports match for the big streaming service you work for?
Has that match just started?
And now you can hear a faint rumbling.
But wait — it’s getting louder and louder and … yep, you got it — it’s a thundering herd.
A thundering herd may sound great in the new Attenborough doc, but trust us: a thundering herd isn’t great when it’s about to hit your Origin.
A thundering herd happens when hundreds or thousands of simultaneous requests are made for content that isn’t yet cached (like at the start of a live match or concert). If your cache would blindly pass on all of those requests to your origin server, that server might collapse under the load of the stampede.
I know what you’re wondering now. How do we calm the herd?
Not just by caching. Two words. Cache locking.
Cache locking is achieved by adding an Origin Shield cache in front of Unified Origin or as part of your CDN.
No cache locking:
But if cache locking is enabled, the cache will hold off on handling additional requests for the content and wait until Origin has returned its response to the first request, which can then be used to fill the cache and fulfill all requests for the content.
This reduces the load on Origin significantly, since it will need to deal with only one request instead of a multitude.
With cache locking:
Varnish Cache and Nginx are two of the most popular caching solutions on the market today. Varnish Cache works off a thread worker-based architecture and enables cache locking by default. Nginx requires its cache locking to be configured.
The following is an example of configuring cache locking inside the location directive.
proxy_http_version 1.1;
proxy_set_header Connection “”;
proxy_cache_background_update on;
proxy_cache_use_stale updating;
Thanks for reading! We hope this helps. Stay tuned for next month’s customer support question!