Mitigating High-Traffic with Simple Static Content Generation

Your application probably does not need to execute some code each time it is used. We have been caching for decades; reducing database queries, using key-value stores, OPcode caching and the list runs on. Things become overwhelming. I believe often we can find the best answers in simplicity; complexity should not be introduced where a simple solution can help.

TL;DR

  1. When your application bootstraps, build combinations of all the possible states that might result in different content among the views.
  2. Generate the actual content for all those combinations generated above.
  3. Store the generated content as static files, serve with nginx.
  4. Proxy the non-static requests directly to the application.

Depending on the design of your application, you probably will see a considerable number of requests getting served by nginx directly without any involvement of your backend application.

To Cache, or not to Cache?

You probably cannot staticify everything in your application. You have to choose what needs to stay static and what needs to remain dynamic. A simple rule of thumb is volatility.

  • If new records/data/rows get added frequently, that’s a big no-no for staticifying it.
  • If the number of possible values is very high for any parameter you wish to base content generation upon, that is also a big no-no.
  • You cannot use static sites for validation and processing of a form.

Imagine you will generate static content based on a user’s account type and some other parameters.

So, you ran into 5 x 4 x 2 = 40 possible states. So the math is easy. Things can go out proportions unless you are careful! So carefully generate and manage the combinations.

Generation

Simply think of this passing the context variables to a templating engine for it to render an output. That’s it. You got it.

Hyperlinking and Directory Structures

One approach I personally prefer is to create directories for each possible value combination. So it would look like,

Entry

The application should have a dynamic entrypoint, from where the user is redirected to the appropriate static directory.

I just did a quick write-up, not sure if this will spark anyone’s interest. If you want to know more about this, please leave a comment so that I can get to know that I have to write another followup!

(I used this approach to build a USSD service, where authentication doesn’t need to be performed by the application. So yeah that’s why it was practical!)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.