When a Facebook post gets several million likes and is backed by a WordPress post on one of our servers, we handle some serious traffic. Maintaining the fast response we strive for can be a challenge.
We continually search for ways to increase server performance for our busiest sites. For the sake of reliability and stability, most of our servers run Centos 6, but Centos has become a source of frustration. It’s obsolete. Centos 7 (based on Redhat Enterprise Linux 7) is in the works and it will be a big relief when it finally appears. When I read that RHEL 7 will be based on Fedora 19, I decided it was time to try Fedora as a server. Fedora lists “First” as one of its core values and tries to implement the most recent innovations with releases twice per year.
Our latest server is running Fedora 20. I can hardly believe the performance gain. To test, we moved 5 very busy sites onto this server. A server load average is a rough estimate of how many processes had to wait over the last second. It correlates directly with web site performance, speed as perceived by a site visitor. At quiet times on the previous server I was seeing loads at 1 and 2. At the same times on this server, the load is 0.0, sometimes spiking up to 0.2. At busy times I was seeing loads of 2 and 3, sometimes spiking to 5 and 7. On this server it’s 0.2 and 0.3 with spikes up to 0.5. On top of that, spikes disappear much more rapidly. Where I had been seeing spikes fade away in 5 or 10 seconds, now I am seeing them fade in 2 or 3 seconds. That’s huge. It means not only that site visitors are getting page loads 10 times (or more) faster, but the fastest performance is being seen by 5 times as many visitors.
The gains came from a long list of improvements. Before getting into what they are, a disclaimer. To make this understandable to people who aren’t techno nerds, I’m oversimplifying a bit.
Where loads are coming from can get really complicated with so many things going on at the same time in a server. It matters what those things are because they are dynamically interacting with each other. For example, under certain conditions we see loads more than double when the visitor hit rate doubles. Sometimes the increase is logarithmic and sometimes quadratic. But it can also increase by less than double. It depends on the server software, the site application software and on the interaction between them. Having said that, here’s the list:
- The Linux kernel V 3.13 – Recent optimizations have a substantial effect.
- The network stack and drivers have been improved.
- Web server software: Apache 2.4.9 or Nginx 1.5.13 – Nginx is a bit faster, but when some features are essential, Apache is a better choice. Apache 2.4 outperforms 2.2.
- XFS file system. This is disk reads and writes. Under load it’s twice as fast as EXT4. Disk operations are the most common performance bottleneck.
- MariaDB (from MySQL 5.5) – this version is said to be 2 – 3 times faster in the most common use cases.
- PHP 5.5.10 – by being less tolerant of errors and changing semantics, performance gains of 20 to 40% are often possible.
- Zend Opcache – saves the step of compiling scripts into executable code by holding the code in memory. It also saves a disk read.
- PHP-FPM – this keeps the php binary up and running so that the web server process can pass it script names to execute, passing back the results.
We are now offering virtual private servers set up this way. We are also offering the opportunity to run sites in a shared VPS set up this way. You can find it in our ordering system. This is a strong value for sites too busy to run on a cPanel shared hosting server.
I was asked, “Where’s Varnish?” Varnish is a page cache which resides in memory. It saves a disk access when a static file (html page, image, etc.) has been read recently. This is useful on proxy servers with a server farm behind them. In other situations it’s counter productive. Modern operating systems try to take full advantage of available memory. All memory not in immediate use is allocated to disk cache buffers. When more memory is needed by a program it is taken from the least recently used (LRU) memory pages. In other words, it’s already doing what Varnish does and much more efficiently in terms of the whole system. Adding Varnish is much more likely to increase disk reads than reduce them.
You will find many pages on the Internet recommending Varnish incorrectly. Something like 80% of the information and recommendations about getting the most out of your web site or server is dead wrong, too incomplete to be useful or doesn’t apply to the most common environments. People like to write about the wonderful new things they’ve learned, but often fail to realize they don’t know enough about the bigger picture. Be careful what you believe.