Symfony Benchmarks: PHP 5.6, HHVM 3.11 and PHP 7.0.1
Facebook's PHP compatible HHVM runtime pushed the envelope in terms of performance and is a stable platform to host PHP applications on. Large services like Wikipedia have been able to give their aging PHP codebases a lease of life with it.
Since the first functional versions of PHP 7.0.0 were made available, there have been a number of benchmarks comparing PHP 5.6, HHVM and PHP 7. Some of them run a hello world application with frameworks such as Laravel, while some run default installations complete products such as of WordPress, Drupal and Magento.
The recently released eZ Platform is a CMS built on the Symfony framework. It's a good representation of a modern PHP application with complex functionalities and no legacy code from the 2000's. Thus making a good candidate benchmarks for testing an application built with the Symfony Framework (version 2.7.8).
So let's see how an application built with the Symfony2 framework fares on PHP 5.6, HHVM 3.11 and PHP 7.0.1.
Test setup
For the test I used the demo installation of the platform with the standard theme and configuration. By default the production configuration uses the standard Symfony Reverse Proxy written in PHP.
The proxy supports ESI rendering and other complex functions, so it might have to do a fair bit of processing, more than just storing HTML and a timestamp.
The server setup is two of virtual servers hosted on UpCloud (Try free trial VPS) and connected through the internal 1 Gbit network. The setup has 1 CPU and 1 GB of RAM. This is not abundant, but ample to run the three runtimes with stash caching limiting DB calls.
Benchmarking was an anonymous user against the front page with staple features such as navigations, image lifts and article listing. In addition a REST API call for content locations was done to get an idea how a reduced bootstrap and functionality are reflected.
Caches were warmed up by running the code manually. The process is as follows:
- Clear all caches and restart runtime & web server
- Load Front page
- Load Top Stories page
- Load Projects page
- Load Front page
- Start benchmarks
The test cycles were ran three times and an average of the results were used in the results.
Memory usage and Response Times
Most benchmarks focus on the performance enhancement for processing, but I'll start with memory usage. The above cache warmup process was ran in the dev environment of Symfony to display the dev toolbar, so the numbers are not comparable to prod environments.
Load Front page with empty runtime and Symfony caches
Peak memory usage between all the runtimes seems relatively even. PHP 7 uses the least memory, but HHVM and PHP 5.6 are practically equal.
At response times PHP 7 is the clear winner by bootstrapping more than twice as fast after a complete cache purge. PHP 5.6 beats HHVM by a small margin, leaving HHVM to be the slowest from a standing start.
Load Top Stories page
On second page load with cached data memory usage varies greatly with PHP 5.6 consuming as much as 18 MB of where as HHVM cuts it down to 4.5 MB, but PHP making do with a mere 2 MB.
In response times PHP 5.6 is now left behind significantly, where as HHVM and PHP 7 are close to a tie.
Load Projects page
Here the previous trend from cached pages continue. PHP 7 excels in memory usage, with HHVM and PHP 5.6 requiring two and five fold increases respectively.
In response time HHVM manages to produce a slightly lower response time than PHP 7. PHP 5.6 is not far behind in real world terms.
Load Front page
As expected memory usage after caches are filled drops down to a level comparable to that of the other pages. PHP yields the lowest memory use for the most complex page, but still four times as high as the paltry 2 MB required for the Top Stories page.
HHVM suffered the most from empty caches on the first load, being slightly slower than PHP 5.6. With caches filled it inches close to PHP 7 and leaves PHP 5.6 far behind.
Requests served per second
After warming up the caches a set of runs is done with each runtime:
- Front Page without Symfony Proxy (concurrencies 1, 10, 50)
- Front Page with Symfony Proxy (concurrencies 1, 10, 50)
- API without Symfony Proxy (concurrencies 1, 10, 50)
- API with Symfony Proxy (concurrencies 1, 10, 50)
Front Page without Symfony Proxy
The front page is loaded without the Symfony reverse proxy. HHVM and PHP 7 are quite close to each other where as PHP 5.6 is able to serve significantly less results.
Front Page with Symfony Proxy
With the Symfony Proxy in front, PHP 7 easily outperforms both PHP 5.6 and HHVM. The difference only grows as concurrency rises with PHP 7, possibly pointing to an advantage with less memory usage with limited resources.
API without Symfony Proxy
On REST API calls without the Symfony proxy results remain fairly consistent regardless of concurrency. PHP 5.6 delivers around 30 requests per second where HHVM improves to an average of 60 requests and PHP 7 pull ahead by adding 15-20 ontop of HHVM.
API with Symfony Proxy
Behind the Symfony Proxy the API results closely resemble those of the front page results. At low concurrency PHP 5.6 and HHVM are tied, but HHVM scales better. Both of these pale in comparison to PHP 7.
Conclusion
Both HHVM and PHP 7 are a great option to increase application performance of existing PHP applications without adding resources or modifying application code. Installation is straighforward given an existing PHP-FPM setup, a simple FastCGI configuration change is sufficient.
PHP 7 is a significant improvement both over PHP 5.6 and HHVM. Together with the highest performance it features an impressively low memory usage. PHP 7 should be especially valuable in memory limited applications where individual processes consume a large amount of RAM. Swift recovery from cache purges improves real world performance.
HHVM does a great job in improving over PHP 5.6 and seems to be close to PHP 7 in performance, but falls behind in startup cycles and when serving cached content via the Symfony Proxy. HHVM performance could be improved by an estimated 20% with Repo Authoritative mode, which I could not turn for tests due to an issue with WholeProgram caching.
In the next article we observe how increasing CPU and RAM affects the performance of the same application.