Forest header image

Symfony Finland
Random things on PHP, Symfony and web development

BigPipe and the Symfony Framework

The Drupal 8 development team is doing great improvements to the product regarding HTTP caching. With validation tags, contexts and so on it feels familiar to people working with Symfony2 framework and the FOSHttpCacheBundle.

Another interesting addition is an adaptation of Facebook's BigPipe rendering strategy, which downloads HTML snippets in parallel for improved load times. Symfony2 as a HMVC framework can load bits asynchronously, but there is room to improve.

BigPipe itself was introduced by Facebook way back in 2010, but after the initial hype has not been a house hold name. Inspired by Facebook, the Finnish social media site IRC Galleria released their implementation of BigPipe to the wild.

But as Drupal is a strong brand, I expect the hype to pick up again. Especially since it was highlighted in a recent post on the future of Drupal from the BDFL:

Currently, Drupal 8 is the only CMS with BigPipe deeply integrated across the board for both core and contributed modules—they merely have to provide some cacheability metadata and need no awareness of the technical minutiae.

This is an exciting prospect indeed. The code in Drupal 8 is directly not usable in applications built with the Symfony framework. But Symfony2 does have a lot of the infrastructure in place, in fact we've had an easy method of rendering fragments since 2013: New in Symfony 2.2: The new fragment sub-framework

Rendering with hinclude.js is currenty quite barebones and there has been talk about about replacing it with something else. Consider the following snippet from the original Facebook article:

<script type="text/javascript">
    id: "pagelet_composer",

Coupled with what happens on the browser end it's clear this is more complex compared to rendering controllers with render_hinclude does currently. Consider how this quote from the Facebook BigPipe article:

At the client side, upon receiving a pagelet response via “onPageletArrive” call, BigPipe’s JavaScript library first downloads its CSS resources; after the CSS resources are downloaded, BigPipe displays the pagelet by setting its corresponding placeholder div’s innerHTML to the pagelet’s HTML markup.

So we will need to pass an ID, rendered HTML and related JS and CSS assets. ID and HTML will be automatically generated, but currently we would need to pass the related JS and CSS assets manually as there is no de-facto way of binding static resources with controllers.

Asset handling in Symfony2 has not traditionally been a strong point. The Puli intiative aims to provide one generic, framework independent solution for managing all non-PHP files, such as YAML, XML, CSS, HTML, images in PHP: Puli is PHP's Next Package Revolution

So with a render_bigpipe Twig Symfony extension in place, we could call something like this in the template:

{{ render_bigpipe(controller('BatmanBundle:Bat:Signal', {
    'css': '/batman/css/signal.css',
    'js': '/batman/js/signal.js'
})) }}

The above is obviously all related to the Backend framework, so we'll also need to find (or create) a suitable library to handle loading of the pagelets. There are plenty of implementations out there, so looking at the front something like Ping-Play for Scala might have something Symfony could adopt.

BigPipe may not be that novel or even a proper standard, but I look forward to widespread client side includes. This allows sharding applications to many different technologies such as Scala, Node.js and Symfony2, loading parts of from where they are best served. Not just from within a single framework or CMS.

With HTTP/2 reducing overhead of individual requests the future looks faster, even on congested mobile networks. It's worth noting that technology originates from five years back and things like web components have come a long way since that.

Related links:

Written by Jani Tarvainen on Saturday September 26, 2015
Permalink - Tags: esi, drupal, bigpipe, facebook, javascript, caching, puli

« Facebook is an Open Source Powerhouse - Comparing HTTP/1.1 and HTTP/2 on Aircraft WiFi »