By now, you have not only heard a lot about Progressive Web Apps (PWAs), but also seen them live on the web. You may simply not know about it, and that’s fair, since most of the features that make a web app progressive aren’t visible to the eyes of a viewer. According to Google Lighthouse – a PWA analytic tool – the features that define a Progressive Web App are:

  • HTTP/2 connection that improves loading speed,
  • Service Worker set to cache data and let a PWA work offline,
  • Progressive Web App Manifest that allows a PWA be installed on a home screen.

Broadly speaking, the main idea behind these features is to improve performance and step up user experience.

Do PWAs fulfill their promises?
Hype always surrounds ‘new and improved’ technologies, and Progressive Web Apps are no exception. Yet, PWAs appeared only 3 years ago and haven’t had much time or chance to prove their worth. That’s why it’s reasonable to doubt their bold promises.

Do web apps actually improve after being revamped as progressive web apps? Are you free to fill your PWA with all sorts of heavy data and be sure it’ll load fast just because the web app is ‘progressive’? And, given the complex definition of progressiveness, are all PWAs progressive to the same extent?

To find the answers to these questions, we’ve chosen eight Progressive Web Apps of four different categories from the PWA directory hosted by Appspot, and used Google Lighthouse to audit them.

Please, note that loading time, as well as ‘performance’ and ‘accessibility’ scores can slightly vary from a session to session. We’ve put an average score of 3 audit sessions.

News:
Forbes
PWA: 45 | Performance: 8 | Accessibility: 83 | Best Practices: 56
A meaningful page loads in: 6 – 8 seconds

Forbes’ mobile PWA was a huge success in spring 2017. The app increased user engagement by 100% and had its 6 – 13 seconds of loading time reduced to only 1 second. Unfortunately, it seems that no improvements were made to Forbes’ desktop web app version.

Google Lighthouse reports lack of both PWA Manifest and a Service Worker, which means caching and offline mode are disabled. The web app does use HTTP/2 to speed up data transfer, but has a huge network payload (4,400kb out of ~1,600kb) and isn’t performing fast enough for 3G network.

The Guardian
PWA: 64 | Performance: 36 | Accessibility: 89 | Best Practices: 89
A meaningful page loads in: 1.5 – 3 seconds

Since The Guardian PWA is linked to a Manifest and registers a Service Worker, it’s fully available offline and can be saved to a home screen or browser. It uses HTTP/2 for all of its resources, too, and loads a page in less than 3 seconds. Still, Google Lighthouse estimates this Progressive Web App to be not fast enough on 3G and reports a significant network payload (2,300kb/1,600kb).

Takeaway:
Although both web apps have a similar design structure and use HTTP/2, they’re still quite different in their loading speed. According to Lighthouse, the problem lies in how Forbes handles visual data and stylesheets: they’re too heavy to load and need optimization. Guardian could also standardize its images, which would improve overall performance, but this PWA’s situation is noticeably better as is.

Blogs:
Medium
PWA: 55 | Performance: 43 | Accessibility: 89 | Best Practices: 75
A meaningful page loads in: 4 – 5 seconds

As it transpired, Medium’s current web app neither registers a Service Worker, nor is it linked to a Manifest. Still, reborn as a 55% PWA with HTTP/2 connection, the web page improved its initial Performance score from 26 to 43. The time Medium needs to load a first meaningful paint has sped up from 7 seconds to 4.5, yet it still takes the web app almost 30 seconds to become interactive.

Smashing Magazine
PWA: 100 | Performance: 70 | Accessibility: 94 | Best Practices: 94
A meaningful page loads in: 1 – 2 seconds

Smashing Magazine can take pride in recreating its website as a fully Progressive Web App. According to Lighthouse, it’s almost impeccable: all the Manifest settings are tuned up, caching is on, and the HTTP/2 data transfer is fast enough on 3G. In contrast to Smashing Magazine’s previous 4 – 6 seconds of loading, the main page now needs 2 – 3 seconds to become fully interactive.

Takeaway:
Despite a bright and vivid look, the Smashing Magazine’s revamped interface doesn’t impede its performance. Meanwhile, Medium has a simpler UI but is much slower. Yet, Medium PWA can improve by reducing server response time and enabling lazy-loading for off-screen images.

Product home pages:
Ionic Framework
PWA: 45 | Performance: 49 | Accessibility: 89 | Best Practices: 81
A meaningful page loads in: 4 seconds

Ionic doesn’t register a Service Worker and has a linked Manifest that, according to Lighthouse, doesn’t work correctly. The audit reports that although Ionic’s web app uses HTTP/2 for all of its own resources, it has huge network payloads and a low loading speed: the fully interactive paint appears only after 14 seconds.

Angular
PWA: 100 | Performance: 69 | Accessibility: 89 | Best Practices: 88
A meaningful page loads in: 3 seconds

Angular got a perfect 100% score for their web app. It successfully passed all 11 audits that confirmed the presence of a Service Worker and a Manifest, as well as approved the quality of HTTP/2 data transfer. Since Angular’s Progressive Web App becomes fully interactive after mere 5 seconds, it’s estimated to be quite fast on 3G networks too.

Takeaway:
The 20-point-difference in the Performance scores of the two PWAs can be explained by Ionic’s huge network payload. Along with recommending Ionic to fix it, Lighthouse also suggests storing images in WebP format to speed up Ionic’s interactive paint by 6.5 seconds.

Travel and lodging:
Airbnb
PWA: 55 | Performance: 7 | Accessibility: 97 | Best Practices: 81
A meaningful page loads in: 6 – 10 seconds

Airbnb’s web app has an improperly tuned Manifest and no registered Service Worker. Yet, not being installable or available offline isn’t the main problem of this PWA: Airbnb needs almost 10 seconds to show a meaningful paint and 20 seconds to become fully interactive. This is long enough for a user to give up and try another lodging company.

Trivago
PWA: 82 | Performance: 53 | Accessibility: 86 | Best Practices: 69
A meaningful page loads in: 2 – 3 sec

Google Lighthouse reports a Manifest and HTTP/2 connection all in use and working properly. Service Worker is also registered but not set to offer installation (hence 82 points on progressiveness.)

Trivago’s performance is quite high and could be even higher if not for the time this PWA needs to build an interactive version. But although 8-10 seconds aren’t sufficiently fast for a 3G network, it’s not a bad result considering that a user sees a first meaningful paint after 2 – 3 seconds.

Takeaway:
Airbnb’s obvious performance issues are caused by a huge network payload and excessive DOM size, which aren’t easy to fix. Trivago PWA successfully avoids these issues, but has minor problems with render-blocking stylesheets and server response time.

Conclusion
Most importantly, it’s now clear that the road to progressiveness isn’t a straight one. Not all web apps are progressive to the same extent: while all eight PWAs that we’ve audited opted for HTTP/2 connection, only half of them had a registered Service Worker or a linked Manifest.

Yet, simply setting up HTTP/2 connection doesn’t necessarily guarantee high loading speed. Combining HTTP/2 with caching, enabled by Service Workers, can bring better results. What’s more, it’s still important to think about optimizing visual and textual content as well as to watch out for network payloads and excessive DOM size use.

Ultimately, if the app is progressive, it has better chances of having high performance, but being progressive alone isn’t a solution. One shouldn’t rely on progressive features without being able to create a well-performing regular web app. Using these features to enhance a properly developed web app is the key to creating a truly successful PWA.