Recently, I’ve seen a number of businesses touting exceptional web page performance metrics based on the Lighthouse tool. And while it’s great to see so many organizations recognizing the importance of loading speed, I believe their methodology is misunderstood. As a result, these businesses are not getting the full picture when it comes to the user experience.
But to make a change and ensure that we all have the correct data, we need to understand the tool. With that in mind, I’d like to walk you through the Lighthouse tool, how different organizations misuse it, and other options that might provide a more realistic outlook on website performance.
Let’s start by actually defining the tool in question. Lighthouse is a free, open-source tool provided by Google. A business can use it in Chrome DevTools to analyze any page, whether it’s public or requires authorization. To do so, Lighthouse runs a series of audits, after which it provides a report of how the page performed. In addition, the tool then provides a reference document summarizing each failed trial. This report details the issue, why it’s important, and how it can be addressed.
During these audits, Lighthouse looks at the page’s performance, accessibility, web development best practices, search engine optimization (SEO), and modern web capabilities. In doing so, the tool covers a range of performance metrics, reviews the user experience, and measures visibility.
All of that sounds great, right? A free tool that covers a range of factors that contribute to web page performance seems like a boon. So, why the need for this article? It seems like a slam dunk!
Well, as much as I’d like to say that you now have the perfect web performance auditing tool for free, there’s more to it. We need to discuss how Lighthouse is being used and why its results can be deceptive.
While running a deep dive into web performance is great, Lighthouse can’t simultaneously test a page on a variety of devices. When the audits are run, they only target the specific device being used. As such, if you’re using a high-quality desktop or laptop, you’ll only see how the target page performs on that device. As such, unless you specifically run Lighthouse on different devices, the results can end up skewed.
After all, there’s no website out there that’s only going to receive traffic from one type of device. Let’s say that you use Lighthouse on a computer and receive great results. While that’s obviously great, you’re getting an incomplete picture at best.
What about all the users who navigate to your site on their phones or tablets? If your website loads slowly on mobile devices, Lighthouse won’t catch it without multiple uses of the tool. As a result, you might come away from a Lighthouse review thinking your website is in great shape. Meanwhile, in reality it could be struggling to provide a satisfactory experience for a large portion of your audience.
While Lighthouse absolutely does have a limited ability to simultaneously test website performance across devices, this isn’t the biggest problem. At the end of the day, I’d argue that the key issue with Lighthouse isn’t with the tool itself but the businesses using it.
As I mentioned earlier, the core premise of this free service is that it runs multiple audits. While this is very useful for ensuring that you’ve thoroughly tested a page, it also provides a wide range of results. Now, this doesn’t sound like a major issue. After all, the whole point is to make sure you’ve checked for every potential performance risk!
However, businesses can choose to focus on the best audit scores while neglecting to share the negative results. This can make it look like a website loads quickly and offers an industry-leading user experience without sharing the full spectrum of results.
More importantly, Lighthouse can be easily manipulated to produce certain results. As I mentioned earlier, the tool only analyzes performance on the device you use or another simulated device, which is obviously chosen by the organization running the test. If a user wants to, they can select a high-performance computer that has a strong and stable internet connection as the focus for Lighthouse. This will likely produce exceptional results without providing much insight into the actual user experience.
Keep in mind that it’s much harder to achieve elite scores through mobile devices as opposed to a desktop. But with so many of your potential users connecting via those devices, reviewing connections via those devices likely represents a more realistic view of the customer journey and will enlighten you regarding the issues that need to be addressed.
Lighthouse will provide results that are equal in quality to the effort put in. If a company is thorough and runs multiple tests on different devices, it can be a great tool. But too often, I see it being manipulated and producing misleading results.
Now that we’ve reviewed Lighthouse and identified its limitations, we’re left with an obvious question: what other tools or approaches operate similarly but provide a more comprehensive summary of website performance?
Fortunately, we don’t have to go far afield to find an effective alternative. Google offers one of the most widely-used page performance analytic tools out there: PageSpeed Insights. Let’s cover how it works, compare it to Lighthouse, and where it comes up short.
PageSpeed Insights provides a variety of information collected from different sources in order to thoroughly analyze the user experience.
Similar to Lighthouse, PageSpeed Insights provides a laboratory environment in which to test your website. However, this solution also reviews field data to more accurately report on how well a site functions. Powered by the Chrome User Experience Report (CrUX) dataset, field data examines actual visitor data. As a result, when using PageSpeed Insights, you can isolate issues that are negatively impacting performance while also understanding how to optimize the user experience.
To accomplish this, PageSpeed Insights offers two different test options. First, it performs a Lighthouse test on an external computer or Google server. While this enables you to examine device-based web page performance, you can’t determine the exact conditions in this case. And considering that you can’t change the examined device the results are volatile.
In addition, the tool can perform a real user monitoring (RUM) analysis based on real user data drawn from Chrome. And while this uses actual information drawn from your visitors, it doesn’t provide an option to perform a deep dive into that data and to debug.
With a more realistic assessment of factors like Core Web Vitals, PageSpeed Insights enables changes that directly address visitor interests. That means that you’ll gain insights into how your site loads on different devices and where you can improve.
So, problem solved, right? Use PageSpeed Insights rather than Lighthouse.
Not so fast. While PageSpeed Insights certainly marks a substantial improvement over its fellow Google tool, every solution has weaknesses. In this case, the primary limitation comes in the form of timing. While PageSpeed Insights collects information from users in order to suggest optimization opportunities, it does so within 28-day collection periods. As such, if your website must be updated regularly, the solution can struggle to keep pace with the rate of those changes.
Businesses who rely completely on PageSpeed Insights will either be forced to wait for critical data or make updates blindly. Imagine adding a high-resolution video to a page and not learning that it has slowed down loading for several weeks! How many users might navigate away during this time while facing an intolerable wait?
Moreover, like Lighthouse, PageSpeed Insights does not allow for the simulation of other devices or locations. Instead, this tool relies on a predefined setup and doesn’t offer the option to make a change, which Lighthouse actually does. In addition, PageSpeed Insights exclusively analyzes data from Chrome, meaning that you won’t have any insight into how your website performs on other browsers. So, while optimization becomes more effective while relying on real user data, gaps will likely still remain without thorough repetition.
Alright, so Lighthouse is misunderstood and PageSpeed Insights can result in delays. Both limit your ability to understand performance across multiple devices. What exactly are we supposed to do then?
My personal recommendation is real user monitoring (RUM), a strategy through which an organization tracks and measures website performance. More specifically, the technique collects data from users when they interact with your website. This information then summarizes how your site is performing out in the wild.
Data gathered from your visitors forms the basis of your understanding of the true user experience. And while PageSpeed Insights also draws upon user data, there are key benefits that separate RUM from the pack.
As I mentioned earlier, Google’s tool performs analyses in 28-day windows. One primary point of RUM’s value is its ability to gather and analyze data in real time. Rather than waiting weeks to understand the exact impact of change to your site, you can run a new test after each update. In addition, you can utilize data from a variety of browsers in order to cover the full gamut of user pathways to your site. Moreover, while PageSpeed Insight places its emphasis on optimizing website performance, RUM emphasizes measurement and tracking. So, whereas the former’s primary output is suggestions for improvement, the latter demonstrates the actual metrics as it relates to the user experience.
In short, with RUM, you gain real-time insight into the performance of your website as it pertains to actual visitors. This leaves little room for manipulation, exaggeration, or misunderstanding. RUM delivers rapid analysis as quickly as in a few minutes or hours and describes everything you need to know about how your audience views your digital presence.
Few aspects impact the success of online engagement as much as the user experience. So, you need to make sure that your business can obtain an accurate and honest assessment of its website whenever needed. There are many options and different tools out there that can be used to analyze page speed and performance. Each of Lighthouse, PageSpeed Insights, and RUM can provide value when used correctly.
I hope that now you have a better understanding of their uses, limitations, and defining features. If you have any more questions, I’m always happy to discuss this at greater length! Reach out and let’s chat!