How to Test Your Website on Multiple Browsers: Four Solutions Compared

[article]
Summary:

Robbie Bridgewater writes on the difficulty in finding bugs during testing since no single computer can run all of the major browsers—not to mention the added challenge of testing various mobile operating systems. In this article, Robbie compares four possible solutions to this dilemma.

Differences in how your site looks between browsers can be one of the most frustrating bugs to resolve, because it’s so hard to notice the bugs in the first place. No single computer can run all of the major browsers—due to Microsoft’s Internet Explorer’s not existing for Mac and Safari’s no longer existing for Windows—not to mention the added challenge of testing various mobile operating systems. To tackle this issue, multiple solutions have appeared in the marketplace. I compare four of the most useful here.

Lunascape Browser 

This is an interesting idea: a web browser that can change its engine on the fly (Trident for Internet Explorer, Gecko for Firefox, and WebKit for Safari and Chrome). For regular use, Lunascape seems to lack some of the polish that other modern web browsers possess, but for a quick test of each engine, it can be great. However, this is an imperfect solution, because clearly not all bugs are engine related (as anyone who has seen a bug show up only in one version of a browser but not a newer version of the same browser can attest). To change which browser engine you are using, you just click the icon at the bottom left of the browser.

Figure 1

Browsera.com 

Browsera is an online solution that examines your website page by page, presenting you with snapshots of different browsers side by side, as well as attempting to actually detect variations and pinpoint where they are. The free version only allows a comparison across two versions of Firefox and one version of Internet Explorer, while the paid version contains all popular web browsers, with both past and current versions available.

Figure 2

Browsera’s error detection does not always work perfectly. On some sites I tested it against, it would simply state that only a certain percentage of content matched, while on others, it would throw up false errors anywhere that an element might be random (such as where an ad might load). While the error detection may give false positives, it does not seem to give any false negatives, meaning that it shows large areas of your site that you can safely ignore with respect to browser differences.

Browsera also shows thumbnails of the pages side by side, which can be expanded if you are using the paid version of the site. The value for this tool definitely lies in the paid service, which allows you to view your website with all popular web browsers, both past and present. You can examine each page closely, switching between browser views by expanding thumbnails in the paid version of the website.

Browserstack 

Browserstack uses virtualization on the cloud to give you access to a testing platform. It offers any possible combination of OS and browser version to be virtualized, including mobile platforms (the mobile browsers are run on desktop emulators normally used by developers, such as iOS simulator for Mac). This solution works for almost any situation, but can be quite frustrating to use, as even scrolling on Browserstack creates lag. Compared to the previous solutions, though, Browserstack has the advantage of truly showing you what a computer examining your website sees, and compared to the following solution, you do not have to worry about operating system licenses.

Figure 3

The program runs inside of your browser, and you can switch which browser you will use and reload the page instantly.

Local Virtualization 

If you have a computer powerful enough to handle virtualization and the licenses necessary for it, I would definitely recommend this as a solution. This provides the same benefits as Browserstack does, but without the slowdown of remote access, it is much snappier. You can use the same techniques that Browserstack does to test mobile sites as well.

If you have a Mac, a free download of xCode includes the iOS Simulator, and an Android development kit is free on any OS. VirtualBox, Parallels, and VMWare are all good virtualization tools, and with them you can make your one computer switch between whatever operating system and browser that you need. One potential difficulty for Windows users is that it is more difficult to virtualize a Mac in Windows than it is to virtualize windows on a Mac. If you wish to test iOS devices, then you need to either begin with a Mac or run the iOS simulator inside of a virtualized Mac.

Summarily, the Lunascape browser is the fastest tool for testing a website, essentially allowing a quick test of IE, Firefox, and Chrome in quick succession, but it is also the least thorough, having only the underlying engines as similarities. The other options are all more thorough, but each has its own difficulties. Browsera essentially gives you some automated error detection and allows you to examine a web page across each browser side by side by expanding the thumbnails, but it does not allow you to interact with the web pages. Browserstack is a powerful solution, but the lag inherent in its cloud-based solution may frustrate a user. Finally, local virtualization can easily be considered the ideal solution, but also the most expensive and difficult to initially set up, requiring licenses for each OS you wish to use as well as a fairly powerful computer.

 

 Works for Mobile TestingHas a free versionHas a paid premium versionWorks with MacWorks with Windows
LunascapeXXX
BrowseraX
BrowserStackX
Local Virtualization*

* VirtualBox is a free option, but you still need access to licence keys for the OS you will emulate to do so legally.

 

User Comments

2 comments
Mike Feeney's picture
Mike Feeney

Very timely article as I am looking at different tools for cross-browser testing. I was playing a little with IE Tester (IE only) and the noticed it didn't have many favorable reviews. Also been taking a test drive of BrowserStack. I have already used up the free 30 minutes and will need more time on that to see if it is worth it.

June 28, 2013 - 9:09am
Kevin Dunne's picture

Robbie, 

 

Great article - we see a number of our customers using BrowserStack in particular, but I am excited to explore these other options as well. 

 

One thing I wonder is - how do you guys manage the results of this testing?  Are you just setting up a test case for each page tested under each browser and then passing them for each page that loads correctly?

In the interest of full transparency, I work at a company that provides a test case management solution.  I am looking to better understand if testers doing compatibility testing are looking to track these results over time and see trends in what particular pages looked like from build to build and browser to browser, or if these tools are mostly leveraged as more of a "sanity check" and the results are only documented in the case of failure. 

Looking forward to reading more of your stuff in the future!

Kevin

August 20, 2014 - 12:32pm

About the author

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.