In this interview, Genefa Murphy talks about how to test a mobile app in a real-world environment, how much of your testing should be automated in a given project, and the differences between hybrid and native apps in the world of testing.
Josiah Renaudin: Today I am joined by Genefa Murphy, the VP of product and partner marketing at HP Software. We'll be discussing the key considerations for mobile testing today. Genefa, thank you very much for joining us.
Genefa Murphy: Excellent. Glad to be here.
Josiah Renaudin: That's great. First, could you just tell us a bit about your experience in the industry?
Genefa Murphy: Definitely. I've been in the software industry probably for about the past ten years now. Worked on various different activities, everything from doing traditional IT, like SAP upgrades, all the way through to now focusing on things like mobile, Internet of Things, and also looking at the full application lifecycle management. I've covered everything from strategy to product management to product marketing, and it's definitely one of those interesting, exciting industries that you can just keep on evolving every time. There's always something that can surprise you with software.
Josiah Renaudin: Let's start with a broader question. How can extensively testing mobile apps increase revenue, customer loyalty, and brand image?
Genefa Murphy: Definitely. I think the important thing to think about here is we've seen a resurgence in the past few years around the value of user experience and what credibility people put around user experience. Some examples of this, if you look at the examples that people always quote. People like, companies like Uber. Uber has a ... is expected to have an annual revenue run rate of $10 billion by the end of 2015, according to Business Insider. That's a huge number.
How do you access Uber? You access it from your mobile phone. There are so many companies out there. Uber's just one example of them, where their primary revenue stream, their primary interaction point with the customer, is via a mobile application. I think the reason why is because we live in a society now where everybody's on the go, everybody's moving. The mobile device, whether that be a tablet or a smartphone, that's the primary way often in which people often interact with the vendors that provide them services.
Josiah Renaudin: You mentioned Uber a little bit ago. No one could have imagined how big Uber got and be able to test that load, that pressure, with all the people accessing that app. How can you test a mobile app's real-world performance and not just how it reacts in a more controlled environment?
Genefa Murphy: Definitely. There are a few different pieces to it. Of course, with mobile, you have to test the three core pillars that we talk about. You have to test the functionality, you have to test the performance, and you have to test the security of the application. The other element that mobile brings into the piece is network. When we're working on Wi-Fi, 3G, 4G environments, networks introduce a lot of different variables that often people don't account for—things like jitter, or noise on the network.
One of the key things that I've definitely seen with customers, if you want to be successful in testing the real app's performance, it's not just looking at the core fundamentals, but also looking at network conditions and the context of how that user is going to use your mobile application. We're not always going to be connected to a super-fast 4G environment. There's going to be times where we're on the go, and maybe we're moving between different types of connectivity. Being able to simulate or virtualize those network conditions in the testing side of the house, before you go to production, that's the real key to making sure you're testing the full context of the app.
Josiah Renaudin: In virtualization and testing in real-world environments, that's an issue that has to be dealt with in a lot of different platforms. What are some of the unique challenges brought about just through mobile?
Genefa Murphy: Definitely. I'll hit the unique challenges. First of all, when it comes to actually replicating that environment, it's definitely one of the challenges. Being able to ... You don't want to have wait until you're in production and use your customers as guinea pigs. The other thing that you need to account for when you're looking at mobile, things like being able to deal with common interruptions and functionality. What happens to the application if I'm using an app and I get a phone call or I get an SMS or I get an email or an alert. What's going to happen? Is my app going to crash? If I'm working on data, is that data going to be lost? You also need to look at things like ... You've got the interrupts, you've got the networks, you've also got things like the gyro, or the GPS, or the interaction with the camera. There are specific functionalities which only come with putting an app on a mobile device that you need to be able to test for.
Josiah Renaudin: Let's talk a bit about test automation. What is your ideal mix of manual and automatic testing in a given project? That can be a little bit of a difficult question to answer because from product to project, you always hear that there's a different balance between how much you want to do manually and how much is automated. Do you have a basic percentage in your mind of maybe 60 percent automated, 40 percent manual, or is yours much different than that?
Genefa Murphy: Yeah, that would definitely be the ideal state. I think it would be great for customers to be able to have in a project, 60 percent or more automation. I think there's a few different variables to it. The first thing that I will say, is whilst that's the ideal state, the reality is much different. We did a survey with, in partnership with one of our large partners, and we surveyed a number of customers, and a majority of them, at the moment, only had 28 percent of their test cases automated. These were customers who were dealing with the challenges of mobile and big data apps, etc. The reality is often different from what the ideal would be.
In a world way, you've got 60 percent automation as the ideal state. I think there's a few things you need to consider. Where to automate first. For example, automating regression tests first. Also, trying to automate as much as possible using things like field verification testing so that you can integrate your automation, test automation process, into your build automation process, and what that allows and where we see that practice being used more is in customers who are using more agile or DevOps techniques in order to deliver the mobile apps faster. That's really one of the key and the expectations around mobile applications is that when a change is needed, that change is delivered almost instantly, versus having to wait three months, six months, one year, which were the release cycles of the past.
Josiah Renaudin: We've talked a lot about being able to effectively load test a mobile device, a mobile app. How difficult has it become to correctly security test an application?
Genefa Murphy: Security, I think one of the biggest issues with security is that security's still treated in some respects as an afterthought, although there's definitely been more public awareness of security because of the various different hacks that we've seen of the large dot coms, or even the traditional bricks and mortar type organization. Security's still left to an afterthought, and it's something that is struggling to actually bring it into the release cycle versus it's being something that is either done at the end of the life cycle. As we know, you often don't have time to do what you leave for the end. Or it being dealt with a separate company. I think the key is to make sure that one, you're aware of what are the top security vulnerabilities. Making sure that you prioritize effectively. Also, that you bring in that security expertise into the test cycle, versus leaving it to the end.
Josiah Renaudin: Absolutely. We've talked about a lot of different areas of mobile testing today, but to close, let's discuss native versus hybrid mobile apps. Which do you find to be the most pervasive in the industry, and how different are they to test?
Genefa Murphy: Sure. I think it really depends. I think in the rush that customers—most customers that we've spoken to—had in order to get that mobile app out there, because it is a de facto standard. I know when I, I'm going to use—whether it's staying in a hotel, whether it's an airline, or whether it's walking into a retail environment—one of the first things I do is I check to see, does that organization, does that vendor have an app? If they don't have a mobile app, honestly, I ... They go a little bit down, in my opinion, because mobile is one of the means that customers expect right now.
Because there was that rush to get the mobile application out there, a lot of customers, they started to build native, because that's what they thought was the best approach to do. What we started to see change over time is because instead of building native, they didn't have the skills. They thought that native was the best thing to do, but they didn't have the skills in house, so they often outsourced it to third parties who would build the app for them. Now because they want it to be, they want to have economies of scale, they want to be able to iterate on that app effectively, they're now starting to bring it in house. The skills that they have in house tend to be more hybrid based. We actually see a lot more customers starting to develop hybrid-based applications. They're leveraging HTML5, JavaScript.
There are a small percentage of customers who started to adopt the mobile application development platform, and there are many out there, but I would still say that it's still primarily around native and hybrid. Some of the differences, obviously, in testing those applications, come down to the fundamentals of how well can you interrogate the application in order to identify the key pieces of functionality in order to test. The main differences come from the technical point of view in terms of the scripts you create to actually test those applications, when you've got a native versus a hybrid application.
Josiah Renaudin: All right. Well, thank you very much, Genefa. I appreciate your stopping by and talking with us today about the key considerations of mobile testing, and I'm looking forward to hearing more from you in the future.
Genefa Murphy: Excellent. Thanks for your time. Thanks, Josiah.
Genefa is the VP of product and partner marketing for the Application Delivery Management business unit within HP Software. As part of this role, Genefa leads a team of product marketers, business development leaders, and technical enablement specialists who are responsible for creating market awareness for HP Software’s Application Delivery Management offerings including HP Quality Center, ALM, LoadRunner, Performance Center, Service, and Network Virtualization and our next-gen agile, mobile and DevOps solutions. Current responsibilities include marketing strategy encompassing demand gen, content creation, digital presence, pipeline generation, social, and channel alignment.
