In this interview, IBM's Dennis Schultz discusses the emergence of mobile applications in the business world, how some companies tend to rush their products to market before they're ready, and the challenges that go along with testing mobile apps compared to those found on the web.
Josiah Renaudin: Okay. Today I'm joined by Dennis Schultz of IBM. Dennis, thank you very much for joining me here today.
Dennis Schultz: Hey, Josiah. No problem.
Josiah Renaudin: All right. First off, if you could, just tell us a bit about yourself and your experience in mobile testing.
Dennis Schultz: Okay. Gee. I guess my entire career I've been in testing of one form or another, maybe about ten years in avionics testing, and then moved into rational software. Then with IBM it opened a lot of other doors. With the explosion of mobile over the last several years, that's just become a natural transition of quality assessment and the quality assurance area for me.
Josiah Renaudin: Your talk is almost all about mobile and its explosion. What do you think has made mobile apps so integral to the world of business these days?
Dennis Schultz: I think it's really the whole "always on," "always there" thing. For me, and it seems like most people, the phone is never very far away. It's in my hand; it's in my pocket; it's on the table next to me. It's connected. We're connected to our devices, and they've become almost personal. You see the kids in the mall with their custom cases and things; but, really, as adults, we're somewhat the same way. We've got the apps that we like that we use. We've got our desktop arranged the way we want it on the device. So it's become a very personal method for businesses to engage with their customers with their business. I think that that's one of the big things.
When you think about it, it's very different than a desktop application or a web interface, where everybody sees the same thing. In a mobile environment, the companies that are really doing it right, I think, are giving a very personal experience to their users.
Josiah Renaudin: Like you said, we carry mobile phones all the time. I have mine right next to my laptop right now. Really, it seems like we always want new things. We always want new apps to try on our phones. Are companies rushing some of their mobile apps to market in order to stay relevant in this high pressure, fast-moving industry?
Dennis Schultz: That's a good one. I think that does happen. I think there are some. It's a very competitive business. I read something not long ago that the mobile application that a bank delivers is becoming one of the distinguishing characteristics of why a customer chooses that bank. So banks are really differentiating themselves by the applications they provide. There is a lot of pressure to be the first with the new features and the new capabilities in your industry.
Banking, maybe, isn't the best example, because that's a very regulated and controlled industry, where we can't do things wrong, at least not from a business standpoint; but I think there are a lot of apps that go out there that maybe are not really ready for prime time, just because they want to be first. Yeah. I think that's legitimate.
Josiah Renaudin: Staying on that topic ... With the advent of rapid iteration of updates in mobile applications, do software engineers have more room to actually make mistakes without being panned by consumers, mainly because you can fix problems so quickly right now? There's so many updates. I was speaking to someone with Facebook recently who talked about how they have 3 alphas going to their team ... 3 to 5 every single week ... and every month they release a new program to their consumers. Do you think there's actually more wiggle room because it can be updated so quickly?
Dennis Schultz: I think it's a double-edged sword. I think what you've said there is true, that if we have to fix a problem, if we have to patch something, we can do it in the next iteration, which may be in another hour or so, where with traditional software development, you have to wait for the next release train, which may be 6 months or a year away. So what you've said is true; but, on the other hand, the feedback mechanism from the users is so much reduced and so visible. If I rush a mobile app out there too quickly, and it's not up-to-snuff, I'm going to hear about it in my reviews very quickly, and the world's going to hear about it, more importantly. Those reviews are very public. I think you're right that we have the ability to react to problems more quickly, but those problems can also surface more quickly.
That's one of the big challenges I think we have today in the mobile industry is recognizing trends, recognizing problems that are arising out in our reviews and the sentiment from our users, and reacting to that quickly, not letting those problems fester and become a real problem.
Josiah Renaudin: What are some of the advantages of using real, active devices when doing mobile software testing? Are enough companies testing in this manner in order to prevent bugs and to prevent botched software that you were saying you might hear about?
Dennis Schultz: Yeah. It is a bigger challenge than I think we've ever faced in desktop, client/server, web-based applications ... the proliferation of different platforms, form factors, all of that. I can only do so much with emulators and simulators. I think the ideal ... yes, definitely. Clients should be using real devices to do their testing.
Of course, there's limits to that, because there are hundreds and thousands of real devices out there in use. We simply can't test on all of those. So we have to strike a balance somewhere. What I'm seeing a lot of our clients doing is they will test on the most popular, the most widely-used physical devices. Then maybe some of the corner cases, the lesser-used devices and form factors, they will test through simulation or emulation. Another option that's becoming more prevalent these days is things like device clouds. There are vendors out there that provide physical devices, but they're in a rack somewhere in the Cloud that you remotely access and can do your testing.
I think, in summary, yeah. Physical device testing is just absolutely essential, but it's also something that can be extremely costly if I have to buy all of those devices and maintain them. They're really the only way that I'm going to get an assurance that I'm running in a real world platform with issues like signal dropping, low battery, all of those things that are much more difficult to emulate in a software environment.
Josiah Renaudin: You talked about doing the most popular device and testing on that. When it comes to iPhones, there's a new one, it feels like, every year at this point. There's new devices all the time, just always different iterations, different updates. What kind of challenges does that pose when you're trying to test on real devices? You're trying to knock out all the bugs, but, maybe halfway through the development of an app, there's a new phone right there. How do testers deal with that?
Dennis Schultz: Right. It's a difficult challenge. I just had an email, I think, yesterday or the day before, that there's another beta available for iOS 8. If you're a company that's really trying to get your app out there on day one when that next version of the OS comes out, that's tough to stay on top of that. You're working from beta software that's maybe not complete from the vendor. That's only Apple. You've got Android. You've got other platforms that people potentially are trying to support as well. It's a very big challenge.
That's one of the reasons that I think the traditional center of excellence, a large, very well-planned-out, thought-out testing organization is not as relevant in the mobile world any more. We need to not only be able to develop very lean and very quickly; we need to be able to test the same way, so more testing is done by the developers or in teams very closely linked to the developers. We don't have time to develop all the software, make it nice and pretty, and then throw it over the wall to a testing team that will take a couple of months to get it tested. That's just not a model that's going to scale in the mobile environment.
Josiah Renaudin: All right. I don't want to give away all your STARWEST secrets. I am very much looking forward to that talk. What do you see as the biggest change we've seen in testing that was spurred by the explosion of mobile development?
Dennis Schultz: Hmm. There are a number of things. You're right. We'll probably touch on a number of those in the talk at STARWEST. What I think is one of the most impactful things, and really to me one of the most encouraging things, is the focus on the user, that if the user's not happy, nobody's happy. If we're developing an operating system, or something, for a desktop computer, that's a long process and we put together our best guesses and things, and the user just gets used to that. With mobile, that's not the case. The user can be very opinionated, and we need to look at those opinions very closely and react to them very quickly. It's a glass half full, half empty thing. It's pushing us and challenging us even more for software testing for mobile, but it's also improving our relationship with our users and our end customers. Ultimately, I think it's improving the user experience with the software.
Josiah Renaudin : All right. Thank you very much for your time, Dennis. It was really great talking to you. I am very much looking forward to your talk in just a couple of months.
Dennis Schultz : Great. Great. Thanks a lot, Josiah.
