In this interview, Anders Wallgren talks about the Internet of Things and how it played a role in the recent Honda recall. Anders also covers how future technology can affect our quality of life, as well as touching on some best practices for continuous delivery.
In this interview, Anders Wallgren talks about the Internet of Things and how it played a role in the recent Honda recall. Anders also covers how future technology can effect our quality of life, as well as touching on some best practices for continuous delivery.
Cameron Philipp-Edmonds: Today we have an interview with Anders Wallgren. Thank you so much for joining us today, Anders.
Anders Wallgren: Great to be here.
Cameron: To start things off, could you tell us a little about yourself?
Anders: Sure. I've been in the software industry for about twenty-five years now, working in all kinds of different areas—multimedia, text retrieval search—but I've spent about the last ten years of my time at Electric Cloud, working on helping our customers do continuous delivery and agile processes.
Cameron: Can you tell us about your role at Electric Cloud?
Anders: My role at Electric Cloud is a CTO. I kind of have a dual role there, because I do quite a bit of hands-on implementation work, but that sort of comes and goes. I also work a lot with our sales organization, marketing organization, and with customers as well. Handling sales, architecture, support incidents, and those kinds of things. I like my job a lot because I get to do a lot of very different things which is very cool.
Cameron: Now, you're going to talk to us today about the Internet of Things, and how it kind of relates to the recent Honda recall. To kind of start things off can you briefly explain what the Internet of Things is, and how it relates to the Honda recall?
Anders: Yeah. The Internet of Things very simply is just everything in the whole world is going to have software in it pretty soon. Sort of related to that, is every company is becoming a software company. So of course, when you're driving around in your car, unless your car still has a carburetor, there's a lot of software in a car. A new car has on the order of two- to three-hundred million lines of code.
Cameron: Wow.
Anders: A lot of that is engine management systems, transmission systems, but the bulk of it these days I think is starting to be in the entertainment systems. You practically have a digital entertainment system in every single car. It's pervasive and a modern car, I think I'm going to get the numbers wrong, but I think the latest Mercedes has over fifty or sixty different processors in the vehicle. This is a travelling network these days, it's not just a car.
The Internet of Things… look at all the computers that we carry around with us, laptops, tablets, phones, our televisions are now computers, they're not just glass tubes anymore. Our thermostats are computers, our smoke detectors are computers, so it's really down to the point where everything is going to be on the internet, everything is going to be connected. The ice bucket in my hotel room is probably connected to the internet somehow.
Cameron: Right, it measures temperature with the Wi-Fi?
Anders: Yeah. (laughing)
Cameron: How does that relate to the Honda recall?
Anders: The Honda recall specifically—and I don't want to throw Honda under the bus, obviously—but the unintended acceleration is due to a software bug. When you had a recall in the past for a car, it used to be that they had to go in and replace some physical part. Now what they're doing is bringing the cars in and doing a software upgrade to fix that bug, and that's very common. Toyota had a recall very recently with the same problem with the Priuses, where, a similar problem I should say, it was related to acceleration, but it was also fixed with a software update.
The newer car companies like Tesla for example are a little bit more sophisticated on their approach because their cars are internet connected, so they just download the firmware updates over the air at night at 2 a.m. is what I'm told, and hopefully people aren't driving the car at that time.
Cameron: Right.
Anders: So in order to do software upgrades Tesla doesn't need to bring the car in, they just do it over the air, and that will be the norm in the future obviously. The expense of having to bring a car into the service establishment and get the software upgraded like that is pretty expensive. The service establishments like it because they get a chance to sell oil changes and all kinds of other stuff, but the consumer has to waste an hour of two to wait for that to happen. Quite frankly for Honda itself to have to pay for that is a very expensive way to deliver software.
Cameron: While it's unfortunate that these cars are being recalled, it also kind of serves as a great example for us to take a step back and see how software is driving the quality of goods in our everyday lives. With that being said, and you kind of mentioned it earlier, how everything is connected now, including your ice bucket… is this an example of trying to over complicate something that was relatively simple at one time?
Anders: That's probably a little bit of a cynical thing to think, and I'm such a gadget freak that personally my answer to that would be a resounding “no.” I think a lot of these things in the end are quality of life. If you've got a GPS system in your car, try driving without it, you'll be completely lost. So do we become dependent on these things? Absolutely we do.
That's to our detriment in some ways I'm sure, but it sure is nice to be able to get to a place where I've never been before very reliably, and to be able to listen to the music I just downloaded at home five minutes before I left, and all of those kinds of things. I guess you could in some ways classify this as a first-world problem if we're concerned about that.
Having said that, do I want my refrigerator to be Internet connected? I don't think I found a killer application for having my refrigerator being connected to the Internet yet. I'm sure somebody will come up with it, but it sure is handy to have Bluetooth in my car and be able to be hands-free without having to do any intervention and all the other cool things that are in cars these days.
It's progress, it's inevitable, and I think we do benefit from it. There's always going to be room for doing it the purist's way of no software and just use a carburetor, and none of this fancy fuel injection stuff, and I think there's absolutely always a place for that. But I think this is where the world is headed—it's going to be electric cars and they're going to be run by computers. That's where we're headed.
Cameron: So it's definitely a necessary growing pain?
Anders: Absolutely, and I think this is just starting now to kind of hit at a consumer level. Where before, it would have been gear-heads and gadget-heads that would have run into this. Whereas now, we've all got cell phones, so when there's a bug in iPhone's SSL implementation, that's a big deal because there are so many of them out there, and all of a sudden these kinds of things hit consumers and end up in consumer news stories and on the news.
I think it's something that's always been there, software has bugs, always will. There is no way to make sure there are zero bugs in every piece of software, that's more complicated than a few lines of code. It's really just something that's always been there, but now has a lot more visibility because these things are in our lives, from the moment you wake up to the moment you go to sleep, you're living with computers. They're so much more pervasive now that there's no way to not notice these types of things.
Cameron: To expand upon that, there's such a high propensity for pretty much everything you do an everything you touch now, to be connected to the internet or have some sort of technological advancement in it. With that being said, I'm going to ask you to put your Nostradamus hat on for a little bit here and kind of predict into the future, what heights, in terms of quality, do you think the future of software will bring to everyday lives?
Anders: I think we're getting better at doing this, and in some ways it's probably a little controversial to say it - but I think software is where manufacturing was about fifty years ago. We've had our Henry Ford, he's come along, and we've learned a lot from that, but now it's the Hondas, the Toyotas, and their manufacturing prowess that we're trying to go after. Which is kind of ironic, given that the start of this was Honda's software recall.
But they figured out a lot of things in their manufacturing, and manufacturing in general; do small batches, don't have work in progress inventory on the factory floor because that's taking up space and costing you money, you have to track it and spend all kinds of expensive time doing that. Well that's very similar. We used to have software where you would check it in, and a week later it would get tested. Nowadays, most software companies are running things like continuous integration, so your software's going to be built and tested minutes after you checked it in.
So just like Honda would never install a radio in a car and then park the car in the factory for a week before they test it to make sure the radio worked, we're no longer doing those things with software either. I think we have a lot further to go until we have that same kind of mentality that physical goods manufacturers have had for many years, and have had to have for many years. But I think we're getting better at it, and the agile processes have helped a lot, and really focusing on this notion, especially with continuous delivery, we have to always be ready to ship our software. If we're not ready to ship our software today, everyday, then a red light should go off and a horn should sound.
That means we're not paying attention to the quality of our software, we think we can do waterfall methodology, where we write a bunch of software and then we "test it." That's where all the problems happen generally, and if you look at the software recall by Honda, I bet they're going to do some kind of root cause analysis, and figure out, "OK, where did we miss that, and did we miss it because we skipped a step in our process, or because we're missing a step in our process? Is there testing we're not doing that we should be doing, is there quality assurance programs that instantiate that we don't have today, or did we just forget to do something?"
I'm a lot happier today with the quality of software five or ten years ago, and I think five and ten years for now we'll be even better at it. Ten to fifteen years ago, it was controversial to write unit tests for everything, or to do test-driven development, or extreme programming, those kinds of things. Nowadays, those types of techniques and processes are very mainstream, and I think that's a good thing, but we could get better at it for sure.
Cameron: All right. You didn't mention how to take a step back and do kind of an analysis of what happened. Do you think the problem could have been averted, and if so, how?
Anders: I think by definition, yes the problem could have been averted, it's a software bug and you can test it. I guess they'll never come out and kind of say "here's what we fixed in our process" or "here's how we could have caught it and how we'll catch it next time," and those kinds of things. It kind of starts fundamentally with having good testing methodology and understanding that good software quality is everybody's problem, it's not just the QA guy's problem or the release manager's problem.
It starts with the person who writes the code, and it goes all the way through the organization until the software hits the end user. There's definitely a test that could have been written for this, just like with the Heartbleed Bug with SSL, there was a very simple unit test that could have been written to catch that, and sometimes it's shocking when you find these things out, and realize we've been relying on critical pieces of software that were tested poorly.
Having a lot of high visibility incidents like this happen brings this to the forefront a little bit, which I think is good. I think it helps educate people and really brings the cost of this up front. I think you've probably seen that graph in every software theory book, about how the cost of fixing a bug is exponentially more expensive the later in the process you get, and of course by the time it's in the car on the road with a customer, that's a hideously expensive way to discover a bug. I think we're really understanding now, especially with how pervasive software is in our lives, that yeah, we got to pull that in, we got to find those bugs a lot sooner in the process. The sooner, the better.
Cameron: To have a little follow-up here, you mentioned about how it's a good chance to educate people. With this incident, Honda can kind of be a teaching moment for implementing and insuring your team is using the best practices related to continued delivery and agile. Can you speak a little bit on that, and how that's true?
Anders: Yeah. What I'd love for them to do, and my guess is they probably won't do it because they'll have PR people telling them it's a dumb idea, or lawyers, or both, is to use this as a teachable moment, and say, "look, here's our process, here's where it fell short, here's what we've changed and how we're going to catch these types of problems in the future."
As with anything like this, one of the things that you find once you implement a continuous delivery process, is that it gives you defense and depth. Your engineers are writing unit tests, you've got longer integration style or system tests that run, you're running burden tests, you got multiple sets of tests that get kicked off as a result of check-ins, because you're trying to keep yourself in a constantly releasable state.
So you're automating this. Anytime you need manual intervention to do testing, you've got a problem basically, you've now slowed down your process, you introduced the chance of human error, and all of those things. One of the valuable things about continuous delivery, although I don't think you'd see it as one of the principles of continuous delivery, I think you end up with, as a side-effect of the sort of defense and depth in terms of your testing methodology, because you're going to have lots of different kinds of tests you're running and automating that form the gateways you have to pass through in your release pipeline before the software makes it out to the customer. I'd love to see Honda use it as a teachable moment, we'll see what happens.
Cameron: My other follow up with that, is you mentioned Heartbleed, and there's a lot of instances now where stuff is kind of breaking and people are finding bugs. With all of these incidents going on with the Honda recall, the security breaches… do you feel like testing and the importance of testing is starting to get more wide recognition, or is there a bigger trend and picture that a lot of people on the outside are missing?
Anders: I bet the average person the street doesn't think about this too much, and that's probably a good thing. I certainly don’t want to have to worry about it every time I get in my car. In the industry, and that pretty much implies every industry because every company is a software company these days, thinking about testing and your approach to quality is important. When you read a headline like a software recall for a car, that kind of makes you take a step back and go, "OK, how do they test that? How do they make sure the software works?"
Back when it was all mechanical linkages and nothing was electronic, and the only thing that was electric were your headlights, there's a method for testing that. Very physical, very simple, compared to what it is today. Today you've got lots of systems interacting in cars with each other, every car these days is basically a distributed network of computers that need to operate together.
If there's a glitch in the MP3 player, that's one thing, but if there's a glitch in the accelerator pedal, that's an entirely different kind of scenario. One of the valuable things about continuous delivery is even if you are Honda, an embedded software company in essence, because they're shipping a car with the software baked into it, I think you still need to apply the lessons and a lot of the techniques of continuous delivery.
It is all about quality, and I think quality is a dial that once you start implementing these processes; you are able to crank up a lot higher than you were before with much better timelines. You can bet that they raced to fix this bug. You can bet that when they run into these kinds of issues, they're burning midnight oil trying to get this stuff fixed. Wouldn't it be nice to have just found that during the development process as a side effect of some testing that happened? No big meetings need to be had, no PR people need to get involved, you sleep a lot better at night when you take a more rigorous approach to quality and testing.
Cameron: You mentioned this earlier, how with the way that software is advancing so rapidly and it's becoming a part of everyday life, the Honda incident was a necessary growing pain. With that being said, and even though in the future you think that we're going to get much better at these things, do you think we'll see another incident like this anytime soon?
Anders: Absolutely, I think it's inevitable. I hate to be cynical about it, but I think we will. It will take a few of them before there's a lot of pressure. What I don't want to see is governments get involved and commissions are had, and we start talking about how we should regulate this and that. That, I think, would be a shame. The industry needs to be self-policing in that sense and really see this as a serious incident. This really does require us to take a step back and look at how we do things. But will it be the last one? No, I don't think it will be. Just enough of the cynic to see that that's going to happen again.
There's a positive side to it as well, you get better at doing this. Again, if you look at a Tesla, they're able to in a very short turnaround time update the software on their cars over the air. We’ll see it again.
Cameron: Anders, you've had various positions from technical leadership roles, to management roles, and now you're in the executive role with Electric Cloud. How has your view of technology and it's relation to the success of a company and its project changed through these various roles?
Anders: The arc of change in my twenty-five years in the industry has really been that software has gotten much more pervasive. It used to be that you would go in and sell a piece of software to an insurance company, and they barely had computers on their desktops. For about ten years it was going to be the year of the lamb, everybody was going get a network. Well, now the whole world is network, so that has had just huge impact and implication on how the software industry works. We have infinitely more people writing software now than fifteen to twenty-five years ago. It really used to be kind of a specialization, and I think it's becoming much more of a mainstream type of job now.
Again, every company is a software company and everything ships with software inside of it, so that brings a lot more focus on the practice and mainstreams it a lot more, and in the end that's good. Getting a little more attention on software quality, and you really would have a time shipping a product if it had bugs the way most software does if it was anything else. If half the time your cereal didn't taste like it should, you wouldn't accept that.
On the other hand, there's a lot of feedback loops in place now. So if you're a mobile app developer for example and you start to get a lot of one-star reviews, that's death. Part of what's happening, as well is the feedback loops are a lot tighter than they used to be. Again, there's a lot more attention given to problems with software. Even ten years ago, there was still lots of software in cars, but I bet you would never have had a headline in the newspaper regarding a recall for software in a vehicle, even though I'm sure it happened.
The fact that we are so used to now having, whether it's through social media or apps stores or whatever, have a way to throw feedback back at software manufacturers in a public shaming way, you might say. It is a very powerful influence on the industry, and I think it can influence the industry for good, that is ammunition. When you go to your management or go to decide how to spend your resources, that's ammunition to know that if we have a bug like this, not only is it embarrassing, it's bad for business. It hits our bottom line, that can be good for software practices I think.
Cameron: Right, it's very good for the consumers, too. We talked about your various roles here, but now you're with Electric Cloud, and Electric Cloud offers solutions for many different things, like embedded software, automotive software, mobile devices, the Internet of Things, Enterprise Technology, and a whole lot more. With that being said, do you have a favorite area of technology?
Anders: What I like to work on myself is automation of complex processes. That's one of the reasons why I really enjoy working at Electric Cloud and working with our customers as we do. We're working with some of the most complex pieces of software in the world, and some of the largest, hairiest bill test release deploy processes, helping our customers fundamentally implement continuous delivery powered by our products. That's cool, I really like that a lot. The automation part of it I think is really cool because automation is acceleration, and acceleration is a good thing. We definitely like to do that, I like to do that a lot.
Cameron: To kind of wrap things up here, one more question for you. You have fifteen years of experience designing software, so reflecting on your career, and knowing what you know now, what advice would you give to yourself in your younger days?
Anders: It's funny, I think one of the biggest lessons I've learned in the last ten years is just the importance of good unit testing. That sounds like motherhood and apple pie, but you'd be kind of shocked how much software is written without it. We get exposed to a lot of different software organizations as you can imagine, and it's a little shocking which ones out there aren't doing it. That's one of the cheapest ways to sleep better at night, is knowing that bug will never come back because you wrote a regression test for it.
Cameron: All right, that wraps up our interview for today, thank you so much. Once again, this was Anders Wallgren of Electric Cloud. He spoke to us today about the Internet of Things and how it affects the Honda recall.
Click here to read more insight from Anders Wallgren.
Anders Wallgren is Chief Technology Officer of Electric Cloud. Anders brings with him over 15 years of in-depth experience designing and building commercial software. Prior to joining Electric Cloud, Anders held executive positions at Aceva, Archistra, and Impresse. Anders also held management positions at Macromedia (MACR), Common Ground Software and Verity (VRTY), where he played critical technical leadership roles in delivering award winning technologies such as Macromedia’s Director 7 and various Shockwave products.
Podcast Music: "Han Solo" (Captain Stu) / CC BY-NC-SA 3.0
