Why Testers Should Take Control of the AI Narrative: An Interview with Tariq King and Jason Arbon

Jennifer Bonine: All right. We are back with more interviewers. I'm excited. I have Tariq and Jason, nice to have you both.

Tariq King: Nice to be here.

Jason Arbon: It's very nice to be here. Thank you for inviting me here.

Jennifer Bonine: Yes. Or, dragging, either way. Inviting or dragging willingly or unwillingly. So we get both you guys all at once.

Tariq King: Yep.

Jennifer Bonine: And, I heard that both of you like to talk about AI.

Tariq King: We do.

Jennifer Bonine: Yeah, and you have a new organization that was started.

Tariq King: Yep.

Jason Arbon: Yep.

Tariq King: AItesting.org.

Jennifer Bonine: So if people want to find it they just go to ...

Tariq King: www.AItesting.org.

Jennifer Bonine: AItesting.org, and they can sign up?

Jason Arbon: Yep.

Jennifer Bonine: And, what is the intent behind it? Maybe tell us a little bit about what you guys want to do with that.

Jason Arbon: AI is just to bring people together. Everyone's curious about AI for testing. Some are concerned about it, some really want to dive into it. The idea is to have one place where we can get together and talk about AI.

Jennifer Bonine: Yeah.

Jason Arbon: The origin was actually STAREAST. Tariq and I were talking about how there's ... We don't get together very often, there's very few of us working on the problem today. And so, we wanted to ... We're basically lonely people, we try to meet on the internet, and there's a mailing list. We find things, we have papers, we run into sort of interesting ... Or we survey the community, too, to see what they think about AI.

Jennifer Bonine: Yeah.

Jason Arbon: You sign up on the mailing list and start to get in the groove of what's going on with AI.

Tariq King: Yeah.

Jennifer Bonine: That's awesome.

Jason Arbon: It's AItesting.org.

Jennifer Bonine: AItesting.org. Got it. All right, so those of you out there that want to learn more about that, you can go to aitesting.org. A great way to bring people together, as you've said, right?

Jason Arbon: Yep.

Jennifer Bonine: Because, there's, I think, a combination of people who are probably super excited about it, a smaller group. A group that just doesn't know much, and really wants to get educated. And then a group that's probably a little bit fearful of what that means. I've even heard some testers saying, "I just hope I retire before it happens."

Tariq King: Yeah.

Jason Arbon: Yeah.

Jennifer Bonine: I think I've heard that, right?

Tariq King: Yeah.

Jennifer Bonine: And a lot of that comes from lack, I think, of awareness and education. So, having places where people can get information, education, awareness, is always good.

Jason Arbon: Right.

Jennifer Bonine: And one of the ways we did that this week here at the event, whether you guys saw it or not, was your keynote ...

Tariq King: Yeah.

Jennifer Bonine: ... that you did on it. So, for those that didn't get a chance to see that keynote—which by the way, if you didn't get a chance to see it, these are recorded, so go back and watch that. I've even heard some folks here that watched it live but are gonna go back and watch it again.

Tariq King: Awesome.

Jennifer Bonine: Because it had a lot of really good content for folks. So, maybe we talk a little bit about what the intent behind that keynote was, what you were trying to accomplish with that.

Tariq King: Okay.

Jennifer Bonine: Discussion with this group would be great.

Tariq King: So, when you think about it, artificial intelligence is really a hot topic these days. Whether you're in the business of building software or consuming software, it's all over. For me, I think that one of the biggest things in terms of the testing community is we've seen folks who are using AI for testing. I've seen folks who are kind of averse to it, like I said, there's these different pockets of people.

Jennifer Bonine: Yeah.

Tariq King: And I really wanted to just get everyone familiar with what AI means and what AI brings, and start to get people comfortable with thinking about, what does the future look like? Does being, quite frankly, a lack of innovation when it comes to pushing that boundary and testing, and testing seems to lie behind other aspects of software engineering. And I just wanted to try to open up people’s minds to say, hey, look. Let's not think that these things are impossible and that they're far away. They're actually here and they're coming.

Jennifer Bonine: Yeah.

Tariq King: And as a community, we have to get in front of it.

Jennifer Bonine: Mm-hmm.

Tariq King: And so, my message was really around, let's stop, think about what we can do to leverage this technology to make what we do better. But let's also notice that in terms of, for AI to successful, as with any paradigm, testing has to be a key part of that.

Jennifer Bonine: Yeah.

Tariq King: And so, the message was really to try to bring those two worlds together: Have people try to get comfortable with thinking about, "Hey, if our future looks different, it's okay." Right?

Jennifer Bonine: Yeah.

Tariq King: But then also, what ideas can we bring as testers? What things can we bring to this forum to kind of make our impact and make sure that at the end of the day, AI doesn't take over?

Jennifer Bonine: Right.

Tariq King: We need testers to kind of help be that ground and that foundation.

Jennifer Bonine: Yeah.

Tariq King: Yeah.

Jennifer Bonine: And that's great. And what's kinda nice about it, I think, for people that didn't see it, was you told the story backwards, right?

Tariq King: Yes.

Jennifer Bonine: So, you gave the end of the story before the beginning, right? So, you showed them what it is and then walked them through it.

Tariq King: Yeah, I think that when I look at it, because of the potential for AI to be so general and represent all types of problems, for me, AI is already here and it's already taking over, right?

Jennifer Bonine: Yeah.

Tariq King: And so, it's really about recognizing, okay, people, this is actually coming and it's already here. And it's only gonna get better.

Jennifer Bonine: Yep.

Tariq King: So, bringing them back to the beginning is like, now is the time to get ready.

Jennifer Bonine: Yeah.

Tariq King: So, the way that we actually were able to be successful was that we actually got ready in the beginning, because we saw it coming and knew how are we gonna respond. And so, it's really a call to people to say, "Right now's the time for you to get ready and realize that this is gonna come, and we have two sides of the coin, and what are you gonna do to get prepare for those two things?"

Jennifer Bonine: Yeah. Well, it was interesting, I was just talking to one of the other folks, right before you guys, about the fact that technology and change comes faster than ever before, things are moving and changing.

Tariq King: Mm-hmm.

Jennifer Bonine: And if we bury our heads in the sand and pretend stuff isn't here, it's not coming, or don't get aware or educated. You get, that gap gets bigger and bigger, and the opportunity gets wider spread. And people get more afraid, I think, sometimes, when they don't get information. It progresses, it gets more mature. They haven't been involved ...

Jason Arbon: The more they adapt, the more angst they have.

Jennifer Bonine: Yeah. And it just creates a bigger angst. And so, as stewards of information, it's good that you're creating opportunities for education and awareness. Because I think a lot of times what people don't realize is, change creates some level of fear, and a lot of times it's fear because they don't know, right?

Jason Arbon: Yeah.

Tariq King: Yeah.

Jennifer Bonine: It's the unknown. So, get some information and that can really help a lot with that gap ...

Tariq King: Yeah, definitely.

Jennifer Bonine: ... of just creating information and having, since it is a small community, people that have some good information, have been involved in it, know it's here, what it's doing, to educate others. And I see that with both of you. A lot of times at this conference, instead of just giving presentations and standing up in front of rooms, you guys are talking to people all the time, and they line up to get that information from you guys.

Jason Arbon: We even talk to each other sometimes.

Tariq King: Yeah.

Jennifer Bonine: Which is awesome. I'm glad that you guys talk to each other.

Jason Arbon: Yeah, it's awesome.

Tariq King: No, I think, when you think about conferences, this is really a place for us to build relationships.

Jennifer Bonine: Mm-hmm, yeah.

Tariq King: That's how I got to know Jason and what he was doing.

Jennifer Bonine: Yeah.

Tariq King: And I was actually just looking for who else was working in this space, and it was fortunate, so. People ...

Jason Arbon: We talked about this, collaboration is not relationships.

Tariq King: Well, collaborations, relationships, whatever. Potato, patato.

Jennifer Bonine: (laughing) You can tell they super agree on everything.

Tariq King: But, when you really think about that, we come here, we get inspired from other people. People are a valuable resource that we have. And the ideas that come out of conferences like this, everyone is all excited and ready to take things away, but I think maintaining those relationships and our collaborations is very important.

Jason Arbon: And AIST wouldn't have happened if it wasn't for STAREAST.

Jennifer Bonine: Right.

Tariq King: Mm-hmm.

Jason Arbon: It just wouldn't have happened.

Jennifer Bonine: Right. It was an idea born out of one of these events, right?

Tariq King: Yep.

Jason Arbon: Yep. Now there's hundreds of people, many hundreds of people, and we're sharing and taking surveys. I wrote, like, a twenty-page paper, which I never really do—I'm trying to copy you, you didn't like it that much, either.

Tariq King: I did.

Jason Arbon: A twenty-page paper, and I got feedback from like sixty different people.

Jennifer Bonine: Wow.

Jason Arbon: So, the community's growing, and they're pointing out flaws in my logic and in my grammar.

Jennifer Bonine: Oh, I have that happen all the time, too.

Jason Arbon: But, that's awesome. Yeah, thanks.

Tariq King: Exactly.

Jason Arbon: There's clearly available tools. If you put this into Word, it fixes it for you.

Jennifer Bonine: Yeah. Not always. It's not perfect, yet.

Jason Arbon: Well, that's what we said.

Jennifer Bonine: I know.

Jason Arbon: But, AI's not smart enough.

Jennifer Bonine: It's not.

Tariq King: Yes, it will get there.

Jennifer Bonine: It will get there.

Jason Arbon: But, that happened because of STAREAST. And then, I think what really happened more at this conference was, this is the turning point. So, when I was at STAREAST, I was like, "Are you able to talk about AI?"

Tariq King: Mm-hmm.

Jason Arbon: And there's one, and I was pretty lonely.

Jennifer Bonine: Yeah.

Jason Arbon: There was even a small room. This time it's a keynote, Paul had a session ...

Tariq King: Yeah.

Jennifer Bonine: Yeah.

Jason Arbon: ... and you guys both kinda synthesized, it brought the world together.

Jennifer Bonine: Yeah.

Jason Arbon: And then, I got a little manic a couple hours ago talking about contest animation in AI.

Jennifer Bonine: Yeah.

Jason Arbon: But we, on paper, tripled.

Tariq King: Yeah.

Jason Arbon: So, this is technically maybe exponential growth in the interests around this topic in testing. So, like you're saying, it's coming, and it's coming faster than ...

Jennifer Bonine: Yeah.

Jason Arbon: Change is coming faster than it used to.

Jennifer Bonine: It is.

Jason Arbon: So, we need to get ahead of the curve.

Jennifer Bonine: Yep. Well, and what's interesting, I think, I've talked to you before, I've talked to you before, at these same types of forums right where we're interviewing ...

Jason Arbon: And breakfast this morning, which was awesome, thank you.

Jennifer Bonine: And breakfast this morning. But, what we found is that some people are visionaries, right? Like, they see something before it comes and you see it coming, and having talked to you, I've seen this coming as well for a while. But it's interesting to watch the curve of other folks starting to get involved, get on board and see that this is, it's here. It's here.

Jason Arbon: Inevitable.

Jennifer Bonine: Yeah.

Jason Arbon: Right, right.

Jennifer Bonine: Exactly.

Jason Arbon: You've seen it actually kind of evolve, right?

Jennifer Bonine: I have.

Jason Arbon: Actually, you said awhile, it made me a little bit painful for a second because I'm like, I wish I could just code faster.

Jennifer Bonine: Yeah. But, it's two things, I think, right? It's not even just having the technology and the code, but it's the adoption, right?

Tariq King: Yes.

Jason Arbon: Mm-hmm.

Jennifer Bonine: Can people ... We have a lot of technology that's ready today, but people aren't ready for it.

Tariq King: Yep.

Jason Arbon: Yeah, absolutely.

Jennifer Bonine: Right? So, it's not even that it's not we couldn't have it, there's lots of stuff out there we could have.

Jason Arbon: Yeah.

Jennifer Bonine: But, people aren't ready for it, right? To absorb it, to accept it, to embrace it, right?

Jason Arbon: Right.

Tariq King: Yeah.

Jennifer Bonine: We have to get them to that stage of being okay with it.

Jason Arbon: Right. That's perfectly put. You're not supposed to upstage the people you're interviewing. But that's better said than either of us had said it this week.

Jennifer Bonine: I should be your PR person, right?

Jason Arbon: Yeah.

Tariq King: Yeah.

Jennifer Bonine: And Jason, before we leave, I wanna talk about something that's near and dear to my heart that I've talked to you about for a while, which is Appdiff, which now, there's a new name. So, people who are looking for that should look for ...?

Jason Arbon: Test.ai.

Jennifer Bonine: Test.ai.

Jason Arbon: So, things aren't going well, you change your brand. (laughs) No, really the reason we're doing it, just to be super clear: We built some really cool robots that will automatically test your app, right?

Jennifer Bonine: Yeah.

Jason Arbon: And people are paying for it, people like it. But what they also said is, "I want these—these are cool and smart robots, can't they do all my regression tests?"

Jennifer Bonine: Yeah.

Jason Arbon: And they go, like, "Aw, man," right? And so, I spent, we got more money from venture capitalists. Thank you out there, wherever you are. And built the engineering team even more. And now we have robots where you can very simply tell them what you want them to do, and they'll figure out how to execute that on any app for you.

Jennifer Bonine: Yeah.

Jason Arbon: Which is pretty cool. So, really what we're doing around the brand change is that we're just focusing on what people say. "What is Appdiff?" I have to tell them, it's an app and it finds differences in your application. So, what does test.ai do? We test with AI.

Jennifer Bonine: Yeah.

Tariq King: Yeah.

Jason Arbon: So, it's just super simple.

Jennifer Bonine: So, clarifying it for them?

Jason Arbon: Yep. Exactly.

Jennifer Bonine: Now, they're silly though, because I knew what it was right away. I knew exactly what Appdiff did, right?

Jason Arbon: I know you did, you did, yeah.

Jennifer Bonine: I get it, but.

Jason Arbon: But, you're also not normal.

Jennifer Bonine: Okay. But, test.ai makes sense.

Jason Arbon: Yep.

Jennifer Bonine: So, for those folks out there that haven't checked it out, the website ...

Jason Arbon: Is up as of 10 a.m. today. Test.ai.

Jennifer Bonine: ... is up as of 10 a.m. So, go check it out.

Jason Arbon: And if you go up there we're gonna—I sound like a ShamWow commercial—we're trying to get it. We have too many test members on my team. So, I don't know if we'll hit the date because they always find bugs. But, we hope to have it downloaded by Halloween.

Jennifer Bonine: Yay.

Jason Arbon: At test.ai you can sign up for the beta right now.

Jennifer Bonine: Awesome.

Jason Arbon: And the cool thing is, it's going to be AI that you can actually download to your local machine, run it against your app, and you've got automation for your app.

Jennifer Bonine: Awesome.

Jason Arbon: It's gonna be in a beta version but downloadable. And, you can have it on your MacBook while you're on an airplane and it will still run.

Jennifer Bonine: That's awesome.

Tariq King: Yeah.

Jason Arbon: Yeah.

Jennifer Bonine: So, check that out, you guys. I mean, newly released as of 10 a.m.

Jason Arbon: Yep. And the exciting thing is that I also just demoed for the first time on the planet at a conference, that AI running and executing a test case. I think that was actually the first time it's actually been demoed in a public forum.

Tariq King: Yeah.

Jennifer Bonine: Wow.

Jason Arbon: Half my team at home base hadn't even seen the demo because I had to fill in for another session.

Jennifer Bonine: Now, do you have the video of the demo?

Jason Arbon: I do have the video.

Tariq King: Yeah.

Jennifer Bonine: Can they see the video of the demo? Do you wanna be a part of that?

Jason Arbon: Yes.

Jennifer Bonine: Okay, you'll post that somewhere, right?

Jason Arbon: Actually, you'll have to talk to my marketing team.

Jennifer Bonine: Product marketing team? We need to talk to them for you so you don't forget.

Jason Arbon: Yeah. Actually I would love to, so.

Jennifer Bonine: That would be neat.

Jason Arbon: Yeah. And if you, if they won't let me put it on the web right away, email me at jarbon@—well, my official one, [email protected], or [email protected], and I'll send you a link to the video if you want.

Jennifer Bonine: Perfect.

Tariq King: Cool.

Jason Arbon: Cool.

Jennifer Bonine: So, that's how we find you, Jason. How do we find you, Tariq?

Tariq King: You can find me a number of ways. Tariq King on LinkedIn, there's a couple different Tariq Kings, but you should be able to find me there ...

Jennifer Bonine: You're the OG, though.

Tariq King: ... it's relatively easy. Yeah, exactly. [email protected].

Jennifer Bonine: Right.

Tariq King: It should be there, so. It's great. Cool.

Jennifer Bonine: Perfect. So, how to find Tariq. And congratulations, I hear there's gonna be a new King in the family.

Tariq King: Yes. We are expecting November 22.

Jennifer Bonine: Wow.

Tariq King: So, after this I go to make sure that I'm not in trouble by missing the test, so.

Jennifer Bonine: Right.

Tariq King: It's gonna be awesome.

Jennifer Bonine: Awesome. Well, congratulations and good luck.

Tariq King: Thanks.

Jason Arbon: I've got one more quick thing Tariq did not represent that should be represented.

Jennifer Bonine: Okay.

Jason Arbon: If you look at his keynote, I highly recommend you listen to it.

Jennifer Bonine: Yeah. We'll watch it again. We'll watch it.

Jason Arbon: But, what we're doing at test.ai is trying to—and I'll say it and scare everybody—we replace Selenium and Appium with AI, right?

Jennifer Bonine: Yeah.

Jason Arbon: So you don't have to code anymore and AI just does it, 90 percent of the work for you. The really smart thing that's gonna happen five to ten years from now is what Tariq was saying, and that is that systems will eventually test themselves from the inside out, right? Rather than from the outside in.

Jennifer Bonine: Mm-hmm.

Jason Arbon: And, the testing I do is from the outside in, it's very nontraditional, with swapping the brain out with machine learning. But his idea and vision is software that actually tests itself.

Jennifer Bonine: Mm-hmm.

Tariq King: Yeah.

Jason Arbon: And that's the stuff that stands between us and the singularity and the robots taking over. So, I'm really excited to see where he takes that research.

Tariq King: Yeah. There's really a lot of research that started from when I did my Ph.D. That's kind of the area that we looked at.

Jennifer Bonine: Yeah.

Tariq King: But, that's 2009, the time and the technology has really gone further.

Jennifer Bonine: Yeah.

Tariq King: And so, for us, like right now we're trying to see with all the advances in technology, microservices, artificial intelligence, can we really design systems in the way that they can test themselves?

Jennifer Bonine: Yeah.

Tariq King: Right? Because, when you think about software development, yes, we do a lot of testing from the outside. But we also do a lot of testing from the inside, and if we actually think about designing these systems with these features, then they can be a lot more thorough and actually get to a point where systems may not only be able to detect when they have issues but, they may be able to fix themselves.

Jennifer Bonine: Right. And theoretically, people like Jason and I, who have poor grammar, they'll learn how to adjust ...

Tariq King: Adjust them?

Jennifer Bonine: ... for our poor grammar. And we'll look like grammarians. We'll look amazing.

Jason Arbon: Grammarians. That means you have a good grammar.

Jennifer Bonine: I know, did you like that word? I liked that word.

Jason Arbon: That was pretty power ... That was a power word you dropped in.

Jennifer Bonine: I know. All right. I suppose we're done.

Tariq King: All right.

Jennifer Bonine: Thanks for watching, and check out our next interview.

Jason Arbon: Hi, kids.

Tariq King: Bye.

Tariq King is the senior director and engineering fellow for quality and performance at Ultimate Software. With more than fifteen years’ experience in software testing research and practice, Tariq heads Ultimate Software’s quality program by providing technical and people leadership, strategic direction, staff training, and research and development in software quality and testing practices. Tariq is a frequent presenter at conferences and workshops, has published more than thirty research articles in IEEE- and ACM-sponsored journals, and has developed and taught software testing courses in both industry and academia. His primary research interest is engineering autonomous self-testing systems. He is cofounder with Jason Arbon of the Artificial Intelligence for Software Testing Association. Contact Tariq via LinkedIn.

Jason Arbon is the CEO of test.ai, which is redefining how enterprises develop, test, and ship mobile apps with zero code and zero setup required. He was formerly the director of engineering and product at Applause.com/uTest.com, where he led product strategy to deliver crowdsourced testing via more than 250,000 community members and created the app store data analytics service. Jason previously held engineering leadership roles at Google and Microsoft and coauthored How Google Tests Software and App Quality: Secrets for Agile App Teams.