In video game development, the pressure is on to add more features to sell more games. Getting buy-in to automate testing can be a challenge. Fazeel discusses the sports video game development environment and how to convince product owners to spend precious resources on automated testing.
Fazeel Gareeboo will be presenting a presentation titled "Game On: Automating Sports Video Game Testing" at STAREAST 2014, which will take place May 4–9, 2014.
About "Game On: Automating Sports Video Game Testing":
Sports video games are generally on a short cycle time—tied to the start of a particular sport’s season. Like all video games, the pressure is always on to add more features to sell more games, and the list of "cool" features is endless. Getting buy-in to implement automated testing in this environment can be a challenge. And once you get that buy-in, your next challenge is to ensure it provides significant value to the game team. Fazeel Gareeboo shares the lessons they learned at EA Sports—lessons you can take back to your project. Fazeel describes the sports video game development environment and discusses how you can overcome the reluctance of product owners to spend precious resources on implementing automated testing. Fazeel addresses how to get buy-in from the team and finally how to make your product—game or not—more successful using automated testing.
Cameron Philipp-Edmonds: So today we have Fazeel Gareeboo, and he will be speaking at STAREAST 2014, which is May 4 through 9. He will be doing a presentation titled "Game On: Automating Sports Video Game Testing." Fazeel is a software development director at EA Sports Tiburon. Fazeel manages a team that provides automated game testing for the studio. Before moving into management, he developed computer-aided design software and wrote device drivers for Windows and MicroStation. Fazeel has a special interest in automating any repetitive work and in creating great teams for the kinds of work that computers cannot do yet. He grew up on the island of Mauritius and worked in Europe before settling in the United States. Anything to add to that?
Fazeel Gareeboo: No, that sounds good.
Cameron: OK. Because you’re doing a session titled "Game On: Automating Sports Video Game Testing," which covers the world of testing as it relates to video games, I would like to ask you some related questions. So, the first question is: What makes testing sports video games different than testing any other video games, or any other software, for that matter?
Fazeel: I would say the big difference between testing games and most other software is that technically, most other software, there is a fixed set of functionality that you're writing to, while with games, generally it's speculative. What you are going to add to creating the game is your judgment of what is going to make the product sell. And technically, most teams tend to put as many features, as many bells and whistles, because they obviously want to sell the game. So the bias tends to be more on doing more features and doing more bells and whistles. And I think that's the challenge—that creates a challenge for testing because you always have a limited amount of time. So the focus is "Well, let's focus on getting the features in, because that's what's going to sell the game."
Cameron: Right. So it's more about how it looks and the appearance of everything as opposed to how it actually runs.
Fazeel: And the functionality. And the focus is being pushed towards "Let's make the game. Let's have the features. Let's make it something fun," before they look at testing. It tends to be more pushed towards the end, not like the traditional "build it, test it, release it" model.
Cameron: And I imagine that with sports video games, because of the way that sports season run, it would be more of a strict timeline to testing.
Fazeel: Yeah, and that's another constraint that is added to the sports video games. It's exactly right. It's typically like a nine-month turnaround time. We don't really have twelve months because we are dealing with a release, and then you have some downtime, and then you're ready for the next one. So, it's more like nine months of development time that you have. And, yeah, you can't really typically push the release date.
Cameron: And you're a key proponent of automated testing, and you even discuss in your session how to overcome the reluctance of product owners to spend precious resources on implementing automated testing. So, what about automated testing is so appealing to you?
Fazeel: Well, I would say what's appealing is really automation, to me. If I'm doing something that's repetitive, I usually start with if somebody else can do it for me, and then I extend it to, well, maybe a machine can do it for me. That's even better, because a machine doesn't need breaks, typically doesn't make mistakes. I mean, there are other challenges with machines, of course. But that's where I start off.
If I look at something that I'm doing on a regular basis that takes a significant amount of time but that's repetitive and does not really require a lot of thinking, a lot of creativity, I'm looking for a way to automate it. And checking whether a game launches or just runs to the start screen really does not need to require a human. Checking whether a game can go through a set of menus without crashing really doesn't require a human. Other types of testing do, but this particular type—I would call it my stability testing—that doesn't require a human.
And the other side of it is, the more of it we can do the more frequently, the easier we can pinpoint where the defect was introduced in the game. So then we can go talk to the software developer who has introduced it. And because we're fast—so the software developer does something that checks it in and that breaks the game—if we can go back to them five days later, it takes them a long time to go figure out they did, what could be wrong. Whereas if we turn around within fifteen minutes, we say, "Hey, you checked something in that broke the game and here is where it broke," they still have it fresh in their minds and it's a lot faster for them to fix it.
Cameron: So, you're kind of saying that automated testing or automation can kind of be a prerequisite to manual testing.
Fazeel: Yes. I would say it frees the manual tester from a lot of the repetitive stuff so that a human being can then focus on really the added value that the human being brings to the testing process. A computer doesn't know that—well, currently, at least. It doesn't know that, "Hey, we've got the wrong face on this player," or "This particular character is not running right," or "Hey, the game is just not fun to play." The current state—at least of our systems—don't do that.
So there's other things that we need human beings to do. Because at the end of the day, we're selling it to human beings—we are not selling it to computers. So we need human beings to try and test it.
Cameron: You know, not everyone who will attend this session that you're giving will work on video game software or has worked on video software. So, what is something you would like everyone attending the presentation to learn? What is kind of the grand takeaway?
Fazeel: I would say there are a couple of things. But the main thing is identify repetitive parts of your testing. Automate them, but also include this automation as part of your development process. There is a movement in the software industry now called continuous development. So basically it's you develop, you test, and then you release, and then you keep repeating the cycle—as opposed to in the old days, we worked on specifications for a couple of months, then we work on the program for maybe a year, then we release. So we're moving to smaller bits of development being more agile. But I think a big component of that is having automated testing in that loop so that you're not having to rely on an army of people to test the software.
Cameron: Now, you have somewhat of an interesting link to your career. You developed a computer-aided design software and you wrote some device drivers for Windows and MicroStation. But you've also done some development for none other than Bill Gates's home. Can you speak a little on that?
Fazeel: Yeah, sure. That was an interesting thing. I had just started this job at a graphics card manufacturer in the UK. And my first task was to do this 2048x2048 Windows driver for a card that we made, and it was meant for Bill Gates's house. Apparently, he has these $10,000 monitors in his walls and he can change which artwork is displayed through some software. So, that was my first job. And there was an existing Windows device driver—I think it was Windows maybe 3.1 or 3.5. It was one of the early versions of the ones that kind of work, not the first ones. It was definitely a very interesting first job at that place. The cost of the graphics card itself was $10,000; the cost of the monitor was $10,000. So, I guess it's not surprising that he was Bill Gates.
Cameron: So, I take it you don't have any monitors in your walls?
Fazeel: Not as of yet. I'm working on it, though.
Cameron: OK. And you also have a passion for creating teams for the kinds of work that computers can't really do yet. So, what kind of work are you really referring to?
Fazeel: I think it's more the creative side. Computers are good at, you know, you can program it and then it will do a certain set of things. So you have to know beforehand what is it that you want it do, whereas human beings, you can ask them to create something as a team. And as a team they will create something, but you typically have no idea what it is. I would say some examples would be a performance, a piece of music, a piece of theater. I don't think we're at the stage where we can get computers to create those for us, because they just don't get it. I mean, they aren't meant to create those types of things. But typically most difficult problems these days require teams of people. And that's the other thing that I'm interested in, is how to get teams of people to work together and create great things.
Cameron: And, you know, you talked about computers being able to kind of decipher creativity. Do you think there will come a time when computers are able to do that and kind of take on the role of humans in testing? And when do you think that will be?
Fazeel: I don't think they will be able to completely take over that role. I think there are more and more things that computers will be able to do. But again, it's got to be more along the lines of automating something that's repetitive. There is a new area of artificial intelligence where people are using kind of neural networks as a way to, in a sense, mimic human behavior. That may be an area where people could, in a sense, train neural networks to be able to do the job of the tester. But we're not there yet. I don't know when we're going to be, if at all.
Cameron: And if you did know when it was going to be, you'd be a very wealthy man in that aspect.
Fazeel: Right! Yes.
Cameron: All right. So, you work for a video game company. Do you have a favorite sports game or video game title that you've worked on?
Fazeel: I wouldn't say I worked on it—I've had some contribution to it—but it's FIFA; it’s the soccer game. It's actually built out of our Canadian studio in Vancouver. I love the game because I play soccer. So for me, it's a way to actually play the game without being out on the field and actually getting hurt or hurting someone.
Cameron: Do you have a favorite team you play with?
Fazeel: Typically it's Barcelona. Sometimes with Barcelona or Real Madrid. There's a funny story. So typically after work, I play against a couple of guys. And for the longest of time, I think it was about two years ago, we both picked Spain, so we played Spain versus Spain. And that was just to kind of iron out the differences. So we can't say it's the team, it has to be the person.
Cameron: There you go. Now, it isn't uncommon for people who work in the software industry to be somewhat video game enthusiasts, kind of like yourself. So, with that being said, what is it like working for a video game company?
Fazeel: It's great. I mean, typically, for EA—and I can only speak about EA—they have lots of perks, I would say. We have free breakfasts. We have lots of game machines all around the floor. I mean, typically when people come and visit, they think maybe that's what we do, we play games. Obviously that's not true. We do work. And because of the types of games that we work on, sports video games that have a fixed deadline, there are usually periods of time when we have to work longer hours because we have to get this thing out. But it's a great place, great people, great benefits. I'm still here after eight years. What can I say?
Cameron: Fantastic. So, is there anything you'd like to say to the attendees of STAREAST before they attend the conference and before they attend your presentation?
Fazeel: I look forward to seeing you guys.
Cameron: All right. Fantastic. Once again, this is Fazeel Gareeboo, and he will be speaking at STAREAST 2014, which is May 4 through May 9. Make sure and check out his presentation titled "Game On: Automating Sports Video Game Testing." All right, thank you so much.
Fazeel: OK, thanks, Cameron.
A software development director at EA Sports Tiburon, Fazeel Gareeboo manages a team that provides automated testing for the studio. Before moving into management, he developed computer-aided design (CAD) software and wrote device drivers for Windows and MicroStation, including the first 2048x2048 Windows device driver—for Bill Gates’s house. Fazeel has a special interest in automating any repetitive work and in creating great teams for the kinds of work that computers cannot do yet. He grew up on the island of Mauritius and worked in Europe before settling in the United States. Reach Fazeel at [email protected].
