From One Expert to Another: Paul Poutanen

Paul Poutanen, a Calgary-based high-tech executive for the past seventeen years, entered the mobile field after a long stint as a senior management consultant with Ernst and Young. He developed extensive mobile expertise working in management for wireless hardware and cellular location firms such as Wi-LAN and Cell-Loc. Paul became president of Blister Entertainment, where he developed the first cellular location-based games for North America, including the award-winning title Swordfish. Inspired to find a solution to the complex global testing process for mobile devices, Paul launched Mob4Hire in 2007.

Jonathan Kohl: Tell me about SMS. What is it exactly?

Paul Poutanen: SMS (Short Message Service), aka “text messaging,” is the number one mobile application in the world. All handsets support it; there is no handset that does not allow text messaging. Even with the simplest handsets in the world, SMS can be sent and received.

All you need is the person’s mobile number, and a 140-character message can be sent from another mobile handset. SMS is considered ubiquitous in the mobile space, and the only standard digital app that can be found on any device. It is also extremely popular: According to Wikipedia, approximately 2.4 billion handset users use SMS.

Jonathan Kohl: I keep hearing people say that SMS is dying. Is this true?

Paul Poutanen: Well, no. It is an enormous industry. Again, citing Wikipedia, over 6 trillion SMS messages were sent in 2010. Also, in 2010, it was a $114.6 billion industry. For comparison, this is bigger than the global music, video game, and movie industries’ revenues combined.

The predictions of the death of SMS stem from the growth of other services such as email, instant messaging, and other communication services. However, service providers can’t count on other services being on every device they need to interact with. Also, while message systems like instant messaging, voice over IP, and video chat are growing, not everyone knows everyone’s ID. To use SMS, all you need is a phone number.

Businesses that require a service to communicate with customers using a ubiquitous platform still reach for SMS for a variety of reasons. If they want you to know something, they know that your mobile phone will have an SMS app. They can use it in many different ways. Companies may use it to try to sell you goods and services within marketing campaigns. If you respond to their SMS with a code, you may win a prize or get a special deal on something you would like to purchase. Companies like eBay use SMS for text alerts on bids for products you are bidding on. Mobile telecommunications companies use SMS to inform customers on usage limits, minutes remaining on a phone plan, and roaming charges. Some providers will send weather information, driving directions, business listings, movie information, and all kinds of things that mobile users are interested in via SMS.

Jonathan Kohl: I didn’t realize this, but you have told me that companies are relying on SMS for business-critical and even mission-critical services. Do you have any examples of how SMS reliability is becoming more important?

Paul Poutanen: There are a number of business-critical and mission-critical services using SMS as one of their means of communications. With email getting overloaded, institutions like banks are using SMS messages as an alternative communication system to alert their customers about overdraft, payments required, etc. Money can even be sent via SMS via payment service providers and financial service providers. If an SMS message doesn’t come through, you might lose money.

Security companies are using SMS as a mechanism for password backup. For example, if you travel, you might get an access code in an SMS message as part of your login ID. If an SMS message doesn’t come through, you may not be able to login and do your work.

Medical services or pharmacy providers can use SMS to send medication reminders to patients who are on a schedule for medication. Sometimes, medications can be complex and require a strict schedule to be safe and effective. If an SMS message doesn’t get through, it could have tragic effects on the patient, who is depending on outside help to take the right medication at the right time.Jonathan Kohl: Why do we need to test SMS messaging?

Paul Poutanen: Unfortunately, in a lot of cases it does not work. Again, according to Wikipedia, “… around 1 percent to 5 percent of messages are lost entirely, even during normal operation conditions, and others may not be delivered until long after their relevance has passed.”

The way it works is not simple, especially when looking at international SMS routing. There are over 1,000 carriers and network operators in the world. (No one ever seems to know how many.) If every one of those carriers had to make a contract with every other carrier, it would be very difficult (say, 1000!, or 4.0239 x 10²⁵⁶⁷ contracts required). So, they make deals with SMS aggregators. For instance, in one country, a carrier may have a deal with five aggregators that agree to send and receive SMS to every country and carrier in the world. Those five aggregators may have agreements with twenty other aggregators that then have agreements with one hundred other aggregators. By the time the SMS gets to the end carrier, it may have gone through ten servers of aggregators. That would be considered the “route” of the SMS.

You think your SMS has made it through, and you have confirmation of that from the last link in the chain (the carrier). Hurray!

Hold on!

The end carriers are very cognizant of spam SMS. If they think an SMS might be spam, they may not let it through. However, they have sent a signal to the last aggregator that the SMS has been received to the carrier gateway. The aggregator believes this to be a signal that the SMS has been received and their service-level agreement (SLA) has been agreed to.

So, if you roll the dice and your SMS makes it through the last part of the chain, the carrier might block it but not tell you. It isn’t a black hole; in some cases, the carrier tells you the SMS made it to their gateway. That is what most SMS SLAs aim for.

You can see it is a mess, but it gets worse. This process of getting an SMS to the end handset is dynamic. SMS aggregators may change their routes every day, meaning a message that was successful when sent in the morning may not work in the afternoon.

Jonathan Kohl: How do you test SMS messaging?

Paul Poutanen: It is quite simple, if you have the infrastructure in place to manage it. At Mob4Hire, we’ve developed a combination of factors to test this. We have an enormous amount of testers from all over the world (the “crowd”) who have signed up their devices. We have a test system in place just for SMS message testing, and testers register their devices within that system. They install test software on their devices that helps us determine if the message arrived to them or not.

When we test messages, we confirm that SMS messages make it to a particular country and carrier by utilizing testers who use that carrier in that country. The test is simple. It’s binary. It’s a pass or fail. Did the SMS make it or not? Our server software sends indication back to the SMS customer we are testing for with proof the message worked on that particular route. We also show them what their message looked like, especially if they have been utilizing Unicode to cope with different languages.

Jonathan Kohl: One knock on crowdsourced testing is that you can’t guarantee skilled testing. What sort of skill do people need to be able to test SMS messaging?

Paul Poutanen: Our testers really just need to own and have simple operating knowledge of an Android handset that has data and carrier connectivity twenty-four hours a day. There is no skill required, other than basic handset knowledge and being able to download and install an app. There is the extra bonus of having a human at the other end who can actually observe and tell you about any strange problems they are having.

Jonathan Kohl: Why do we need to look at testing services like crowdsourcing for mobile testing, particularly when testing SMS? Why can’t we just test this internally within our organizations?

Paul Poutanen: SMS testing depends on testing a lot of platforms (a combination of a handset, carrier, and country). Testing only within your organization won’t tell you much, unless you have offices in many locations where your services need to be used. Testing SMS requires that you have handsets ready to receive an SMS in the countries and on carriers that you need to confirm. If you have people in 1,000 country-carrier combinations with handsets and paid subscriptions, then you could do it all yourself. This is incredibly difficult and expensive to replicate.

Jonathan Kohl: Remote-device-access services for handsets are becoming more popular. Do they work with SMS testing? What benefits does crowdsourced testing have over these services?Paul Poutanen: Here, again, it is a geographical location game. Most of the remote-device-access services cover from one to six countries. It is incredibly difficult to try to replicate what is out there in the world without enormous cost. An SMS service provider has to purchase messages from different brokers in different countries. How many do you buy in each country? What if you buy too few in one and too many in another?

We have taken advantage of the natural ecosystem that exists out there, and we “rent” time on people’s devices. Mob4hire has over 57,000 people in our community in over 150 countries. If we tried to replicate that internally, it would be next to impossible.

Another bonus of using crowdsourced services is the human skill factor. Every device that is used for platform testing is controlled by a human. If something goes wrong, they can troubleshoot and bring their own devices back online. Since these are their own devices, they have a vested interest in the devices’ operating and being available for their own use, not just for platform testing. With other services, the devices are accessed remotely, so if something goes wrong, you have to wait for someone else to troubleshoot for you. Furthermore, since crowdsourced services have a lot of people involved, you can often quickly move to another device owned by another person if the one you were targeting is offline.

Jonathan Kohl: You’ve told me that there are brokers or middlemen who trade SMS messages in a type of commodities market. How does that work?

Paul Poutanen: Above, I described the relationships that providers go through by working with any combination of 1,000 aggregators worldwide. It sounded complex, right? When providers need to use SMS messages, they purchase them from these aggregators based on the number of messages they think they will need. It’s a complex system, and the prices to purchase messages can vary widely, depending on the level of service they promise, location, scarcity, etc. This system is simplified by using middlemen. Some people find price gaps for the same level of service in this market, so they buy up a group of messages from one provider for one price and sell it for a higher price to someone else who is willing to pay that price for the messages.

People have found creative ways to find margins in this exchange, just as they have for trading stocks, futures, or commodities like oil and pork bellies. Since the SMS market is international and complex, I’m not aware of any formal governance around this system. If you are a company depending on SMS in various companies, you may be getting different SMS products when you purchase the same package from a broker next time.

It’s a fascinating space with a lot of money and data involved, and I don’t think anyone knows exactly how it all works! This is just another example of the dynamic nature of this market, where testing can help businesses get more confidence in their SMS services.