New Ideas for Effective Localization Testing Practices in an Agile Cycle

The past decade has seen the rise ofagile development, leading to a significant change in the way software is created, released, and accepted in the marketplace. Today’s software organizations are more nimble and more receptive to end-user feedback than ever before. In the market, there is healthy competition amongst players with a true drive for the “survival of the fittest.”

While internal challenges continue around team and project management, largely speaking, the industry has embraced the agile implementation fairly well. A survey conducted in late 2012 and early 2013 asked how agile organizations claiming to be agile truly are, and of the five points they were evaluated on, most organizations fared well with the criteria.

A New Challenge: Localization

Agile teams tend to think of releasing one product per iteration, and that shift from “every year or two” to “every week or two” is a good thing. More challenging is the simultaneous release of multiple products, each tied to a different language. While this endeavor might seem reasonable with versions in English, French, or Spanish, some teams have to support Greek, with special letters; Hebrew, written right to left; or Japanese and other character-based languages.

In the waterfall days, localized releases were often handled sequentially, and localization functional testing was carried out after a product was first certified in English. This was not a coincidence—it was because of the time required to carry out additional activities such as finalizing strings, getting them translated by linguists, and retrofitting them into the application. Given all of this work, it was intentionally taken up later to keep localization operational costs low after having tested and cleared defects from the English implementation. However, we do not have that luxury in time now, and global markets expect to see localized products released along with or soon after the English release.

While effective practices in localization testing have been suggested for optimal operations in any environment, it is becoming even more important to have such a list for an agile localization test effort. This is a list of practices to help ensure on-time, on-cost product releases, a synchronized effort with the English development effort, and a good handshake between all team members to keep their motivation levels high. For easy reference, I will categorize this list under three broad groups: technical and testing practices, team processes, and team positioning.

Technical and Testing Practices for Localization

As mentioned previously, traditionally localization schedules have run behind the English releases so as to get a localization testing-ready build that has translated strings retrofitted into it. In my experience, what is helping in an agile mode of operations is taking on feature-level localization. For instance, if four features are ready of a total of twelve features planned for a certain sprint, teams are encouraged to go ahead with the localization effort on them rather than waiting for all of them to be ready.

Picking up a small set of features to localize is very efficient and enables all teams (core/English, localization functional, and linguistic teams) to work in parallel. This helps verify the selected features in all languages, including English, in one go. Also, working up front with the content team to identify strings for translation even before they are ingested into the system helps the test team take on linguistic testing early in the game.

While the product team and specifically the test team are busy with core English testing, they may not set aside adequate cycles for internationalization and pseudo-localization testing. Even if these are done right, people may question the value of localization testing. Creating the right awareness in the team is worth an upfront investment from the test management team. Once the awareness has been created, a robust testing strategy for globalization testing with a specific focus on automation and regression testing is very valuable. These are areas where a lot of reuse can be built in—reuse where you leverage the work done by the English testers. Often, the lack of collaboration between the English testers and localization testers leads to a lot of wasted effort where potential reuse opportunities (such as test cases, automation framework, etc.) are missed out on.

The right areas of test optimization also need to be looked into, such as what kind of a test matrix to build that accommodates testing across operating systems, browsers, mobile devices, and locales. It must also consider locale nuances (single byte, double byte, or multibyte character sets) so as to build an optimal test matrix that provides maximum test coverage in the minimal available time. Ideally, a locale nuances checklist is a good knowledge repository to build whenever the team has time, and it also can serve as a training material for any new team members so they can understand the localization testing process specifically followed by the team.

Team Processes for Localization

At a team level, there are a few core things to keep in mind to build an effective localization testing effort. Firstly, collaboration between the core functional testing team and the localization testing team is imperative. When my projects had just moved to an agile lifecycle, those of us in the core functional team hardly had any idea who the localization test team members were. We just had a localization testing project manager to work with, to whom we would turn in the required information to get the testing done.

Looking back, this may have been possible in the waterfall days, but it is definitely not scalable in the agile days. There is so much room for cross-collaboration between the two test teams that they should even look at building some paired test slots over the testing lifecycle. Understanding what tests are being tried by the English-language team, areas where the product needs more stabilization, what kind of test data (positive, negative, blank, or boundary) are being used, what goes into the test automation and regression effort, and how the test optimization matrix is being built for compatibility testing are all discussions the localization team should have with the core team and can significantly benefit from. In turn, they also can contribute to these discussions and help the English-language team arrive at the right internationalization and pseudo-localization tests to find any potential globalization issues early on. Such discussions will improve the entire test team’s business understanding of the product, too.

Similarly, localization bug bashes could be arranged with a few members across the product team. Such practices go a long way toward building tighter collaboration with the localization test team, empowering them to become a part of the core product development team.

Planning for the localization test effort also should be prioritized along with the core testing effort. It is completely justified to have localization test representation in sprint planning meetings. If kanban is being adopted, it should be extended to the localization efforts as well. This not only brings consistency to team-wide operations, but also makes the overall process easy to manage and scale.

Team Positioning for Localization

Typically, in the waterfall days, a localization test team was called in on demand. There was not much value in retaining them on the team throughout the lifecycle, as they would then be idle for several days and months. So they were shared amongst products and would jump from product to product whenever there was a need. While this created resource conflicts at times, for the most part it made sense to build an optimal resource cycle in using the localization testers.

However, with the advent of agile, the localization test team is as busy as the core team throughout the product development cycle. They are actively involved in product discussions, mostly from a localization angle, and can engage them productively even as the English-language product is still shaping up. Given this ongoing engagement model, it makes complete business sense to have a resident localization test team on board. This also supports building internal subject matter expertise and minimizing context switching between projects, making operations more efficient.

Finally, considering a crowd test team for localization testing is very valuable. A crowd, in simple terms, is the community at large, whether internal or external to the organization that contributes to the testing effort. They typically partake in a test effort based on their testing skills, subject matter expertise, or end-user representation. In the case of localization testing they often pitch in for their subject matter expertise (given that they are proficient in the language under test) or for their end-user representation (where they would be a potential end-user when the product is released).

Localization testing is one of the very appropriate test areas to engage a crowd team on. A few years back I asked someone from an organization called Live Mocha, which builds software to learn languages, about how they test their language software. They often source language experts from the community at large to create their content. The employee mentioned that they leverage the crowd not just to create the content, but also to verify it through their linguistic testing work.

While an external crowd worked for their situation, an internal crowd could be very valuable in the case of large organizations. Take an organization with ten thousand employees that has a global presence and is building software across locales. While there may be a resident localization testing team along with a translation team, they could look at leveraging their diverse employee base across geographies to additionally verify the product. This could be a bonus test organized in the form of simple bug bashes or games to yield desired outcomes, yet keeping the crowd’s overhead minimal.

Given the limited time you have on hand in agile testing cycles, bringing in new ideas for testing can be very valuable. It may seem like additional work the first few times, but in the long run it will definitely stand out as an effective localization testing practice.

Making It All Work

The success of a software development effort largely hinges on effective team collaboration. Over the last decade, the product team gave the test team the right recognition and embraced them into their fold to make agile development a reality. Adding technical, testing, and team practices will help achieve the same result for localization work—giving teams the recognition and empowerment to build localized software that will sell and play in a global marketplace.