The old adage goes "you cannot manage what you cannot measure". True enough, but where do you start in implementing a measurement program? What information is really important? And, how do you decide what to measure and what will matter to others?
No one in this audience will disagree that metrics and ongoing analysis is a crucial element in effective software project management. However, metrics programs do have their
challenges. Let's admit it, metrics can be difficult, time-consuming and challenging to derive. Development managers will relate tales of woe of their teams spinning through cycles to pull out metrics lacking in relevance to the business. This approach to measurement is a thankless exercise that can paralyze the entire project team.
But, there is a light at the end of the tunnel. Experts recognize and are beginning to agree on the key components that make up an efficient and effective measurement program. And there are methodologies available in the market today aimed at connecting metrics directly to business benefit.
This rest of this article looks at the vital role metrics play in effective software project management; gives some guidance on metrics that experts in the industry consider to be fundamental to good software project management; and touches upon a industry-accepted methodology called GQM, that can be a valuable approach in helping project and development managers define and align measurement to business goals.
Three Application Development Trends
Before we talk solution, let's take a look at some emerging trends in application development. There are three main trends that I've noticed. The first is that project-based development is moving to a process-based approach where automated processes and workflows are required to be more fluid and must accommodate ongoing adjustments and changes . The emergence of agile methodologies, compressed delivery schedules, and the recognition that software evolves on a continuous basis are principal drivers
here.
The next and related trend is the move from waterfall development to a more iterative development style. This methodology evolution is really in response to the speed of business transactions today. No organization can wait a year for a software project completion. Innovation must be rapid and ongoing.
Finally measurement within development organizations must make the move from ad hoc management reporting to real-time. The logic is there; if organizations are going to move faster and respond more quickly to business change - they cannot afford to wait
on information.
All of these trends indicate a dynamic shift in development towards greater speed, collaboration and agility (in the broader, business sense of the term).
Defining Metrics
So let's look at today's rapid application development environment and acknowledge that real time metrics should play a central role in keeping projects on course and making incremental efficiency improvements. For these purposes, let's define a metric in software development as the measurement of a particular characteristic of a program's, or processes, performance or efficiency. A metric is sometimes used directly and sometimes as an element in an algorithm calculating a more complex metric. For instance, in programming, a benchmark or milestone will include, or be driven from, metrics.
When you are talking measurement, it is important to realize that context gives the data meaning. It is not enough to know what a metric is, it is important to also know why the data is being collected in the first place and how that data, and the metric derived from it, will be used.
In addition, software measurement programs should allow project managers to quantify schedule, work effort, cost and quality performance in a way that is accepted and understood by the business. For many reasons, software projects frequently run over schedule and quality problems persist often derailing projects and some of this is attributable to a miscommunication or lack of appropriate and timely measures.
We attribute this challenge to the fact that most measurement programs employed for software project management rely on manual collection and manual analysis of data, or at the very least don't specify or point out that automated data collection would be beneficial. Today's fast paced business reality is all about having real-time, on-demand data representing actual activities instead of rough estimates and manual analysis. Manual measurement is just too slow. To sum it all up - metrics give software managers control and visibility and the more precise and real-time the metrics, the better the decision making capability of those managers and hence the better the project execution.
Four Benefits of Measurement
Overall, measurement provides four fundamental benefits - to characterize, to evaluate, to predict and to improve.
- To characterize - Metrics allow decision-makers to better understand
processes, resources and project/work status. They also establish baseline measures for future comparisons.
- To Evaluate - Metrics enable project managers to see how they are performing against plan or quality goals. The data shows when and where projects and processes drift from plan and allow for course corrections.
- To Predict - Metrics can help predict future performance by providing
understanding of the relationships between processes and projects. Through reliable data, software development managers can better estimate costs, schedules and work effort.
- To improve - A thorough metrics program isolates roadblocks, inefficiencies and root causes in order to improve efficiency, productivity and software quality. Once metrics have been established, they can be used to demonstrate alignment with business goals and can assist in communication with senior executives.
These four benefits result in increased project efficiency, greater collaboration, enhanced productivity and ultimately better use of resources (time, money and people).
Aligning IT to the Business Goals with Goal Question Metrics (GQM)
One way of establishing an overall metrics program is to begin with the business goals and demonstrate alignment of the software development objectives within that context. This is where a good process framework and measurement methodology comes into play. Take the Goal Question Metrics approach as an example. In this methodology, developed by Victor Basili, overall business goals are defined first. The criteria of success are then identified as questions - what do we want to achieve to satisfy our business
goals, what questions will help us plan and manage progress toward our goals. Finally,
the key criteria are broken down into an array of measures that answer those questions. GQM is all about knowing what you want to find out, identifying the right data to collect and why, and how that data is used to show progress towards the overall business goals.
Regardless of the methodology used, defining metrics that are relevant to the business needs to involve the business. By business I mean the audience of the metrics you are collecting. This may be the CIO/CTO and LOB owners or may be the team lead and their team members. Metrics defined in a vacuum will not be believed by the business so involve them in their definition.
The Role of Dashboards
When collecting data, however, there may be some pitfalls that can hamper your efforts. Manually collecting data can increase the overhead burden, be incomplete and subject to bias. Is the act of collecting the data outweighing the value of the data itself? It is too time consuming to collect metrics to promote change. You end up in a vicious circle when
implementing a manual collection paradigm.
Automating data collection to extract information is the key to making a strong connection to aligning work efforts to meeting business goals. A measurement system has to be analytic and able to be stored and show trending over time. It must not be a burden to the workers who input the data but the input of the data should be a natural by product of their day to day responsibilities.
Once the data is in the system, how is it communicated and presented to the business? This is where dashboards come into play. They are a great way to consolidate the data and make it accessible, visible and meaningful to everyone in the business. Further, if your dashboards are well designed and active, not simple a static snapshot, individuals will be able to use them as management tools rather than simply reports of data. Dashboards with drill-down capabilities that derive data from a live and connected ALM system enable you to manage what needs managing and avoid costly time delays and subjectivity.
Five Key Metrics to consider
Typically, there are five essential metrics or more specifically metric categories that software development should look at:
- Team Efficiency
This metrics takes a look at outputs/time. To maximize the value of this metric, it needs to be trended over time and projects. Team efficiency metrics can give software project managers insight into how time is invested and ensure that the efforts are aligned with business goals and how those efforts can be improved.
- Process Efficiency
Process efficiency metrics track and report on the health of internal processes as they relate to maturity goals (CMMI, Six Sigma, ISO etc). When working towards process maturity and achieving various levels of process excellence, software development managers gain knowledge about where process improvements can unlock efficiency and drive
increased productivity.
- ProjectEfficiency
These metrics demonstrate how well the project performed against the project objectives. Access to data around these project metrics can save money, time and effort and increase output considerably.
- Quality
Quality metrics assess whether the project meets the standards set by the organization. Software managers can include a pure measure of defects/ unit of output (function, LOC, etc) by project, by phase and can also include quality assurance metrics. Armed with this data, management can review overall processes and remove roadblocks to enable higher quality outcomes such as a lower number of defects per line of code.
- Value and Effectiveness
This set of metrics evaluates how well the project and its output align to the organization's strategic objectives. Software development managers can engage senior management in prioritizing projects and can align resources to the overall business goals.
Some final words of advice -
In today's businesses, increasing competitiveness for larger wallet share means that every department within an organization has to demonstrate efficiency, productivity
and the best use of resources. To demonstrate the value of IT within an organization, software development managers must be able to provide meaningful measures
of the effectiveness of their efforts and demonstrate the value of every budget dollar. With the alignment of a measurement program and metrics to overall business goals, development and project managers are better equipped to demonstrate their effectiveness and value at every level. And dashboards powered by automated real-time metrics are the icing on the cake - they give managers a never before seen insight into the workings of a project. Increased real time visibility and metrics allow you to make fact-based decisions to improve process, streamline workflow and tune team performance for higher quality and efficiency. The toughest part of establishing a metrics program within the overall business context is bringing it to life. Good luck and good measuring.
Doug Akers has been in the application lifecycle management industry for the bulk of his career after graduating from McMaster University with a BSc in computer science and a minor in psychology. Joining MKS in 1995 Doug has worked with support, development and marketing groups before moving into product management where he is currently responsible for the Portfolios and Requirements feature sets within the MKS Integrity product. In his role Doug works with global 1000 organizations to help them implement solutions and address challenges within the software development disciplines.
