Measurement and Agile – Oil and Water? (Part 4)

Are you feeling more confident that measurement and Agile can coexist? We certainly hope so! As you may have seen in the previous posts (1, 2, 3) of this blog series, many Agilists may shy away from measurements and metrics, but there are ways to not only make it safe for organizations to measure, but even derive great value from the insights gained.

A new home

Bringing quantitative and qualitative data together for a holistic and balanced view of the organizational performance and health is critical for the success of a measurement effort. Since ALMs are solely focused on their own data, they tend to not lend themselves to serve as a repository for all pertinent measurements. Therefore it’s a good idea to look for other systems that can store and aggregate the information as well as provide capabilities around analysis and visualization. Doing this in Excel and PowerPoint is certainly feasible in the short term, but in order for sustained success, a higher degree of automation is likely needed. In my experience, the operationalization of a measurement system is critical and can make or break the initiative. If too much manual work is involved, often by several parties that are sources of data, the effort can quickly grind to a halt.

There are several reputable platforms available in the market which can assist in gathering and reporting on data. The challenge however is that these often don’t actually “understand” the data they’re dealing with. They are general-purpose platforms that can ingest any type of data, from number of razor blades sold per month to throughput by team, but they’re not necessarily designed for Agile teams and organizations.

Another way to solve for this gap is for organizations to create their own custom applications that are cognizant of the Agile approach and organizational context. Since these organizations have development resources, it appears to be a reasonable thing to do to redirect some of these developers towards building tools or even a more sophisticated custom application to help in this area. However, don’t underestimate the duration and level of effort to not only create such an application, but also to maintain and operate it. Building a measurement platform is often not the core competency of the organization attempting it and even if it is, you’ll need to consider the opportunity cost, i.e. would these developers be able to provide more value if they could instead focus on the work they were hired to do and help evolve their organization’s software products?

Since I work for Agile Transformation and am involved with the AgilityHealth platform, I am – in full transparency – biased. However, I have not only been part of metrics projects in my earlier career but have also seen our product being leveraged as a customer. Given that experience, I can appreciate the benefits that AgilityHealth brings to the table: Gathering qualitative data via team-based surveys in a consistent manner across a large organization is not trivial. Even before that, designing comprehensive and meaningful survey questions covering all aspects of team health for teams following different frameworks and practices and aligning all of this to maturity stages takes a good amount of time. Once the information has been captured, a good platform must also be able to roll all the resulting data up, aggregate and present it in an easy-to-digest manner. As we discussed earlier, putting information in context is also important. AgilityHealth allows the user to look at teams or a whole organization relative to various benchmarks (for example based on industry), since it has data from a large variety of participating organizations. It also provides the ability to trend teams and different groupings of teams over time.

Looking ahead, AgilityHealth is well on its way to do what no other platform has done before, which is bringing together all key quantitative data (including business outcomes) and qualitative data, in the same system. This will provide the first truly holistic and integrated measurement platform accounting for all aspects of organizational measurement.

Lastly, it is more than a measurement platform as its focus is to serve as catalyst to teams’ growth and improvement while providing the tools, content, and processes to do that (see also Closing the Impediment Gap).

Given this breadth of functionality, the depth of R&D related to surveys and Agile, and access to a large data set, I recommend at least a closer look at AgilityHealth before you embark on the journey to either implement a generic metrics and dashboarding solution or create your own. Whatever approach you choose, don’t neglect the question of operationalization, especially as your measurement efforts scale to cover your organization.

Conclusion

To bring it all together: No matter how well-intended we may be, we cannot afford to stick our heads in the sand and ignore the need to measure meaningful things in our organizations ‒ not with the goal of judging, comparing or punishing, but to truly understand and affect meaningful growth and improvement and ultimately achieve business outcomes. We’ve seen how to start asking a few key questions, select suitable and balanced quantitative and qualitative metrics, and derive value from the insights we gathered, while staying away from the pitfalls commonly associated with measurement efforts. We also looked at the importance of automation and operationalization in order to make this effort successful and sustainable.
I’m sure this post won’t quell all controversy around measurement and metrics in the Agile space, but hopefully it provides some useful ideas and guidelines around measuring important things while still staying true to the Agile principles. At the end of the day, we want the same thing: help organizations and the teams within them to grow and be successful as well as healthy.

Appendix – Useful Resources

For those interested in additional information on the topics of measurements and Agile, here are some useful resources:

• GQM: http://www.cs.umd.edu/~mvz/handouts/gqm.pdf
• ODIM Model: https://medium.com/@lmaccherone/odim-12d80823222
• Impact of Agile; SDPI: https://www.ca.com/content/dam/ca/us/files/white-paper/the-impact-of-agile-quantified.pdf
• Metrics & SAFe: https://www.scaledagileframework.com/metrics/

By René Rosendahl, AgilityHealth Product Strategist
Image source: Stephen Dawson via Unsplash