Outputs Or Outcomes: Measuring What Matters

This article was based on episode 82 of The Modern Manager podcast. To hear this episode, and many more like it, you can subscribe to The Modern Manager Podcast on iTunes, Google Play, Spotify, iHeart Radio and Stitcher. Get a free digital copy of Jeff’s latest book: Lean Vs Agile Vs Design Thinking when you become a member at mamieks.com/join.

The most successful companies in the world today are the big tech organizations including Facebook, Apple, Amazon, Google and Netflix. These organizations thrive not only because they provide services in a new and modern way, but also because of how they operate. They have prioritized and built capabilities to observe and measure people’s behavior in real time, communicate internally at extraordinary speed, and innovate continuously. They have revolutionized the model of how to deliver value to customers.

Jeff Gothelf partners with executives and companies to help build customer centric and employee centric approaches to management and product development that work within cultures of continuous learning. He works as a coach, consultant and keynote speaker. He helps companies bridge the gap between business agility, digital transformation, product management and human-centered design. He is the co-author of the award-winning book, Lean UX, and the Harvard Business Review Press book, Sense & Respond. Most recently, Jeff co-founded Sense & Respond Press, a publishing house for practical business books for busy executives.

Jeff recommends that every team, no matter what your function, product or industry, can be more effective it adopts the practices of tech teams. In essence, he says every team should focus continual engagement with their stakeholders and measuring outcomes, not outputs.

AMAZON UPDATES ITS CODE FIVE TIMES EACH MINUTE (ONCE EVERY 11.6 SECONDS)

Do you remember when software came in a box? I do. You’d purchase the latest version, load the CD into the disc player, and copy it onto your computer. (If you were like me, you’d then save the discs and all the instruction manuals in a file cabinet never to be referenced again.)

This old model of software release mean that companies were removed from the real-time experience and feedback of their customers. It also meant that updates could only be made at specific times when enough changes had been made to warrant a new release.

Now, in the age of cloud computing, updates are made as frequently as every 11.6 seconds! Most of the time, we don’t consciously experience the changes, but on occasion, you might. I’ve noticed an alert on my Asana page almost daily, telling me a new version of the software is available and I need to refresh the page.

People engage in perpetual interaction with software like Amazon and Asana, giving these companies continuous and valuable feedback. Based on this feedback, teams are able to continually move through the cycle of learn, update, learn, update, which ultimately improves the products.

Clearly not every team, functional area or product offering can have this level of engagement with its customers or gather this level of detailed data. But the approach is still one worth exploring.

THERE IS NOTHING MORE POWERFUL THAN TALKING TO THE PEOPLE WHO USE THE PRODUCTS YOU MAKE

The vast majority of teams will never gather systematized data like clicks and mouse moves. Even though it may have no statistical significance, conversation can provide similarly useful qualitative feedback.

In order for these conversations to generate helpful content, you must be willing to accept the fact that you will be wrong about your ideas, no matter how brilliant and beautiful you think they are. In software, ideas are tested by beta users and if they don’t produce results, those ideas are discarded in favor of new ones.

The same mentality is required for any customer conversations. You experiment with a new service or alternative approach, gather feedback from your customers, (have your vision shattered just a bit as you realize your idea wasn’t as perfect as you hoped) and then iterate.

Jeff believes that this customer conversation is by far the most impactful and powerful thing that you can do, even if only in small quantities. It is humbling, it is educational and it will improve the product you make or service you deliver more than any other activity.

SHIFT THE FOCUS FROM OUTPUTS TO OUTCOMES

The critical question you may be asking at this point is, “what are we iterating towards?” This is the second aspect of thinking like a technology team.

Often, teams focus on outputs or shorter term deliverables rather than outcomes, the changed behavior of the people who consume or experience the product or service.

Outputs either occur or they don’t, making success easy to measure, manage, reward and incentivize. Questions for measuring output include: “Did we ship the product?” “Did we meet the deadline?” “Did the project come in under budget?”

Outcomes occur on spectrums. Measurement and management of outcomes is complex and multifaceted. There are multiple shades of grey between failure and success. The questions for measuring outcomes may include: “How do we measure our increase in productivity?” “How should we measure impact on consumer behavior?”

Example: Measuring Outcomes As An HR Team

Methods for identifying and measuring outcomes are not always apparent. It can feel especially challenging when there may be no interaction with customers and few obvious ways to gauge reaction to work products. The following is an example of the effectiveness of managing outcomes as compared to output when implementing a policy in human resources.

Assume that you work in HR on talent acquisition and retention, and that you are in charge of vacation policy. You’ve been asked to update the vacation policy. To help you determine the right policy, you must first understand the outcome the policy is seeking to achieve. Is the long term outcome to make the company more competitive and attractive to new talent? Or, is the objective to have a better rested, higher functioning employee base because you’re worried about burnout?

You must know the behavior change that you want to see in your staff that tells you if the vacation policy that you chose is the right one.

Based on the goal of greater productivity through more well-rested staff, you decide to implement an unlimited vacation policy. The belief is that people will take more vacation and therefore will be better rested which will result in them being more productive while at work.

As it turns out, research has now shown that in the US, when companies have unlimited vacation time policies, their staff actually take less vacation. When vacation time doesn’t expire and there is a seemingly infinite amount of it, then there is no pressure to use it up.

If the measure of success for this project was determined solely by output, (“Did you deploy the vacation policy?”) then this project was successful. The true measure of success can be determined only by outcomes, (“Did you get the staff to be better rested, more productive and efficient?”) and in this case, it would have failed.

HOW TO INTRODUCE A TEAM TO MANAGING OUTCOMES

Working towards outcomes can be a significant change for managers and their teams. Complexity and uncertainty are high. As a team leader, you must be prepared to be on a learning journey along with your team.

According to Jeff, one of the most valuable managerial traits Is humility, which you can model by saying, “Look, I also don’t know the answer.”

The first question to ask of the team: “What is the problem that we are trying to solve?” Then, ask: “If we solve it, how will we know?” This question must be answered in terms of what people will be doing differently. The answer cannot be found in terms of what was built and or delivered.

This approach encourages and inspires creativity, innovation, experimentation, learning and agility. It is more effective than almost any tactic, process or diagram could ever be. It is important to share with the team that things might go wrong, but as long as there is a willingness to strive for the long term outcome, experiment, talk with and learn from your customers, there will be reason to celebrate progress.

KEEP UP WITH JEFF

Twitter: @jboogie

Publishing Company: https://www.senseandrespondpress.com/

Get a free digital copy of Jeff’s latest book: Lean Vs Agile Vs Design Thinking when you become a member of the Modern Manager community at mamieks.com/join.

This article was based on episode 82 The Modern Manager podcast. To hear this episode, and many more like it, you can subscribe to The Modern Manager Podcast on iTunes, Google Play, Spotify, iHeart Radio and Stitcher. Never miss a worksheet, episode or article: subscribe to Mamie’s newsletter.