Measuring the Impact of Design
The outcomes of design become more powerful when measured against business goals.
Business goals are usually measured by revenue, cost savings, feasibility, or other key performance indicators (KPIs).
Surprisingly diverse viewpoints on measuring the impact of design
NNGroup suggests there are four steps to calculating the impact of design while speaking the language of the business.
- Collect a UX metric in a benchmarking study before any changes are made.
- Choose a KPI that matters to the organization such as Profit, Cost, or Churn.
- Convert the UX metric into the KPI. This means taking one unit and turn it it into another. E.g., the number of contacts to customer support before the redesign, the number of contacts after, measured against the cost per contact.
- Report the calculation responsibly.
Alan Cooper is against measuring the impact of design.
“ROI is an accounting term of art that originated in the days of expensive factories and cheap labor. It simply is not a useful measure in the digital age. Those who ask for it are using it as camouflage for something else.”
“The ROI of UX is exactly the same thing as an estimate of how long software development takes. That is, irrational, non-existent, unmeasurable, irrelevant, and ultimately it’s just a cognitive stick used to thump practitioners.”
“Of course there is value in interaction design. Nobody disputes that. Is there a “return on investment”? It’s not that there isn’t, it’s just that you cannot calculate it. And anyone who demands it from you is not interested in an actual answer.”
Tom Landino, Manager of Business Intelligence with Covenant Eyes recommends collaboratively measuring impact with meaning by using a hypothesis related to business measurements.
Landino suggests a specific pattern to measure the effectiveness of products and features, from baseline to new release:
- Fill in this sentence as a hypothesis:
“Our [name of product or feature] effectively addresses[user need]“ - Identify every expected impact if the statement is true and if it is false by finishing the sentence:
“We will know [product or feature] effectively addresses [user need] if we see [specific measurements related to the business].”
Sometimes factors impeding honest measurements
For example, concurrent changes by Design, Marketing, Support, and Product make it difficult to measure the impact of a single area.
Sometimes designs are intended to result in in long-term value and are improperly measured by short-term measurements.
Keep measurements accessible
Landino’s model simplifies measuring the impact of design. Designers collaboratively identifying meaningful measurements with business leaders and sharing research outcomes freely builds credibility and trust.
This template table is an example for showing the impact of design. Create one of your own with metrics meaningful in your industry in the tools you use — Google docs, Confluence, etc.
Fill it in with relevant metrics and links, honestly evaluating your designs. Share it freely with your stakeholders and business leaders.
Discuss it with them when planning user research to compare against business goals. When designers hold themselves accountable for business impact, organizational trust in design grows.
