Measuring the Impact of Digital Products
Digital products in the climate tech space often claim to have a positive impact on users and the world around us. In order to make that claim with integrity, we need to define impact and attempt to measure it. The information we gather through both quantitative and qualitative measures can support our decisions about how and what to develop.
There isn’t a single measurement of impact we can apply across industries or product types. In fact, there are many different frameworks and approaches to evaluating impact. Attempting to measure something like social benefit or even greenhouse gas emissions reductions can get very complicated very quickly.
One simple strategy is to identify potentially relevant indicators of impact for your product and to begin to think about what kinds of questions you can ask throughout the product lifecycle to introduce an impact focus into your decision making. For example, when you’re identifying your users, consider asking questions about how your product will affect people who are differently abled, come from different cultural backgrounds, have different access to resources, etc. In addition, ask yourself who the relevant non-users are (family members, co-workers, friends, etc.) and consider the range of impacts your product’s use might have on those people and on the environment.
Asking good questions and collecting information that can help you answer those questions is a great way to start. Here are just a few examples of standard UX metrics that could be tailored to help provide useful data on a product’s impact.
User Engagement: Measuring user engagement can provide insight into how often users interact with a product. You can measure this through metrics such as session pathing, time on tasks, and click-through rates. You can also use social listening methods to understand sentiment. If you are interested in evaluating your product’s impact on certain groups, you can tailor these metrics to understand engagement better for those groups. For example, if you want to evaluate how well you are engaging underserved communities in a certain geographic area, you can tailor your evaluation of pathing and other metrics to assess engagement in those communities, as long as they consent to sharing their location information with you.
Conversion Rates: Measuring conversion rates can help determine how effective a product is at achieving its goals. This usually involves counting the number of users who complete a specific action, such as signing up for a service, making a purchase, or tracking an action (e.g., booking a ride in a ride sharing app). If some sort of conversion is required for your product to have its intended positive impact, tracking conversion can help determine if it's working. For example, if a product supports users to make more efficient decisions about their transportation use by providing real time information, tracking conversion rates on users that book recommended transportation could help understand how much of a positive impact the product is having.
Interface Design Measures: Measuring the impact of a product’s interface on the environment will depend on many factors. You can track energy consumption or numerous other environmental metrics that might best reflect the goals of your product. Designers can get really creative with low-data use components such as icons, illustrations, shapes, typeface selection, forms, and colors. Being judicious about the use of images and applying optimization, scaling and compression can also be useful tools for reducing the negative impact of digital products.
There is no one size fits all answer to what impact is or how it is measured. Building consensus with your product and leadership team is critical to create a consistent and meaningful definition. By considering relevant metrics, product owners can better understand the impact of their digital products and make data-driven decisions to improve performance. You can also use impact information to tell the story of your product’s positive impact to funders, partners, your users, and their communities.
At Allelo, we help climate tech companies expand their positive impact. We are the designers who work with impact assessment and metrics experts to deliver incredibly impactful designs.