Measuring the success of a product or its features is a crucial aspect of working on a product. It involves understanding the goals, setting clear success metrics, evaluating actions and metrics, and correlating them with user and business success.
Starting Strategy
To measure the success of a product or its features, we can utilize the following framework:
- Clarify and Ask Questions: Begin by seeking clarity on the goals and objectives of the product. Ask questions to understand the user and business needs better.
- Understand the Product and Set Goals: Gain a deep understanding of the product, its functionalities, and the intended user experience. Align with stakeholders to establish specific user and business objectives.
- Define Success Metrics: Identify the metrics that align with the user and business goals. Translate qualitative goals into measurable, trackable values.
- Test Hypothesis: Formulate hypotheses based on the defined success metrics. Create experiments or tests to validate these hypotheses.
- Correlate Behavior: Analyze user behavior and correlate it with the defined success metrics. Evaluate the impact of the product or feature on user actions and business outcomes.
GAME Framework
1. Understand the Goal (Goals).
Start by making sure you understand the product properly and agree with your stakeholder on specific user and business objectives.
A. User Goal
- How will our users benefit from the product? Identify the value proposition and the problem the product aims to solve for users.
- Who will use the product? Determine the target audience and user segments.
- How will our users interact with the product? Understand the user journey and the touchpoints where users engage with the product.
- What problems do our users want the product to solve for them? Identify the pain points and challenges users face.
- How will our users feel when they use the product? Explore the emotional impact the product should have on users.
- What is the vision for how this product will integrate in our users’ life? Envision the role the product will play in users' lives and the value it will bring.
B. Business Goal
- What are the tactical or strategic business benefits? Determine how the product contributes to business objectives, such as increasing revenue, decreasing costs, gaining a competitive edge, or entering new markets.
- What does the business look like if the product is successful? Envision the impact of the product’s success on the overall business.
it’s best if the user goal and business goal is aligned.
2. Follow the user journey or Funnel (Actions).
List the Actions that matter. This should start as a qualitative list. Don’t worry about the numbers or whether they are trackable yet.
Tip
Think about what it really means for a user to be get to the goal above.
Aim to list every relevant action.
prioritize your list of actions.
use the goals you established in the previous step to explain your thinking.
The AARRR framework (Acquisition, Activation, Retention, Revenue, Referral) can be helpful here.
3. Define the metrics (Metrics).
Turn each desired user action (qualitative) into a measurable, trackable value (quantitative). bring engineering and data teams in at this stage to vet your metrics and provide technical guidance on the feasibility of collecting/storing the desired data.
- try to define 1-2 success metrics.
- with at least 1 guardrail metrics.
Good metrics:
- Simple: Easy to understand and calculate, and people should be able to remember and discuss it easily.
- Clear: The definition is explicit and there is no ambiguity in interpretation.
- Actionable: The metric can be moved by improving products, and not easily gamed (making you feel like you are getting results even though it offers no insights into actual business health or growth).
Bad metrics:
- Irrelevance
- Impractical
- Needless Prioritization
Considerations:
- Direct vs. Proxy: Can the action be directly tracked or do we need a proxy metric? (e.g., clicks vs. likelihood)
- Individual vs. Aggregate: Metrics can be measured for the total user base, specific user groups, or per product/service.
- Magnitude vs. Ratio: Metrics can be measured in terms of total values, daily values, or per user.
- Intrinsic vs. Heuristic: Metrics can be objective or subjective, focusing on the essence or quality of user experience.
4. Evaluate (Evaluations).
- Evaluating Metrics: Monitor the data trends and stability of the metrics. Check if they align with expectations and provide meaningful insights over time.
- Evaluating Actions: Assess whether the selected actions reflect the product, user, or business goals. Ensure they tell a coherent story about user behavior.
- Evaluating Goals: Correlate the metrics with business or user success. Analyze if positive metric outcomes result in happier users or achieve the desired business outcomes.