How would you define and measure the success of a new feature — say, a 'Stories' feed added to a social app?
Success definition requires aligning a north-star metric to the feature's goal, pairing it with guardrail metrics that catch side-effects, and deciding on a measurement window before launch. For a Stories feed, adoption rate and daily story views per active user are reasonable primary signals, while core feed engagement and notification opt-out rate serve as guardrails.
How to think about it
Framework: goal → primary metric → guardrails → measurement window
Step 1 — Clarify the goal. Ask: is this feature meant to increase engagement, retention, monetisation, or acquisition? A Stories feed aimed at retention has different success criteria than one aimed at ad revenue.
Step 2 — Choose a north-star metric. Pick one metric that directly reflects value delivered to users and the business. For a retention play: % of DAU who view at least one Story per day (adoption rate). For an engagement play: stories viewed per DAU per day.
Step 3 — Add guardrail metrics. Guardrails prevent local optimisation that harms the product overall:
| Guardrail | Catches |
|---|---|
| Core feed scroll depth | Stories cannibalising the main feed |
| 7-day retention of non-Stories users | Feature drawing users away from stickier surfaces |
| Notification opt-out rate | Aggressive push notifications inflating views |
| App crash rate | Stability regressions |
Step 4 — Define the measurement window. Story-viewing may spike at launch due to novelty. Use a 2-week post-ramp window and check week-over-week trend, not just the day-1 number.
Worked example. Suppose after a 50/50 A/B test over 2 weeks: treatment DAU story-view rate = 34 %, control = 0 % (feature off). Core feed engagement dropped 4 % (guardrail violated). Correct action: do not ship until the feed-cannibalisation root cause is fixed, even though the north-star metric looks good.