A workable test: the metric should reflect real value delivered, predict long-term retention and revenue, be influenceable by your team's work, and be understandable to everyone who hears it. Sean Ellis, who popularized the term, framed it as the metric that best captures the core value you deliver to customers. Start from your product's core action — the thing a customer does when they get what they came for — and measure that, ideally counting only meaningful instances (an active team, not a registered email).
The most common failure is choosing a vanity metric: total signups, page views, cumulative downloads, app-store rank. These rise on their own, feel good in a board deck, and tell you nothing about whether customers succeed — they can climb while engagement collapses. Two more traps: a metric so broad it has no clear owner and no one can move it, and gaming, where teams inflate the number without creating value (counting empty sessions, or signups that never activate). Pair the NSM with a small set of guardrail metrics — retention, activation, support load — so optimizing the headline number cannot quietly break something else.