The Challenge

The executive team was convinced they needed to rewrite a complex validation component (think rules engine) for a data transformation system that would take a full development year and cost approximately $1.2M to modernize. The pressure was on to start building immediately.

Something didn’t feel right. A pattern was starting to develop. We’d just spent over a year on a new offering, brought on additional contractors to supplement the team, and contracts were not getting signed. What it did feel like was a solution without a defined and validated problem. The assumptions were plenty:

• that modernization was neeeded to compete with slicker interfaces
• that users were struggling with invalid data
• that it generated a large percentage of reported incidents

Which incidents? What exactly are the customer’s pain points? What’s the vision for this very mature application? Where’s the roadmap that illustrates which outcomes are directly linked to doing this work first?

No one could clearly articulate. The evidence was missing.

The Approach

Instead of jumping straight into development, I pushed hard for and executed a one-month discovery phase:

• Wrote plan based on Marty Cagan’s The Product Discovery Plan
• Implemented an in-app pop-up poll (Pendo®)
• Analyzed existing product usage data
• Looked at the last 12 months of support incidents
• Conducted interviews with 5 users from top customers and mapped their experiences
• Built trust with Engineers, providing them the time they’d needed to address a few minor technical opportunities (technical debt) that were consistently generating other support tickets

Borrowed Marty Cagan’s Product Discovery Plan and shared it with all stakeholders, including everyone early and up front.

Pendo® Poll results demonstrating user perception of how many attempts it takes to validate complex regulatory data. For most users, 1-5 attempts were needed, yet no one commented on this being a problem considering the complexity of all data. Sentiment did not point to validation.

The Discovery

What we found changed everything:

• The real problem wasn’t what leadership thought it was
• Surveys and interviews yielded no mentions of this component
• The problem was data integrity. Redundant data stored in the customers’ enterprise system and ingested into ours created a problem: system of record (source of truth)
• Our system had no way of knowing about any changes
• “I’d rather the data be invalid if it still matches what’s in our system. I can’t trust your system.” - This was an alarming quote from our first interview.
• Remaining interviews resulted in similar sentiment.
• Engineers were grateful for preventing another tragic waste.

The Outcome

By investing one month in proper discovery:

• Saved $1.2M in unnecessary development costs
• Were able to pivot and focus on the larger data integrity issue customers really cared about
• Also learned that our top customer had tightly coupled operating procedures - changes could have been devastating.
• Paradigm shift - big ideas are very risky assumptions if they’re not backed by evidence. Team moved from being order-takers to product thinkers.

Key Takeaway

Evidence-based discovery isn’t about slowing down—it’s about making sure you’re running in the right direction.

Sometimes the most valuable thing a product leader can do is say “wait” and ask “why?” before the team invests months building the wrong thing.

Healthy teams debate. This isn’t questioning someone’s expertise. It’s asking for the evidence that reduces the risk of an assumption.