When test outcomes became contextual
Tests are the operational center of compliance work in Vanta. Customers rely on them to understand risk, prioritize remediation, and defend audit readiness across multiple frameworks.
When Vanta introduced framework scoping, we knew it would fundamentally change the meaning of a test. A single test could now have multiple valid outcomes depending on framework scope. The existing UI wasn’t designed for that reality.
From one global status to multiple framework-specific test outcomes
Accuracy vs. trust
Framework scoping made test scopes more precise, but precision alone wasn’t enough. The existing UI already supported fast monitoring and remediation well. The challenge was introducing framework-specific outcomes without disrupting those workflows.
As test outcomes began to diverge by framework, we had to decide how much variation to surface, where to surface it, and how to preserve a single, trusted overall status. We had to extend the system in a way that preserved how customers already reasoned about compliance.
Failures were overstated in some cases
Key design decisions
I led design across a set of tightly coupled decisions that balanced correctness, usability, and delivery constraints.We preserved a single high-level test signal to support fast, day-to-day monitoring, even as outcomes diverged underneath. We intentionally used all monitored resources as a proxy for that signal, while making framework-specific outcomes explicit where audit defensibility mattered most.
To support different user modes, I designed a dedicated framework outcomes view, allowing audit preparation to be explicit without forcing every workflow to operate at audit-level detail.
Outcome + what it unlocked
The final system reduced overstated failure in framework-specific contexts while keeping day-to-day remediation behavior intact. Customers incorporated framework outcomes into daily workflows with no increase in support tickets or loss of momentum.
More importantly, this work laid the foundation for supporting larger, more complex customers managing many frameworks and unlocked future investments in enterprise-grade scoping and segmentation.
