AI has made polished proposals a worthless signal. Any vendor can now generate a compelling response, complete documentation, and a convincing demo environment with a fraction of the effort previously required. The gap between what a vendor can present and what a vendor can deliver is widening, and evaluation methods built on proposal scoring, oral presentations, and reference checks are increasingly poor predictors of performance.
The mitigation is structural: stop placing weight on pre-award evaluation and give multiple vendors a low-risk opportunity to demonstrate capability through actual work, then quickly remove those who do not deliver. Let performance be the filter in the context of end-to-end outcomes, not paper.
A polished demo in a vendor-controlled environment is a proposal in disguise: everything works in a vacuum. Capability must be demonstrated in the state's actual environment (its data, integrations, and policy quirks) with the vendor iterating on real feedback before the state commits, not after the contract makes iteration a change order.
For the SMC process specifically, the same logic applies to compliance validation: automated testing harnesses that vendors run against their working systems tell CMS more than any attestation document. On KPIs, we recommend measuring the vendor ecosystem on delivered value relative to cost, tracked as trendlines, rather than certification counts or milestone completion.