Not Diamond combines pretrained and custom model routing with cross-model prompt optimization, but its value depends on representative evaluation data and measured end-to-end savings.
Direct verdict
Not Diamond earns a pilot for teams with meaningful inference volume, varied requests, and enough evaluation discipline to train and monitor routing. Keep a deterministic fallback, test on held-out traffic, separate provider savings from router charges, and refuse deployment if a cheaper average hides unacceptable failures in a critical segment.
What to verify
Build a stratified set of at least 500 recent production requests with blinded human scores and hard constraints for safety, format, tool use, latency, and cost. Compare a fixed strong model, a fixed economical model, pretrained routing, and a custom router on a held-out set. Measure quality by segment, catastrophic misses, router overhead, provider failures, fallback behavior, total spend, drift after model updates, and the staff effort needed to maintain labels and policies.
Personal Recommendation
Not Diamond earns a pilot for teams with meaningful inference volume, varied requests, and enough evaluation discipline to train and monitor routing. Keep a deterministic fallback, test on held-out traffic, separate provider savings from router charges, and refuse deployment if a cheaper average hides unacceptable failures in a critical segment.