A per-model, over-time quality monitor for AI API relay and reseller endpoints
Developers who buy cheap LLM access through relays and aggregators get silently burned when a provider swaps models, throttles throughput, or degrades output quality. A scheduled probe suite that measures correctness, latency, and real cost per model per relay, keeps history, and alerts on regressions would let buyers choose and switch providers with evidence instead of vibes.