RouterFuel

An async proxy gateway that routes inference requests to 330+ language models across 11 direct provider integrations and OpenRouter as fallback, handling multiplexing, semantic caching, automatic failover and circuit-breaking to avoid manual retry logic and redundant provider calls. It is for developers and operations teams in business environments building applications on multiple model providers. It is delivered as a self-hosted Docker deployment with a Rust (Tokio/Axum) core under AGPL-3.0, using bring-your-own keys and zero telemetry sent off-cluster.

Key features

• Route to 330+ models via one API
• 11 direct provider integrations
• OpenRouter universal fallback
• One request format across providers
• Async Rust gateway on Tokio/Axum
• Multiplex concurrent inference requests
• Automatic failover and rerouting
• Circuit-breaking resiliency with probing
• Semantic caching with local embeddings
• Postgres + pgvector auto-provisioned
• LoopGuard runaway-agent detection
• SpendGuard per-client spend caps
• Self-hosted via Docker Compose
• BYOK-only key management
• Zero telemetry sent off-cluster