Carton
Provides a unified API to load and run ML models from any framework, wrapping models without conversion and automatically managing the required runtime.
Carton is an open-source API that lets developers run machine learning models from any programming language without being tied to a specific ML framework. It solves the problem of framework lock-in and complex model conversion by wrapping models in a zip file with metadata, leaving the original model untouched. The product targets software developers and engineering teams who need to integrate ML inference into their applications across multiple languages. It is delivered as an infrastructure API with bindings for many languages, using a Rust core for low overhead and automatic runner management.
Key features
- One open-source API for all frameworks
- Pack models without modification
- Automatic runner fetching
- Framework-agnostic inference code
- Bindings for multiple languages
- Optimized async Rust core
- Low overhead per inference call
- Supports x86_64 and aarch64 platforms
- No social media activity within the last 30 days
- Community
- Docs
- Software developers
- Engineering teams
- Startups