Comet Panels

An open-source observability and evaluation platform for generative AI applications and agents that logs LLM traces, detects errors, runs evaluations, and turns findings into code fixes to improve reliability and reduce costs. It sells to developers, data teams, and engineering teams in companies building chatbots, RAG pipelines, and autonomous agents. It is delivered as a SaaS and self-hosted platform with open-source components, cloud and on-premises deployment options, and API integrations.

Key features

• LLM trace logging and visualization
• 60+ integrations and MCP server
• Diagnostics for error detection and grouping
• Ollie agent for recommended code fixes
• Test Suites with pass/fail scoring
• 40+ LLM-as-a-judge eval metrics
• Golden dataset creation and evaluation
• Production dashboards and alerts
• Cost Intelligence for Claude Code spend
• Prompt and agentic system optimization
• Experiment tracking and comparison
• Model registry and versioning
• Dataset management
• Production model monitoring