Permitech

An AI observability and evaluation platform that evaluates, monitors and protects generative AI applications and agents, addressing hallucinations, failures and safety risks by turning offline evaluations into production guardrails. It sells to AI developers, data teams and engineering teams in enterprises building and operating generative AI systems at scale. It is positioned around an eval-to-guardrail lifecycle that distills LLM-as-judge evaluators into low-latency, low-cost models to monitor traffic and govern agent actions, and is delivered as SaaS with virtual private cloud and on-premises options.

Key features

• RAG evals
• Agent evals
• Safety evals
• Security evals
• Custom evals
• Dataset building from synthetic and production data
• Expert annotation capture
• Auto-tuned metrics from live feedback
• Distill evals into Luna guardrail models
• Monitor 100% of traffic
• Insights engine for failure mode detection
• Prescribe fixes for agent behavior
• Trace and session observability
• Guardrail policies for agent actions
• Control tool access and escalation