Tracely
Production failures become regression tests for AI agents
AILive
Visit site ↗In the record since 3d ago
- Price
- $49 /month · yes
- Free tier
- free forever · yes
- Kind
- api · cli
- Open source
- yes
- AI
- core to the product
“Tracely — LLM Observability & AI Agent Evaluation in CI”its own headline
⚿ 8 contact fields stay closed. Everything it says about itself is open; how to reach it needs an account. Open them →
What it costs3
“$49 /month”
Price$49 /monthMonthly
“Free to self-host.”
Free plan
“$0 forever”
Free plan typefree forever
How it runs7
“Tracely — LLM Observability & AI Agent Evaluation in CI”
AI rolecore to the product
“…tracely.init() switches on auto-instrumentation for whatever your agent already uses — OpenAI, Anthropic, LangChain / LangGraph, LiteLLM — and stamps agent, conversation and turn onto every span.”
API
“Bring your own model key. LLM judges run on your OpenRouter key, scoped to your workspace and encrypted at rest”
Bring your own key
“…tracely.init() switches on auto-instrumentation for whatever your agent already uses”
Kindapi
“…tracely gate exits non-zero, posts the commit status and upserts a PR comment.”
Kindcli
“The whole product is MIT-licensed — API, worker, UI, evaluators, the CI gate.”
Open source
“Self-host The entire product, MIT-licensed, on your own infrastructure.”
Self-hostable
What it offers as proof2
“20k traces / month”
Number claimed20k traces / month
“Start free”
Instant signup
Who it says it is for1
“For teams gating real releases on real production failures.”
For teams
What it names1
“OpenAI Anthropic Google Gemini LangChain LangGraph LiteLLM CrewAI Mistral any OTLP source”
Integrates withAnthropic · CrewAI · Google Gemini · LangChain · LangGraph · LiteLLM · OpenAINamesAnthropic · CrewAI · Google Gemini · LangChain · LangGraph · LiteLLM · OpenAI
What it calls itself2
“Tracely — LLM Observability & AI Agent Evaluation in CI”
Calls itselfLLM observability
“Trace-native CI/CD for AI agents”
Calls itselftrace-native CI/CD
How to reach it1
“…we never bill you a markup on inference, and we never use a shared key”
Voice“we”
About
Tracely grades every agent trace as it lands, clusters failures into issues, freezes bad runs into hermetic replayable test cases — and blocks the PR that would ship them again. The trace is the test: no hand-authored datasets. Open source, MIT.
What it works with6 NAMES
Integrates with
Names the same things
Charges the same way
