Traceloop
Winter 2023AcquiredKnow what your LLM is doing — and why it’s failing
Know what your LLM is doing — and why it’s failing
If you only have a few minutes to spare, here’s what investors, operators, and founders should know about Traceloop (W23).
Traceloop built observability and evaluation infrastructure for teams putting large-language-model applications into production. Founded in 2023 and admitted to Y Combinator's Winter 2023 batch, the nine-person Tel Aviv company paired OpenLLMetry, its OpenTelemetry-based instrumentation project, with a commercial platform for tracing, quality checks, prompt experiments, and drift monitoring.[1]
Traceloop's outcome was an acquisition, not a shutdown. Open source gave the company reach across an unusually fragmented AI stack; the paid layer turned that telemetry into enterprise controls. ServiceNow bought the missing runtime-evidence layer for AI Control Tower, completing the acquisition by May 5, 2026.[2] The consideration, deal structure, precise closing date, and employee transition remain undisclosed.
Nir Gazit and Gal Kleinman came from the same professional orbit: both had worked at Fiverr, where Gazit became chief architect and Kleinman managed an R&D group. Gazit had also worked at Google.[1] Gazit's later account dates the work to 2022, when the pair were watching GPT-3 applications behave unpredictably while teams tracked prompt changes in spreadsheets.[3] The specific circumstances in which the founders met are not disclosed in the observed material beyond their shared Fiverr background.
Their initial bet was infrastructure before interface. OpenLLMetry extended OpenTelemetry so developers could instrument calls across model providers, agent frameworks, and vector databases, then export traces through standard OTLP plumbing rather than accept a proprietary sink.[4] That architecture was also a distribution decision. A developer could add tracing with a small SDK change and continue using destinations such as Datadog, Honeycomb, Sentry, Grafana, or ServiceNow Cloud Observability. The commercial product then moved up the stack, adding evaluation, prompt versioning, model comparison, and cost and performance monitoring.[3]
The evidence handoff contains founder-authored accounts but no verbatim passages observed from the full pages. This report therefore does not manufacture the two direct founder quotations requested by the canonical format. That gap matters less to the chronology than to motive: Gazit's retrospective supports the spreadsheet-and-unpredictability origin, but no independently observed founder interview tests how the founders chose observability over adjacent gateway or development-tool wedges.
OpenLLMetry was the collection layer. It added vendor-neutral semantic conventions and instrumentation around model calls, agent steps, retrieval operations, and related application work. Its public repository documents support across OpenAI, Anthropic, Bedrock, Gemini, Cohere, Groq, Mistral, and Vertex; frameworks including LangChain, LlamaIndex, CrewAI, LangGraph, LiteLLM, and MCP; and vector databases such as Pinecone, Qdrant, and Weaviate.[4] When observed, the repository showed about 7,100 stars, 978 forks, and 1,395 commits. Those are repository snapshots, not customer or revenue measures.
The user journey began with instrumentation. A developer initialized the SDK, emitted telemetry through OpenTelemetry-compatible infrastructure, and inspected the resulting traces. The paid platform added operational interpretation: faithfulness and relevance checks, custom evaluators, drift detection, prompt versioning and testing, model comparisons, and cost and latency monitoring.[3] This separated portable evidence collection from proprietary analysis.
That distinction proved strategically useful. Dynatrace published an OpenLLMetry integration guide, while IBM documented OpenLLMetry in its GenAI observability work.[7][8] These pages verify ecosystem integration, not paid Traceloop contracts. Gazit also named Cisco and IBM as users of the open-source project and Miro as a commercial customer experimenting at scale, but comparable first-party Cisco or Miro case studies were not observed.[3]
Read the complete post-mortem, the rebuild playbook, and the exact reasons Traceloop is still worth studying now.