Engineering Manager / Technical Lead
A topic hub collecting every article tagged Observability. Use it to explore related posts and follow this theme across the site.
8 articles
A practical guide for implementing LangWatch evaluations in Agno and AgentOS systems, from first traces and batch experiments to structured-output scoring, production monitors, and background evaluation hooks.
LLM-as-a-judge can be one of the most useful patterns in agent evaluation, but only if you understand where it breaks: order bias, self-preference, verbosity bias, weak judges, and evidence-free scoring. This guide explains the pattern, the common traps, and the fixes that make it practical.
Why production agent evaluation is moving beyond output-only checks, how trace-aware grading complements scenario testing, and how LangWatch, LangSmith, and Langfuse compare.
A practical, multi-level guide to building an agent app that researches topics, drafts markdown articles, writes them into an Astro site, monitors reliability with LangWatch, and opens a GitHub PR automatically using Agno, AgentOS, and OpenRouter.
Unit tests tell you if your code works. Scenario tests tell you if your agent behaves. But how do you measure quality across hundreds of examples and track it over time? LangWatch evaluations fill that gap.
A deep dive into LangWatch MCP Server — from basic setup to legendary architectural patterns for test-driven AI development and production inference monitoring.
A comprehensive, multi-level deep dive into testing and quality assurance for modern AI-powered systems — from static web apps to agentic pipelines and MCP servers — using LiteLLM, LangWatch, Agno, AgentOS, and the Claude Agent SDK.
A deep comparison of LiteLLM and LangWatch — two essential tools in the modern LLM stack. Discover where they overlap, where they complement each other, and how to use them together for production-grade AI systems.
Quick find
Search by topic, title, framework, or pattern.