Articles

The article from LangChain's blog details five crucial patterns for evaluating "deep agents," which are complex, stateful AI applications. It highlights that traditional LLM evaluation methods are often insufficient due to agents' dynamic nature and the need for specific, context-dependent success criteria per test case. To overcome these limitations and ensure robust testing, the article outlines five crucial patterns. The first emphasizes bespoke, code-based test logic for each datapoint, allowing specific assertions against an agent's trajectory, final response, and internal state. Secondly, it advocates for single-step evaluations as an efficient way to validate immediate decision-making and tool calls, akin to unit tests. Thirdly, full agent turns are presented as essential for providing a comprehensive view of end-to-end actions, useful for evaluating overall trajectory, final responses, and generated artifacts. Fourthly, multi-turn evaluations simulate realistic user interactions but require conditional logic to manage agent deviations and ensure test consistency. Finally, the importance of clean, reproducible test environments and mocking external API requests is stressed to ensure reliable and efficient evaluations. The article positions LangSmith's testing integrations as a flexible framework to implement these patterns, offering practical guidance for AI developers.

The article presents a robust evaluation workflow for Deep Agents, focusing on systematically optimizing their performance. It begins by outlining the Deep Agent architecture, which includes detailed system prompts, planning tools, filesystem management, and subagents. The core problem addressed is how to rigorously measure and improve agent performance. The solution involves three key steps: first, using Harbor for sandboxed, scalable, and isolated agent execution to ensure reproducibility and safety; second, leveraging Terminal Bench 2.0 as a comprehensive benchmark dataset with automated verification for diverse agent tasks; and third, integrating LangSmith for automatic tracing, detailed observability, and reward feedback, enabling developers to understand agent actions and identify failure patterns. A concrete example demonstrates how analyzing traces can lead to optimizations like reducing environment setup latency. The article emphasizes a data-driven improvement cycle, moving from building and running agents to analyzing performance and iterating on improvements, ultimately enabling systematic agent development.

This article announces the public beta of LangSmith Agent Builder, a no-code platform designed to democratize the creation and deployment of production-ready AI agents. It clearly differentiates agents from traditional, fixed-path workflows, emphasizing agents' superior ability to dynamically determine steps, iteratively call tools until tasks are complete, and continuously improve through user feedback and memory. Agent Builder itself functions as an agent, leveraging best practices from millions of developers to guide users via a chat interface from an initial idea to a fully deployed agent. The beta release introduces significant enhancements, including 'bring your own tools' for integrating external APIs, 'workspace agents' for seamless team collaboration and cloning, multi-model support (OpenAI, Anthropic), programmatic invocation via API for embedding agents into existing systems, and a streamlined UI within LangSmith. The platform aims to resolve common AI adoption challenges by offering an intuitive agent creation experience for all employees, while simultaneously providing technical teams with robust governance over tools. Practical use cases are extensively detailed, spanning role-specific research (e.g., sales, marketing, recruiting), transforming ambient information into tracked projects (e.g., bug tickets, customer support), and developing time-saving communication assistants (e.g., email, calendar, Slack management).

The article introduces DeepAgents CLI, a model-agnostic, Python-based coding agent designed for interactive terminal use, featuring shell execution, filesystem tools, and memory. It addresses the critical challenge of evaluating coding agents in isolated environments due to their interactive nature and potential for side effects. To solve this, the authors leverage Harbor, a framework enabling scalable, containerized agent evaluations with automatic test execution and reward scoring across various sandbox providers. They specifically developed `deepagents-harbor` to integrate DeepAgents with Harbor. The evaluation was conducted on Terminal Bench 2.0, a benchmark comprising 89 tasks across domains like software engineering, biology, and security, designed to measure agent proficiency in terminal environments. DeepAgents CLI, powered by Claude Sonnet 4.5, achieved an average score of 42.65%, establishing it as a competitive baseline. The article concludes by highlighting future work on systematic trace analysis for performance optimization.