All videos
0:00 / 0:00
applications

How Morningstar Stopped Flying Blind on Production Agents with LangSmith

LangChain27 August 2026Watch on YouTube

Part of series

Ep. 12 · LangSmith Engine

LangChain's LangSmith Engine automatiseert het verbeteren van AI-agents door productie-traces om te zetten in geheugen en fixes.

View the series

What you'll learn

  • Morningstar's AI Accelerator team operates three AI products: a live MCP server, agentic AI assistants in Morningstar Direct and the Advisory Suite
  • Before LangSmith, debugging production agents was limited to Splunk queries, leading to slow problem resolution
  • LangSmith provides a shared observability language for tracing, prompt collaboration, and measuring time to resolution
  • The prompt playground in LangSmith turns prompting into a team effort and improves collaboration
  • All of the team's metrics have improved thanks to better visibility and faster debugging processes

Frequently asked questions

What are the three flagship AI products of Morningstar's AI Accelerator team?
The team builds a live Morningstar MCP server, agentic AI assistants in Morningstar Direct, and agentic AI assistants in the Advisory Suite. These products help with tasks like surfacing ETF holdings, data visualization, and pulling analyst research.
How did LangSmith change Morningstar's debugging process?
Before LangSmith, the team relied on limited Splunk queries, making debugging slow and opaque. LangSmith provided a shared observability language for agent tracing and faster problem resolution.
What is the prompt playground in LangSmith and how does it help the team?
The prompt playground is a feature that turns prompting into a team effort instead of individual work. This improves collaboration and leads to better results when optimizing agents.
What improvements has Morningstar achieved by implementing LangSmith?
All metrics the team tracks have improved, including faster time to resolution and better visibility into production agents. This was achieved through shared observability and improved prompt collaboration.

Topics

production agentsobservabilitylangsmithagentic aidebugging

Description from the channel

Matt Trivett, Senior Software Engineer on Morningstar's AI Accelerator team, breaks down the AI products his team ships, including a live Morningstar MCP server and agentic AI assistants embedded in Morningstar Direct and the Advisory Suite. He explains how debugging used to mean flying blind through limited Splunk queries, and how LangSmith gave the team a shared language for tracing, prompt collaboration, and measuring time to resolution. A few months in, every metric his team tracks has improved. Chapters: 0:00 Morningstar's AI Accelerator team and its three flagship AI products 0:13 The Morningstar MCP server, live and connectable today 0:30 Surfacing ETF holdings, visualizing data, and pulling analyst research 0:44 Before LangSmith, visibility was cumbersome and debugging was extensive 0:58 What the team needed to see in every agent trace 1:05 Why debugging was anecdotal without a common observability language 1:16 New infrastructure the team no longer has to build themselves 1:23 The prompt playground turns prompting into a team effort 1:30 Faster time to resolution and metrics that keep improving