Hamel Husain - Building LLM Apps in Production

Name: Hamel Husain - Building LLM Apps in Production
Uploaded: 2026-06-17T16:33:48.847Z
Duration: 58 min 19 s
Channel: Aleksa Gordić - The AI Epiphany
Description: Hamel Husain discusses his experience as an LLM consultant and shares insights into common problems ML startups face when building LLM applications in production, including data management, testing, and optimization.

Aleksa Gordić - The AI Epiphany17 June 2026Watch on YouTube

Description

Become a Patreon: https://www.patreon.com/theaiepiphany 👨‍👩‍👧‍👦 Join our Discord community: https://discord.gg/peBrCpheKE Hamel joined us to talk about his work as a consultant in the LLM space and discuss the common problems a lot of ML startups he's worked with share. ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ website: https://hamel.dev/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ⌚️ Timetable: 00:00 - 01:11 Intro 01:11 - 02:52 Hyperstack GPUs (sponsored) 02:52 - 14:00 Hamel's story 14:00 - 21:30 Consulting in the LLM space 21:30 - 29:25 Problems companies face - Looking at data & collect traces 29:25 - 33:30 Write unit tests :) 33:30 - 36:45 How do you find interesting companies? 36:45 - 50:25 Common mistakes 50:25 - 58:18 How does consulting pay? ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💰 SPONSOR The AI Epiphany - https://www.patreon.com/theaiepiphany One-time donation - https://www.paypal.com/paypalme/theaiepiphany Huge thank you to these AI Epiphany patreons: Eli Mahler Petar Veličković ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💼 LinkedIn - https://www.linkedin.com/in/aleksagordic/ 🐦 Twitter - https://twitter.com/gordic_aleksa 👨‍👩‍👧‍👦 Discord - https://discord.gg/peBrCpheKE 📺 YouTube - https://www.youtube.com/c/TheAIEpiphany/ 📚 Medium - https://gordicaleksa.medium.com/ 💻 GitHub - https://github.com/gordicaleksa 📢 AI Newsletter - https://aiepiphany.substack.com/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #llms #llmapps #evals #data

What you'll learn

Data management and trace collection are critical for LLM applications in production, enabling you to diagnose and fix issues effectively.
Writing unit tests for LLM applications helps catch errors early and ensures the quality of your system.
Many ML startups make similar mistakes when building LLM apps, which can be prevented by following the right approach and best practices.
As an LLM consultant, you need specialized knowledge and experience to help startups build their production systems correctly.
Testing and evaluating LLM systems requires preparation and attention to detail to deliver reliable applications.

Frequently asked questions

What are common problems startups face when building LLM applications?

Startups often struggle with data management, writing tests, and optimizing their LLM systems in production. Hamel observes that many companies make similar mistakes that can be prevented with proper guidance and best practices.

How does tracing and data collection help with debugging LLM systems?

By collecting traces, you can see what's happening in your LLM application, allowing you to diagnose and improve problems more effectively. This is essential for maintaining reliable production systems.

Why is writing unit tests important for LLM applications?

Unit tests help you catch errors early and ensure your LLM application works reliably. This prevents issues from reaching production.

What makes someone qualified as an LLM consultant?

A successful LLM consultant has specialized knowledge and practical experience building LLM applications in production, giving them insight into common challenges and how to solve them.

Topics

llm applications production systems ml consulting best practices evals