How do we debug AI agents like Goose? In the Goosing Around stream series, host Rizel Scarlett invited Marc Klingen, Co-Founder at Langfuse, and Alice Hau, Machine Learning Engineer at Block, to demo how Langfuse enables observability into Goose's actions, letting you trace LLM behavior and catch errors.

What is Langfuse

Langfuse is an open source observability platform specifically designed for LLM-powered apps. Mark revealed during stream that Langfuse wasn't originally an observability platform, it was born from early attempts to build an AI agent like Goose.

While they were limited by the available models at the time, especially with multi-file edits, the team discovered the tooling they had built for debugging and monitoring their agent was more valuable to them than their agent.

How Langfuse Works With Goose

Since traditional observability tools don't quite cut it when it comes to AI agents. Langfuse introduces 3 core concepts to make Goose's behavior more observable, and create logs that are easier to parse:

Traces

Each interaction with Goose creates a trace to capture the full story of what happened. These traces include key information, from the initial prompt and user messages to tool calls and their responses. They also store valuable metadata about model outputs and timing information, giving developers a complete picture of each interaction.

Timeline View

The timeline view takes these complex interactions and transforms it into a digestible format. Developers can see parallel task execution in real-time, understand the dependencies between different actions, and measure the actual duration of each opersation. This can be super helpful when debugging a complex sequence of actions taken by Goose, or to help optimize performance.

Structured Data

Alice explained, "Goose sessions can be really long... we have log files, but you'll just see a massive log of JSON."

Rather than rangle raw JSON log, Langfuse helps organize this data to help make navigating longer sessions and their data more straightforward. This approach can help developers easily analyze tool usage patterns, monitor token consumption, and quickly identify any performance bottlenecks and where they may happen.

With this integration, you can instead better understand the sequence of actions taken by Goose, and analyze track token usage and model behavior across LLMs.

Practical Benefits

The observability the Goose and Langfuse integration brings is great for anyone who wants clear insight into what Goose is doing behind the scenes. Alice and Marc discussed the different ways this integration can help you debug faster.

Developers can dive deeper into detailed session logs and identify the root cause to a reported issue and ensure Goose is operating as efficiently as possible. Like checking why certain commands may not be working as expected, or seeing exactly how Goose is processing information for a given task with a specific LLM.

As developers focus on operational efficiency, researchers can use the analytical capabilities of this integration to better understand which models best suit their needs. Through comprehensive model evaluations, they can analyze how different models handle tool calling, understand decision-making patterns across LLMs, and establish a systematic approach to understanding and improving AI systems.

The Future of AI Observability

These powerful debugging and analysis capabilities are only the beginning. This integration between Goose and Langfuse represents a significant step forward in making AI agents as transparent and debuggable as traditional code.

To keep up with the exciting developments as they release, you can check out both of the Goose and Langfuse repositories on GitHub.

You can also watch the livestream discussing the Goose and Langfuse integration, and follow the tutorial showing you how to integrate Langfuse with Goose.

Also, be sure to subscribe to our events calendar to catch upcoming events.