Learn Hermes AI Agent - Basic Observability & Logging

#Introduction

In episode 6 you gave Hermes the ability to act: web search, database query, API fetch, and file system tools are now connected to the agent. The more tools there are, the harder it becomes to answer one question: "why did the agent do that?" This episode answers it with observability — the ability to observe agent behavior from the inside.

Here is the roadmap for this episode: first, event logging for agent conversations; second, debugging action execution and tool calls; third, monitoring agent behavior and replaying logs.

#Why Agents Need Observability

An agent is not a deterministic program. With the same model and prompt, output can differ between runs. Add the layered perception-planning-action-reflection flow, and failures are rarely visible from the final output alone — they often happen in the middle of a tool call.

Without observability, debugging an agent is like guessing what is inside a black box: you only see the last message, without knowing what plan the model put together, which tool was called, and why that tool was chosen. Observability turns that black box into clear glass.

The basic principle is simple: every important step produces a structured event with enough context, and then those events are stored, searched, and replayed.

#Conversation Event Logging

Hermes publishes a structured event for every phase of the agent lifecycle. Each event contains session_id, timestamp, type, and a payload. The format is JSON so it is easy to process with other tools.

{
  "session_id": "ses_order-check",
  "timestamp": "2026-08-03T09:14:22.481Z",
  "type": "llm.request",
  "model": "gpt-4o-mini",
  "turn": 3,
  "input_tokens": 1240,
  "payload": {
    "prompt": "Periksa status order ORD-2026-0142"
  }
}

The main event types you will see often: turn.started and turn.completed as the bounds of one conversation cycle, llm.request and llm.response around model calls, and tool.invoked and tool.result for every tool call along with its arguments and results.

An example of a tool.invoked event recording the arguments before the tool is executed:

{
  "session_id": "ses_order-check",
  "timestamp": "2026-08-03T09:14:22.689Z",
  "type": "tool.invoked",
  "tool": "db.query",
  "turn": 3,
  "args": {
    "sql": "SELECT status FROM orders WHERE id = 'ORD-2026-0142'"
  }
}

#Reading and Filtering Logs

All events can be viewed with the hermes logs command. But conversation logs grow long quickly; that is why Hermes provides filters.

hermes logs --follow --session ses_order-check

hermes logs --filter type=tool

hermes logs --json --since 2h

--follow surfaces new events in real time, --filter narrows by event type, and --json emits the log as JSON lines to be piped into jq or a log collector. For day-to-day work, the combination of --filter with --session is the most used.

#Debugging Action Execution & Tool Calls

When an agent behaves strangely, the first questions are: which tool was called, with what arguments, and what was the result? hermes debug runs the agent with full tracing and shows the flow in the terminal.

hermes debug agents/support.yml --prompt "Cek status ORD-2026-0142"

This mode shows the plan the model put together, every tool called with its arguments, the results returned, and the messages sent to the model on the next step. From here you can directly see whether the agent called the wrong tool, sent wrong arguments, or misread a result.

The most common debug pattern: the agent sends wrong arguments to db.query, the SQL result is an error, and then the agent misinterprets it on the next turn. In the trace, all of that appears as a chain of tool.invoked → tool.result with an error status.

A concrete example: the agent was asked to check an order status, but the argument sent to db.query used the column name order_id even though the table schema is orders.id. The trace shows a tool.invoked with the wrong argument, a tool.result containing a SQL error, and the next turn repeating the same mistake. The fix is directly visible in the trace: adjust the column name in the tool definition, rather than guessing at the prompt.

#Monitoring Behavior and Replaying Logs

Observability is not only for reactive debugging. With stored logs, you can monitor agent behavior continuously: how long a turn takes, which tools are called most often, and how many turns end in error.

To analyze past incidents, Hermes has a replay command that re-runs a session from the log without calling the model again. This is very useful for tracing "why did the agent answer like this last week" with no token cost.

hermes replay ses_order-check --render

hermes replay reads the entire session event stream and displays it as a complete conversation — including the tool calls inserted between messages. The result can be exported to a Markdown file to share with the team.

#Conclusion

Observability turns agent debugging from guessing into observing. You can now record every conversation event, filter logs by type and session, debug action execution with traces, and replay sessions for further analysis.

Key takeaways:

• Every lifecycle phase produces a structured JSON event with session_id and type.
• hermes logs with filters makes it easy to trace a specific event across thousands of log lines.
• hermes debug shows the plan, tool calls, and their results in full.
• Secret redaction in logs must always stay on, especially in production.
• hermes replay re-runs a session with no token cost for analysis.

In episode 8 we move into Memory & State Management — how Hermes stores short-term and long-term memory, strategies for storing, retrieving, and pruning memory, and limits on token usage and state size. See you there!