Learn Hermes AI Agent - Tooling & External Integrations

#Introduction

In episode 5 you designed the system prompt, task instructions, and response formatting through the agent profile in agents/. So far Hermes is only good at rhetoric — it cannot yet touch the real world. This episode is the point where your agent "comes alive": connected to an LLM provider and equipped with tools to take action.

The roadmap for this episode has three parts: first, connecting OpenAI, Azure OpenAI, or another provider; second, integrating web search, database query, API fetch, and file system tools; third, covering the safety considerations you must understand every time the agent calls a tool.

#Choosing and Connecting an LLM Provider

Hermes is designed to be provider-agnostic: one same interface is used for any model. Provider configuration lives in hermes.config.ts, and Hermes reads credentials from environment variables that are automatically loaded from .env.

export default defineHermesConfig({
  model: {
    provider: "openai",
    model: "gpt-4o-mini",
    apiKey: process.env.OPENAI_API_KEY,
    temperature: 0.4,
    maxTokens: 4096,
  },
  tools: {
    registry: true,
    directory: "./tools",
  },
});

For Azure OpenAI, replace the model block to use Azure's endpoint, API version, and deployment name — because Azure has its own deployment structure. The same pattern applies to other providers such as Anthropic or open-source models via an OpenAI-compatible endpoint.

model:
  provider: azure
  model: gpt-4o
  azureEndpoint: "https://belajar-hermes.openai.azure.com/"
  apiVersion: "2025-08-01-preview"
  deployment: hermes-deployment

The key to all of this: .env must contain OPENAI_API_KEY or AZURE_OPENAI_API_KEY, and that file must be in .gitignore. To check which provider is active and whether its credentials are valid, run hermes doctor.

#Registering Tools in the Agent Profile

Tools in Hermes are declared in the agent profile, alongside the persona and capabilities you created in episode 4. Every tool has a name, a description, and a permission level that determines when the tool may be executed.

name: support-agent
persona: "Asisten support produk yang teliti"
goals:
  - "Menjawab pertanyaan berdasarkan dokumentasi"
capabilities:
  - tools
  - browsing
tools:
  - name: web.search
    permission: auto
  - name: web.fetch
    permission: confirm
  - name: db.query
    readOnly: true
    permission: auto
  - name: fs.read
    allow: ["./docs", "./kb"]
    permission: auto
  - name: http.post
    permission: denied

Notice the permission levels: auto means the controller executes the tool immediately without asking, confirm means the agent asks for your approval, and denied forbids it entirely. It is this combination that makes the agent powerful yet controlled.

#Web Search & API Fetch Integration

Web tools give the agent access to the latest information that is not in the model's training data. web.search fetches a list of results from a search engine, and then web.fetch reads the content of a specific page to be summarized.

The two work as a pair: the controller has the agent search first, then the agent picks the most relevant link to fetch. Without web.fetch, search results only contain titles and short descriptions. This is a concrete example of tool chaining, which you will study in more depth in episode 10.

For API integration, the http.get and http.post tools call external REST endpoints. Their results are automatically parsed as JSON and injected into the agent context as fresh data.

{
  "status": 200,
  "data": {
    "order_id": "ORD-2026-0142",
    "status": "shipped",
    "tracking": "JNE-8821"
  }
}

If the endpoint requires authentication, register the credentials separately in a secret store — never put them directly in the agent profile.

#Database Query & File System

The db.query tool executes SQL against a configured database. Safety first: set readOnly: true so the agent can only run SELECT statements, not INSERT, UPDATE, or DELETE. Hermes rejects write statements while this flag is active.

The file system is also restricted with an allow-list of paths. The agent may read ./docs and ./kb, but cannot touch files outside those. Also restrict the fs.write tool — if it is not truly needed, set its permission to denied.

hermes run agents/support.yml --session order-check

If all the configuration is correct, you can ask the agent to check an order status and its answer will reference real data from the database.

#Safety Considerations on Tool Invocation

Giving an agent tools means giving it side effects. There are four principles you must hold on to:

• Least privilege — grant the minimal permission needed for the task at hand. Tools that are not used are denied outright.
• Allow-list, not deny-list — the web domains, database tables, and file paths that may be accessed are listed explicitly. Anything not listed is automatically blocked.
• Human in the loop — risky operations such as sending email or processing payments are set to confirm permission so human approval is required.
• Be wary of prompt injection — the agent can be manipulated through the content of the web pages or documents it reads. Add an instruction in the system prompt that tool content is data, not commands.

In episode 7 we will see how all of these tool calls are recorded — complete with their arguments and results — so you can trace and replay agent behavior.

#Conclusion

Episode 6 made your agent no longer blind and mute: the LLM provider is connected through hermes.config.ts, and the web search, database query, API fetch, and file system tools are registered in the profile with their own permission levels. Most importantly, you now have clear safety principles before the agent is given the power to execute.

Key takeaways:

• LLM providers are configured in hermes.config.ts; credentials always come through environment variables.
• Tools are declared in the agent profile together with the auto, confirm, or denied permission levels.
• web.search and web.fetch work as a pair to read current information.
• Database and file system access must be restricted with readOnly and path allow-lists.
• Safety starts with least privilege and vigilance against prompt injection.

In the next episode 7 we move into Basic Observability & Logging — event logging for conversations, debugging action execution and tool calls, and replaying logs to monitor agent behavior. See you there!