Hermes AI Agent

NousResearch Hermes Series — Open-Source LLMs for Agentic Workflows

NousResearch Open-Source LLM Tool Calling Function Calling ChatML Agentic AI

What is Hermes?

Hermes is a series of fine-tuned open-source LLMs developed by NousResearch. Built on top of base models (Llama 3, Mistral, Mixtral), Hermes is trained specifically for strong instruction following, tool/function calling, JSON output, and agentic multi-step reasoning — making it one of the most capable open models for building AI agents.

Core Idea: Hermes extends capable base models with fine-tuning data that teaches structured tool use, multi-turn reasoning, and role adherence — enabling production-grade agentic applications using open weights.
🛠️

Tool Use

Native function calling with structured JSON arguments

🧠

Reasoning

Chain-of-thought and ReAct multi-step agent loops

📋

Instruction

Superior instruction following with complex system prompts

🔣

JSON Mode

Reliable structured output for downstream pipelines

🔓

Open Weights

Fully open — run locally, privately, no API costs

🔄

Multi-Turn

Long context, coherent multi-turn conversations

Model Family Timeline

Hermes-2-Pro

Built on Mistral 7B. First Hermes with strong function calling. Still popular for lightweight agents.


NousResearch/Hermes-2-Pro-Mistral-7B

Hermes-2-Theta

Mistral 7B merge model. Combines multiple capabilities — good instruction following and tool use balance.


NousResearch/Hermes-2-Theta-Llama-3-8B

Hermes-3 (Llama 3.1 8B)

Major leap. Built on Llama 3.1 with 128k context. Best small Hermes for local agent deployments.


NousResearch/Hermes-3-Llama-3.1-8B

Hermes-3 (Llama 3.1 70B)

Flagship model. Competitive with GPT-4o on agent benchmarks. Excellent for production agentic use.


NousResearch/Hermes-3-Llama-3.1-70B

Hermes-3 (Llama 3.1 405B)

Largest open model. State-of-the-art open weights, matches frontier closed models on many evals.


NousResearch/Hermes-3-Llama-3.1-405B

Hermes-2-Pro (Llama-3 8B)

Updated Pro line on Llama 3 base. Strong structured output and tool calling in a compact package.


NousResearch/Hermes-2-Pro-Llama-3-8B

Architecture & Training Approach

Hermes models are fine-tunes of base models using curated datasets. NousResearch collects and synthesizes training data focused on agentic behavior, structured output, and multi-step reasoning.

Base Model (Llama 3.1 / Mistral / Mixtral) │ │ Fine-tuning on curated datasets ├─── Instruction Following (complex system prompts, role adherence) ├─── Tool / Function Calling (ChatML format, JSON schema tools) ├─── Structured Output (reliable JSON, XML, YAML generation) ├─── ReAct Reasoning (think → act → observe loops) └─── Multi-Turn Coherence (long context, 128k tokens) │ ▼ Hermes Model (open weights, gguf / safetensors) │ ├─── Run locally → Ollama, LM Studio, llama.cpp ├─── Self-hosted → vLLM, Text Generation Inference, Aphrodite └─── API services → Together AI, Fireworks, Groq, Perplexity
ChatML Token Format: Hermes uses <|im_start|> / <|im_end|> special tokens to delimit roles, giving it precise control over multi-turn conversations and tool injection.
Why Fine-Tune? Base models like Llama 3.1 can generate text but struggle with consistent JSON schemas and multi-step tool use. Fine-tuning on agent trajectories teaches reliable, structured behavior.

ChatML Format

Hermes uses the ChatML conversation format. Every message is wrapped with special tokens indicating role boundaries. This is critical for agent loops where you inject tool results back into the context.

# Basic ChatML structure <|im_start|>system You are a helpful AI assistant with access to tools. <|im_end|> <|im_start|>user What is the weather in Tokyo today? <|im_end|> <|im_start|>assistant <tool_call> {"name": "get_weather", "arguments": {"city": "Tokyo", "unit": "celsius"}} </tool_call> <|im_end|> <|im_start|>tool {"temperature": 28, "condition": "Partly Cloudy", "humidity": 72} <|im_end|> <|im_start|>assistant The current weather in Tokyo is 28°C and partly cloudy with 72% humidity. <|im_end|>

Role Types

🖥️

system

Setup & instructions

Defines the agent's persona, capabilities, available tools, and behavioral rules. Set once at the start.

👤

user

Human input

The human's message or task request. Also used for injecting context at the start of agentic tasks.

🤖

assistant

Model output

Model's response — may include reasoning, tool calls, or final answers. This is what the model generates.

🔧

tool

Tool result injection

The result returned by executing a tool. Injected into context so the model can reason over the output.

Tool Calling

Hermes supports two tool-calling styles: XML-tag style (native Hermes) and OpenAI-compatible JSON (via API wrappers). Both require declaring available tools in the system prompt using JSON Schema.

Declaring Tools in System Prompt

# System prompt with tool declarations <|im_start|>system You are a helpful assistant. You have access to the following tools: <tools> [ { "type": "function", "function": { "name": "search_web", "description": "Search the web for current information on a topic", "parameters": { "type": "object", "properties": { "query": { "type": "string", "description": "The search query" }, "max_results": { "type": "integer", "description": "Maximum number of results to return", "default": 5 } }, "required": ["query"] } } }, { "type": "function", "function": { "name": "run_python", "description": "Execute Python code and return the output", "parameters": { "type": "object", "properties": { "code": { "type": "string", "description": "Python code to execute" } }, "required": ["code"] } } } ] </tools> When you need to use a tool, output it using <tool_call> tags. <|im_end|>

Full Tool Call Loop

# Complete agentic tool call loop <|im_start|>user What is the latest Python version and show me how to check it programmatically? <|im_end|> # --- Model generates a tool call --- <|im_start|>assistant Let me search for the latest Python version first. <tool_call> {"name": "search_web", "arguments": {"query": "latest Python version 2025"}} </tool_call> <|im_end|> # --- Your code executes the tool and injects the result --- <|im_start|>tool {"results": [{"title": "Python 3.13 Released", "snippet": "Python 3.13 is now the latest stable release..."}]} <|im_end|> # --- Model calls the next tool --- <|im_start|>assistant Python 3.13 is the latest version. Now let me write code to check it programmatically. <tool_call> {"name": "run_python", "arguments": {"code": "import sys\nprint(sys.version)"}} </tool_call> <|im_end|> <|im_start|>tool {"output": "3.13.0 (main, Oct 2024) [GCC 11.4.0]"} <|im_end|> # --- Model generates final answer --- <|im_start|>assistant The latest Python version is **3.13**. You can check the version programmatically with: ```python import sys print(sys.version) # 3.13.0 (main, Oct 2024) [GCC 11.4.0] ``` <|im_end|>

Agent Patterns

ReAct (Reason + Act)

ReAct is the most common pattern for Hermes agents: the model alternates between Thought (reasoning), Action (tool call), and Observation (tool result) until a final answer is reached.

1

Thought

Model reasons about what to do next. Often wrapped in <thinking> tags or as plain reasoning text before the tool call.

2

Action

Model emits a <tool_call> with name and JSON arguments. Your orchestration code parses this and executes the tool.

3

Observation

Tool result is injected back as a <tool> role message. The model continues reasoning with the new information.

4

Repeat or Answer

Loop continues until the model decides it has enough information and produces a final response without a tool call.

Multi-Agent Architecture

Orchestrator Agent (Hermes-3-70B) │ Receives task, plans sub-tasks │ ├──► Research Agent → search_web, read_url │ └── returns structured findings │ ├──► Code Agent → run_python, execute_shell │ └── returns code output │ ├──► Data Agent → query_db, csv_analyze │ └── returns analyzed data │ └──► Writer Agent → format_report, send_email └── returns formatted output │ ▼ Final Answer assembled by orchestrator
Tip: Hermes-3-70B works well as the orchestrator. Use Hermes-3-8B for worker agents to reduce cost while keeping structured output reliability.

System Prompts for Agents

Hermes is trained to follow complex system prompts reliably. A well-structured system prompt is the foundation of every agent. Key sections to include:

# High-quality agent system prompt template <|im_start|>system You are Aria, an expert AI research assistant with access to real-time web search, code execution, and document analysis tools. You help users answer complex questions by combining multiple sources and reasoning carefully. ## Your Capabilities - Search the web for current information - Execute Python code for calculations and data analysis - Read and analyze documents - Summarize and synthesize information from multiple sources ## Behavioral Rules - Always think before acting: reason about which tool is most appropriate - Use multiple tools when needed to give comprehensive answers - Cite sources when using web search results - When code execution is needed, write clean, commented code - If a tool fails, try an alternative approach - When you have enough information, give a direct, concise answer ## Output Format - Use markdown for structure and readability - Lead with the key answer, then provide details - Keep responses focused — don't over-explain ## Available Tools <tools> [... tool definitions ...] </tools> <|im_end|>

System Prompt Anti-Patterns

❌ Avoid

  • Vague persona ("Be helpful")
  • No tool usage guidance
  • Contradictory rules
  • No output format guidance
  • Excessively long, unfocused prompts

✅ Best Practice

  • Named persona with clear role
  • Explicit tool usage instructions
  • Consistent, ordered rules
  • Output format specification
  • Short sections with clear headers

Structured Output (JSON Mode)

Hermes excels at producing reliable JSON output. You can enforce structured extraction without tool calling — useful for parsing, classification, and data extraction tasks.

# JSON extraction with schema enforcement <|im_start|>system You are a data extraction assistant. Always respond with valid JSON only. No extra text, no markdown — pure JSON matching the requested schema. <|im_end|> <|im_start|>user Extract all people and their roles from this text: "Alice Chen (CTO) and Bob Smith (Lead Engineer) presented the Q3 roadmap." Respond using this schema: { "people": [ {"name": "string", "role": "string"} ] } <|im_end|> <|im_start|>assistant { "people": [ {"name": "Alice Chen", "role": "CTO"}, {"name": "Bob Smith", "role": "Lead Engineer"} ] } <|im_end|>

Using Grammar Constraints (llama.cpp / Ollama)

# Force valid JSON with GBNF grammar — eliminates hallucinated formats # Ollama API with format enforcement curl http://localhost:11434/api/chat -d '{ "model": "hermes3", "format": "json", "messages": [ {"role": "user", "content": "List 3 programming languages as JSON"} ] }'

Deployment

Hermes can be deployed locally for privacy-first workloads or on GPU servers for production scale.

Ollama (Local — Easiest)

# Pull and run Hermes 3 via Ollama ollama pull hermes3 # 8B model (~5GB GGUF) ollama pull hermes3:70b # 70B model (~40GB GGUF) ollama run hermes3 # Use via OpenAI-compatible API (port 11434) from openai import OpenAI client = OpenAI( base_url="http://localhost:11434/v1", api_key="ollama" # ignored but required ) response = client.chat.completions.create( model="hermes3", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is 2+2?"} ] ) print(response.choices[0].message.content)

vLLM (Production — GPU Server)

# Start vLLM server with Hermes-3 python -m vllm.entrypoints.openai.api_server \ --model NousResearch/Hermes-3-Llama-3.1-8B \ --tensor-parallel-size 1 \ --max-model-len 32768 \ --gpu-memory-utilization 0.9 \ --enforce-eager # Call via OpenAI-compatible API curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "NousResearch/Hermes-3-Llama-3.1-8B", "messages": [{"role": "user", "content": "Hello"}], "temperature": 0.7, "max_tokens": 1024 }'

Cloud API Providers

Provider Models Available Strengths API Compatible
Together AI Hermes-3 8B, 70B, 405B Fast, cheapest inference OpenAI
Fireworks AI Hermes-3 8B, 70B Lowest latency, speculative decoding OpenAI
Groq Hermes-2-Pro Ultra-fast LPU inference OpenAI
Hugging Face TGI Any Hermes variant Self-hosted, full control OpenAI + Messages
Ollama (local) Hermes-3 8B, 70B Privacy, offline, zero cost OpenAI

Python SDK Integration (LangChain)

# Using Hermes with LangChain + Ollama from langchain_ollama import ChatOllama from langchain_core.messages import HumanMessage, SystemMessage llm = ChatOllama( model="hermes3", temperature=0.1, # lower for agent tasks ) # With tool binding (LangChain tool_calling format) from langchain_core.tools import tool @tool def get_weather(city: str) -> str: """Get the current weather for a city.""" return f"Weather in {city}: 25°C, sunny" llm_with_tools = llm.bind_tools([get_weather]) response = llm_with_tools.invoke("What's the weather in Paris?")

Benchmarks

Hermes-3 models consistently outperform their base models and are competitive with closed-source models on agentic benchmarks.

Model MMLU GSM8K HumanEval Tool Use Context
Hermes-3-8B 71.2% 84.6% 72.1% ★★★★☆ 128k
Hermes-3-70B 82.5% 93.1% 80.3% ★★★★★ 128k
Hermes-3-405B 88.1% 96.2% 87.5% ★★★★★ 128k
Llama 3.1 8B (base) 66.7% 75.3% 62.0% ★★☆☆☆ 128k
GPT-4o mini 82.0% 90.5% 85.1% ★★★★★ 128k
Claude Haiku 3.5 80.5% 91.0% 80.0% ★★★★★ 200k
Key Takeaway: Hermes-3-70B closes most of the gap with GPT-4o mini while being fully open, self-hostable, and private. Hermes-3-8B is an excellent cost-effective local model for agent prototyping.

Hermes vs Other Open Models

Aspect Hermes-3 Mistral / Mixtral Qwen2.5 DeepSeek-V2
Tool Calling Excellent Good Excellent Good
JSON Reliability Very High Medium High Medium
Context Window 128k 32k–128k 128k 128k
Local (CPU/GPU) Yes (GGUF) Yes (GGUF) Yes (GGUF) GPU recommended
License Llama 3.1 License Apache 2.0 Qwen License DeepSeek License
Agent Community Very Active Active Active Growing
Best For Agents, tool use, JSON General text, coding Code, multilingual Code, reasoning

Best Practices

🌡️

Temperature

Use temperature=0.1–0.3 for agent tasks requiring reliable tool calls and JSON. Use higher values (0.7+) only for creative tasks.

🔄

Stop Sequences

Set stop sequences to </tool_call> to intercept after tool invocation before the model continues generating. Parse and execute before resuming.

🔒

Validate Tool Output

Always validate and sanitize tool call arguments before executing. Hermes is reliable but treat JSON output as untrusted input — especially for shell/code execution tools.

📏

Tool Descriptions

Write clear, specific tool descriptions. The model uses these to decide which tool to call. Vague descriptions lead to wrong tool selection. Include when NOT to use the tool.

🔢

Max Iterations

Always set a maximum iteration limit for agent loops (e.g., 10 steps). Without this, a confused model can loop indefinitely consuming tokens and compute.

⚡

Model Selection

Use 8B for local dev and prototyping, 70B for production agents. Only use 405B when the task genuinely requires maximum capability — it's 5x the cost of 70B.

🧱

Grammar Constraints

Use GBNF grammar constraints (llama.cpp) or format: "json" (Ollama) to guarantee valid JSON output. Eliminates format hallucinations entirely.

📝

Logging

Log every tool call, arguments, and result during development. The agent loop is a state machine — visibility into each step is critical for debugging unexpected behavior.

Complete Python Agent Example

import json from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") TOOLS = [ { "type": "function", "function": { "name": "calculator", "description": "Evaluate a mathematical expression. Use for any arithmetic.", "parameters": { "type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"] } } } ] def execute_tool(name, args): if name == "calculator": try: return str(eval(args["expression"])) # sandboxed in real use except Exception as e: return f"Error: {e}" def run_agent(user_message, max_iterations=10): messages = [ {"role": "system", "content": "You are a helpful math assistant. Use the calculator tool for computations."}, {"role": "user", "content": user_message} ] for i in range(max_iterations): response = client.chat.completions.create( model="hermes3", messages=messages, tools=TOOLS, temperature=0.1 ) msg = response.choices[0].message # No tool call → final answer if not msg.tool_calls: return msg.content # Execute all requested tool calls messages.append(msg) for tc in msg.tool_calls: result = execute_tool(tc.function.name, json.loads(tc.function.arguments)) messages.append({ "role": "tool", "tool_call_id": tc.id, "content": result }) return "Max iterations reached" print(run_agent("What is (123 * 456) + (789 / 3)?"))