Building a Support Copilot with Hermes

TL;DR

• Support copilot (editorial term — not Hermes product language) means an internal assistant for human agents: draft replies, summarize threads, look up policy, recommend escalation — with the human owning every customer-visible message.
• This is not Part 2’s customer bot. Part 2 wires a Chatwoot Agent Bot that auto-replies on pending conversations. A copilot responds to explicit agent actions and posts private notes or copy-to-send drafts.
• Recommended stack (July 2026, editorial): separate Hermes profile (support-copilot) + custom read-heavy toolset + API server + thin assist bridge or sidebar UI → Chatwoot private notes.
• Governance matters: memory.write_approval / skills.write_approval, AGENTS.md draft-only rules, eval traces, and explicit “insufficient evidence” when retrieval is empty.
• Hermes value vs a thin wrapper: sessions, skills, bounded memory, tool scoping, trajectories, and approval gates — not just openai.chat.completions with a long system prompt.

What You Will Learn Here

• How a support copilot differs from a customer-facing WhatsApp agent
• PM vs engineering ownership for assist products
• Reference architecture: Hermes profile, toolsets, Chatwoot private notes
• Memory, skills, and session patterns for support teams
• Human-in-the-loop: confidence, escalation, and approvals
• Quality gates and when to add enterprise RAG
• Operational checklist and failure modes

This is Part 4 of the Hermes Agent series. Read Part 1 for harness concepts and Part 2 for customer-channel wiring. Part 3 covers scheduled ops.

Support Copilot vs Customer Bot

Hermes docs describe a general agent harness — not a packaged “support copilot” SKU. In this series, the distinction is who sends the message:

CUSTOMER CHANNEL (Part 2)          INTERNAL COPILOT (Part 4)
─────────────────────────          ─────────────────────────
Customer on WhatsApp               Human agent in Chatwoot UI
Agent Bot auto-replies             Agent clicks "Suggest" / macro
Hermes → customer-visible reply    Hermes → private note OR copy box
Conversation status: pending       Conversation status: open (human-owned)
Bridge: Agent Bot webhook          Bridge: explicit assist API call

Editorial judgment: Run these as separate Hermes profiles so tool surfaces, memory, and learning loops do not collide.

Delta from Part 2

Who Owns What (PM vs Engineering)

Part 2 defined who owns Meta tokens and bridge ingress. Part 4 defines who owns assist quality and accountability.

Reference Architecture

Support agent opens conversation in Chatwoot (status: open)
        |
        v
+-----------------------------------+
| Copilot UI (you build one)        |
| - Chatwoot Dashboard App          |
| - Internal "Suggest reply" web app|
| - Browser extension               |
+---------------+-------------------+
                | HTTPS + API_SERVER_KEY
                v
+-----------------------------------+
| Hermes API server                 |
| profile: support-copilot          |
| toolset: support-copilot (custom) |
+---------------+-------------------+
                |
     +----------+----------+
     v          v          v
  qmd/wiki   MCP/CRM    session_search
  skills     order API   MEMORY.md
                |
                v
+-----------------------------------+
| Outputs (human-reviewed)          |
| - Suggested reply (copy/paste)    |
| - Private note (private: true)    |
| - Escalation recommendation       |
+-----------------------------------+

Uncertainty to plan for: Chatwoot has no first-party Hermes copilot panel. You build the sidebar UI or macro-triggered HTTP integration. Agent Bot webhooks (Chatwoot Agent bots) are optimized for bot-owned pending conversations — a poor fit for human-initiated assist.

Step 1: Create an Isolated Hermes Profile

hermes profile create support-copilot \
  --description "Internal Chatwoot assist; draft-only; no customer auto-send."

cat >> ~/.hermes/profiles/support-copilot/.env <<'EOF'
OPENAI_API_KEY=sk-...
API_SERVER_ENABLED=true
API_SERVER_HOST=127.0.0.1
API_SERVER_PORT=8643
API_SERVER_KEY=change-me-to-a-long-random-secret
EOF

support-copilot gateway install
support-copilot gateway start   # or: support-copilot gateway run (foreground dev)

Profiles isolate config, memory, sessions, and gateway state (Profiles docs). Keep customer-bot and support-copilot on different ports if both API servers run concurrently.

Step 2: Scope Tools for Assist (Not Defaults)

The API server exposes a near-full default tool surface (hermes-api-server) unless you restrict it (API Server). Do not ship terminal or browser tools on a support copilot.

Per the Toolsets Reference, platform defaults like hermes-whatsapp map to near-full CLI toolsets — Part 2’s narrowing advice applies here too. Define a custom assist toolset in config and verify at runtime:

# ~/.hermes/profiles/support-copilot/config.yaml
memory:
  write_approval: true
skills:
  write_approval: true
display:
  memory_notifications: verbose
auxiliary:
  background_review:
    provider: openai-api
    model: gpt-4o-mini

custom_toolsets:
  support-copilot:
    - safe
    - session_search
    - skills
    - memory

Then scope the API server platform via hermes tools (select api_server / support-copilot toolset). Verify flags against your Hermes version — hermes gateway --help may not expose --toolsets on all builds; config + hermes tools is the reliable path (same caveat as Part 2).

curl http://127.0.0.1:8643/v1/toolsets \
  -H "Authorization: Bearer $API_SERVER_KEY"

The built-in safe toolset includes read-only web tools (web_search, web_extract, vision_analyze) without terminal or file execution (Toolsets Reference).

Step 3: Author Copilot Rules in AGENTS.md

Channel rules belong in Hermes — not hardcoded in the bridge (same lesson as Part 2):

<!-- AGENTS.md -->
# Support Copilot Rules

- You assist human agents only. Never imply you sent a message to the customer.
- Default output structure:
  1. suggested_reply (plain text, WhatsApp-safe length)
  2. confidence: high | medium | low
  3. sources: list skill names, doc paths, or MCP record IDs used
  4. escalation_recommended: true | false
- If policy is unclear or retrieval is empty, say "insufficient evidence" — do not invent refund eligibility.
- For billing disputes over $500, recommend escalation per escalation-matrix skill.

Hermes loads this via Prompt Assembly — layered on top of the core harness prompt, not replacing it.

Seed bounded memory

Mid-session memory writes hit disk immediately but do not mutate the frozen prompt until rebuild (Persistent Memory). Preload critical policy before a shift.

Skills to ship

Author via /learn from runbooks (Skills System).

Step 4: Build the Assist Bridge

Unlike Part 2’s Agent Bot webhook, the copilot bridge handles explicit agent requests:

# copilot/assist.py
import httpx

CHATWOOT_BASE = "https://support.example.com"
ACCOUNT_ID = 1
HERMES_BASE = "http://127.0.0.1:8643"
HERMES_API_KEY = "..."


def extract_response_text(data: dict) -> str:
    for item in data.get("output", []):
        if item.get("type") != "message":
            continue
        for part in item.get("content", []):
            if part.get("type") == "output_text":
                text = (part.get("text") or "").strip()
                if text:
                    return text
    return ""


async def fetch_chatwoot_thread(client: httpx.AsyncClient, account_id: int, conversation_id: int, token: str) -> str:
    """Hermes does not ingest Chatwoot natively — pack thread context yourself."""
    url = f"{CHATWOOT_BASE}/api/v1/accounts/{account_id}/conversations/{conversation_id}/messages"
    resp = await client.get(url, headers={"api_access_token": token})
    resp.raise_for_status()
    lines = []
    for msg in resp.json().get("payload", []):
        role = "customer" if msg.get("message_type") in (0, "incoming") else "agent"
        content = (msg.get("content") or "").strip()
        if content:
            lines.append(f"{role}: {content}")
    return "\n".join(lines[-20:])  # recent tail


async def suggest_reply(conversation_id: int, agent_token: str) -> str:
    conversation_key = f"chatwoot:copilot:{conversation_id}"
    session_key = f"chatwoot:contact:{conversation_id}"

    async with httpx.AsyncClient(timeout=120.0) as client:
        thread_text = await fetch_chatwoot_thread(client, ACCOUNT_ID, conversation_id, agent_token)

        payload = {
            "model": "hermes-agent",
            "input": thread_text,
            "conversation": conversation_key,
            "store": True,
        }
        headers = {
            "Authorization": f"Bearer {HERMES_API_KEY}",
            "X-Hermes-Session-Key": session_key,
        }
        resp = await client.post(f"{HERMES_BASE}/v1/responses", json=payload, headers=headers)
        resp.raise_for_status()
        return extract_response_text(resp.json())


async def post_private_note(conversation_id: int, agent_token: str, content: str) -> None:
    url = f"{CHATWOOT_BASE}/api/v1/accounts/{ACCOUNT_ID}/conversations/{conversation_id}/messages"
    async with httpx.AsyncClient(timeout=30.0) as client:
        await client.post(
            url,
            headers={"api_access_token": agent_token},
            json={"content": content, "message_type": 1, "private": True},
        )

Session semantics (per API Server):

• conversation (chatwoot:copilot:{id}) chains multi-turn assist within one copilot session.
• X-Hermes-Session-Key (chatwoot:contact:{id}) scopes long-term memory providers per customer thread — intentionally separate from the assist transcript key.
• /v1/chat/completions is stateless; avoid it for assist threads.

Part 2 used one key for both roles on the customer bot path; copilot splits them so assist thread resets do not scramble contact-scoped memory.

See Stateful vs Stateless Agents in Production for the general tradeoff.

Step 5: Human-in-the-Loop

Private notes (agent-only)

Chatwoot’s Create Message API supports private: true — invisible to the customer:

{
  "content": "🤖 Copilot draft (not sent):\n\nSuggested reply: ...\n\nSources: refund-policy skill §3\nConfidence: medium",
  "message_type": 1,
  "private": true
}

Chatwoot also accepts "message_type": "outgoing" as a string — integer 1 matches the Part 2 bridge convention.

Use private notes for reasoning summaries, source lists, and escalation rationale. Default the copilot UI to private; require an explicit human action to copy into the customer reply box.

Escalation

Hermes has no native confidence primitive — implement via AGENTS.md output schema and eval. When escalation_recommended: true or confidence: low:

1. Post a private note with the recommendation.
2. Human executes toggle_status → open or assignment in Chatwoot UI (Toggle Status API).

Approvals for sensitive lookups

For thick-client copilot UIs, Hermes exposes POST /v1/runs/{run_id}/approval when a tool call waits on human decision (API Server — Runs API). Useful for PII-heavy CRM lookups.

Learning governance

With memory.write_approval: true and skills.write_approval: true, background review may propose macros from ticket patterns — staged for /memory pending and /skills pending review. Without gates, a single bad ticket can teach “always refund.”

Step 6: Quality Gates

Hermes gives you building blocks; the product eval layer is yours.

What Hermes provides

What you still build (ticket-level eval)

Use scenario and rubric tooling for Chatwoot golden tickets — not the batch trajectory runner alone:

• Evaluating AI Agents with LangWatch — scenario suites
• How to Review AI-Generated Responses — human rubric
• Trace Grading vs Scenario Testing
• RAG Apps in Practice — before customer-facing doc grounding

Source grounding checklist

[ ] Copilot output lists sources (skill name, qmd path, MCP record ID)
[ ] Empty retrieval → explicit "insufficient evidence"
[ ] Golden ticket set (50–200) with expected citations
[ ] Regression on prompt/model/skill changes
[ ] PII scrubbing before logging trajectories

When to add enterprise RAG

Hermes Copilot vs Thin ChatGPT Wrapper

Part 2 argued the bridge should stay dumb. Part 4 argues the copilot product layer — policy, evals, tool governance, learning — is where Hermes earns its keep.

Operational Checklist

[ ] Separate Hermes profile for copilot (not customer-bot)
[ ] Custom read-heavy toolset verified via GET /v1/toolsets
[ ] memory.write_approval + skills.write_approval enabled
[ ] AGENTS.md: draft-only, cite sources, escalation rules
[ ] MEMORY.md seeded with critical policies (<80% capacity)
[ ] API_SERVER_KEY rotated; bind loopback unless behind TLS proxy
[ ] Copilot UI authenticated (SSO); no Hermes keys in browser
[ ] Private notes tested (private: true not visible to customer)
[ ] Golden ticket eval set + baseline scores
[ ] Trajectory/PII redaction policy documented
[ ] Escalation SLA owned by support lead

Common Failure Modes

Bottom Line

A support copilot is a governed assist layer, not a customer bot with a different skin. Hermes fits when you need sessions, skills, scoped tools, and auditable learning — with humans always owning outbound messages.

Part 2 wired the customer channel. Part 4 is where product policy, eval quality, and operational ownership meet the harness. Get the profile, toolset, and AGENTS.md contract right first; the bridge is the easy part.

Sources

Primary sources consulted on July 3, 2026:

1. Hermes Agent — Architecture — harness subsystems, platform-agnostic core. hermes-agent.nousresearch.com/docs/developer-guide/architecture
2. Hermes Agent — Prompt Assembly — AGENTS.md, tiered prompts. hermes-agent.nousresearch.com/docs/developer-guide/prompt-assembly
3. Hermes Agent — Persistent Memory — bounded memory, write_approval, background review. hermes-agent.nousresearch.com/docs/user-guide/features/memory
4. Hermes Agent — Skills System — progressive disclosure, /learn, write approval. hermes-agent.nousresearch.com/docs/user-guide/features/skills
5. Hermes Agent — API Server — Responses API, Sessions API, Runs API, X-Hermes-Session-Key. hermes-agent.nousresearch.com/docs/user-guide/features/api-server
6. Hermes Agent — Toolsets Reference — safe, platform defaults, custom toolsets. hermes-agent.nousresearch.com/docs/reference/toolsets-reference
7. Hermes Agent — Profiles — isolated config, memory, sessions. hermes-agent.nousresearch.com/docs/user-guide/profiles
8. Hermes Agent — Profile Commands — hermes profile create, support-copilot gateway. hermes-agent.nousresearch.com/docs/reference/profile-commands
9. Hermes Agent — Trajectory Format — eval exports. hermes-agent.nousresearch.com/docs/developer-guide/trajectory-format
10. Hermes Agent — Batch Processing — scenario suites. hermes-agent.nousresearch.com/docs/user-guide/features/batch-processing
11. Hermes Agent — qmd optional skill — hybrid doc search. hermes-agent.nousresearch.com/docs/user-guide/skills/optional/research/research-qmd
12. Chatwoot — Agent bots — contrast with human-owned conversations. chatwoot.com/hc/user-guide/articles/1677497472-how-to-use-agent-bots
13. Chatwoot Developer Docs — Create New Message — private: true notes. developers.chatwoot.com/api-reference/messages/create-new-message
14. Chatwoot Developer Docs — Toggle Status — escalation handoff. developers.chatwoot.com/api-reference/conversations/toggle-status
15. Nous Research — Hermes Agent GitHub — profiles, API server, toolsets. github.com/NousResearch/hermes-agent

Series and related articles:

16. Part 1: Hermes Agent architecture and RAG lessons
17. Part 2: WhatsApp agent with Cloud API, Chatwoot, Hermes, and OpenAI
18. Part 3: Hermes cron jobs for agent ops
19. RAG Apps in Practice
20. Stateful vs Stateless Agents in Production
21. Evaluating AI Agents with LangWatch
22. How to Review AI-Generated Responses