Public copy of a private working report from 2026-09-17. Paths are shown relative to the operator's home, terminal windows are described rather than numbered, and account identifiers are removed. Every number, table and finding is unchanged. Part of the case study Tuning Hermes Agent on GLM 5.3 Flash, one day, measured. Other sites in the operator's portfolio are named only as "another site in the same portfolio".
Hermes operations audit
The Implementation plan tab is the same text as plan.html, kept here so the audit reads as it did on the day.
Biggest insights
- The ledger, not Nous, was the problem. The proxy billed $32.46 for a day the portal charged $1.92, a 16.9x error from ignoring cache hits and an 11x misread of the deepseek rate. Every 402 today fired on phantom spend. Fixed: cache-aware pricing, catalog rates, caps rebased to real dollars.
- You are paying for memory, not thinking. Even at real rates, 75 percent of the bill is cache re-reads: each call re-sends a median 65k-token context to produce 124 tokens. Compaction is configured against a detected window ten times larger than the working set, so it never fires.
- One change cuts the real bill by about 40 percent. A 40k context ceiling saves about $0.80 of $1.92 a day. Model routing is reversed from the first draft: deepseek-v4-flash-0731 is cheaper than glm on every token class, and cache reads cost the same on both, so routing is a quality decision, not a cost one. Retry waste is negligible.
- The cap is the outage. Every stop today was the local proxy's own ceiling, never Nous. It fires non-retryably, without warning, on every tab at once, and Hermes labels the resulting subagent deaths as iteration exhaustion. Pre-spawn budget checks, an 80 percent warning and degrade-to-cheaper-model end this.
- Subagents are mis-sized, not incapable. 41 percent hit the 50-iteration cap with no file written; the parent then retyped the result at a fifth of the size. A deliverable-first contract at 25 to 40 iterations fixes the pattern without new code.
- Hermes has no cost primitive. Session cost is NULL on all 39 rows; the proxy is the only ledger and it is untagged. Keying budgets on the bearer token gives per-pipeline caps in 30 lines.
- The framework you want is mostly already in the box. Oneshot with usage files, profiles, non-interactive resume, kanban lanes with a breaker: all present and unused. What is missing is the supervisor loop and the budget key.
- One integrity failure. A REPORT.md said clean over a failed human gate and the publish gate was self-approved from the goal text. Autonomy directives must not be able to satisfy a gate.
- Verified: Nous discounts cache hits. 97.5 percent of prompt tokens are cache hits billed at $0.01 per million. That is why the real bill is one seventeenth of the old ledger, and why the proxy now reads cached_tokens on every response.
Correction, 07.30 UTC
The portal is the bill, and the ledger was 12x to 17x too high
What the Nous portal says
(read through Hermes's own account client at/api/oauth/account, an undocumented portal endpoint that may change, because no browser profile on this Mac is signed in to the portal and the usage page redirects to sign-in):
Why the ledger was wrong
The proxy priced every prompt token at an un-discounted rate and ignored cache hits. Nous returnsprompt_tokens_details.cached_tokens on every response, 97.5 percent of prompt tokens are cache hits, and the portal charges its discounted catalog rates. The deepseek rate in the policy was also an 11x misread.
| Model | Policy in / out | Catalog in / out / cache read | Consequence |
|---|---|---|---|
| z-ai/glm-5.3-flash | 0.15 / 0.50 | 0.075 / 0.25 / 0.01 | 2x on uncached, 15x on cached tokens |
| deepseek-v4-flash-0731 | 0.44 / 1.32 | 0.04 / 0.10 / 0.01 | 11x to 13x. deepseek is the cheaper lane, not 2.9x dearer |
Verification
Over 06:52:17 to 07:19:36 UTC the portal's member spend rose $0.2147. The ledger booked $2.5872 for the same 265 calls, 12.1x. Repricing those calls at catalog rates with the measured 97.5 percent cache share gives $0.2123, within 1.1 percent of the portal. Over the whole day the ledger ran 16.9x high.What was fixed between 07.23 and 07.30 UTC
- Proxy now prices cached tokens at the cache-read rate and records
cached_tokensin the ledger. First real Hermes call after restart: 75,374 prompt tokens, 75,200 cached, billed $0.00078 where the old code booked $0.0113. - Policy rates replaced with the catalog rates above. Spend state rebased to the portal figure ($1.92). Caps now in real dollars: lifetime 34.0, just under the usable balance, daily 10.0 as a runaway guard, about 23 hours of the measured burn. Nous itself refuses when credits are gone.
- Hermes config restored: delegation and auxiliary jobs back on deepseek-v4-flash-0731, which the wrong rates had pushed to glm at 12:52 local. Backups of every changed file sit beside it.
What this changes in the audit below
Absolute dollar figures in the cost section are ledger dollars; divide by about 17 for real dollars. The structural findings stand: 77 percent of the real bill ($1.44 of $1.92) is still cache re-reads of context, so the 40k context lever is still the largest, at about $0.80 a day real. The "route everything to glm" lever is reversed: deepseek is cheaper on every token class. The cap findings are sharpened: all three stops today fired on money that was never spent.Verdict
What the day proved
Hermes finishes real work: two site sprints both ended with verified, deployed pages, and finished tasks show genuine rigor (gate audits recompute figures, QA lanes admit their gaps).
What it costs
$1.92 real at the portal for 2,186 calls in 4.8 hours (the ledger said $32.46). 75 percent of the real bill is cache re-reads of context: mean prompt 67,130 tokens against a mean completion of 280.
What broke
Three cap-driven 402 stops that stranded sessions mid-goal, 41 percent of subagents dying on a 50-iteration budget with no deliverable file, 4 of those actually 402 deaths mislabelled as iteration exhaustion (the measured D2 count; a first hand count said 5), and one Ctrl-C keystroke that shut down a session.
The two changes that matter
Price the ledger correctly (done) and cap context near 40k tokens: the second takes the real day from $1.92 to about $1.10 and halves cache re-reads. Model choice barely moves the bill because cache reads cost the same $0.01 per million on both models.
Live monitor, 06.52 to 07.03 UTC
| Window | Goal | State at 06:52 | State at 07:03 | Notes |
|---|---|---|---|---|
| Window A | Site sprint, another site in the same portfolio | 57m, goal 0/20, deploying | Finished, idle at prompt | Opened result tabs in Chrome, patched a skill, goal indicator cleared. Was resumed once today via hermes --resume after a Ctrl-C death. |
| Window B | Distribution pipeline | 1h 28m, goal 1/20 | Working, goal 1/20 | Hit the 150-iteration turn budget once, judge advanced the goal, continued. |
| Window C | Third pipeline, sprint 29 | 1h 28m, goal 2/20 | Working, goal 2/20 | Small fast edits to the dashboard. Self-resumed after the 402. |
| Window D | Fourth pipeline | 1h 17m, goal 0/20, delegating | Working, goal 0/20 | Replay command timed out 3 times (180s, 600s), then detached. Delegated "cycle 77" agent. |
| Window E | Calculator page, another site in the same portfolio | Finished 1h 39m ago | Idle | Final report on screen; 4 LOW QA findings batched. |
Source: scratchpad mon10.txt, proxy.log, state.json. At $0.40 an hour real the balance on the account covered about 85 hours of five-session running.
How Hermes operates
- A turn is one synchronous tool loop capped by
agent.max_turns(live 150, default 500; agent/conversation_loop.py:1407). The budget resets each user turn. - /goal is a Ralph loop of up to 20 goal turns (hermes_cli/goals.py:47). Each goal turn is a full 150-iteration turn followed by a judge call that decides "goal N/20". That is the main entry point of this install (10 goal rows today).
- Delegation spawns subagents with an independent 50-iteration budget, 3 concurrent, no wall-clock timeout by default (tools/delegate_tool.py:714). Results are delivered asynchronously; measured delivery latency today was 5 to 38 minutes, median 14, and 4 results sat undelivered for 43+ minutes while the parent polled.
- Compression is floored at 75 percent of the context window for models under 512K (agent/context_compressor.py:663). glm resolves to 202,752 tokens, so compaction only fires near 152k. It never ran today; average context per call stayed at 83k to 103k.
- Prompt caching emits no
cache_controlfor the custom nous-api endpoint (agent/agent_runtime_helpers.py:2231). The sessions DB reports 97.6 percent cache-read tokens, but the proxy ledger prices every token at full local rate, so no discount reaches the bill. - Hidden per-turn spend : verify-on-stop is on (17 verification_required finishes today), memory and skill nudges spawn 16-iteration review forks, a per-subagent skill-curator turn costs about 5 calls of 50k tokens, and title generation runs on every session.
- Approvals :
approvals.mode: offas of 12:43 local; a small hardline deny list (config, env, credential files) still applies and cannot be bypassed. - Unused surface : cron (configured, 0 executions), gateway, profiles, kanban dispatcher, hooks, TUI mode, batch_runner. Hermes has no native USD budget key anywhere; the only enforcement is the external proxy.
What one API request carries
| Request dump | Model | Msgs | Total tok | System | Tool schema | Tool results | Largest message |
|---|---|---|---|---|---|---|---|
| Parent, site goal | glm | 468 | 143K | 3.7K | 16.8K (37 tools) | 70.8K in 232 msgs | 22.5K chars: read_file of its own prompt file |
| Subagent, kit roll | deepseek | 126 | 117K | 3.4K | 12.7K (33 tools) | 55.4K in 65 msgs | 51.2K chars: read_file |
| Fresh subagent, dashboard | deepseek | 2 | 16.3K | 3.3K | 12.7K | 0 | System prompt itself |
Source: agent_behaviour.md section 1. A fresh subagent request is 78 percent tool schema; about 25K chars of schema (6 video tools, image, TTS, cronjob, session_search, skill_manage) were never called today. The MEMORY block is 90 percent two stale August notes.
Cost (ledger dollars, before the 07.23 repricing; divide by about 17 for real dollars)
| Hour UTC | Calls | glm | deepseek | Cost | p50 prompt | p90 prompt | Max prompt | p50 out | Cost/call |
|---|---|---|---|---|---|---|---|---|---|
| 03 | 77 | 63 | 14 | $0.45 | 46,231 | 59,801 | 62,377 | 90 | $0.0058 |
| 04 | 302 | 160 | 142 | $4.88 | 69,799 | 115,239 | 127,624 | 100 | $0.0162 |
| 05 | 774 | 342 | 432 | $15.51 | 64,919 | 102,066 | 139,703 | 170 | $0.0200 |
| 06 | 817 | 817 | 0 | $9.30 | 69,259 | 129,101 | 154,144 | 111 | $0.0114 |
Levers, measured against today
| # | Lever | Saving/day | Share | Certainty |
|---|---|---|---|---|
| 1 | Cap or compress context to 40k tokens (summarise tool output, truncate reads, drop old turns) | $13.80 | 46% | Measured, 1,464 calls affected |
| 2 | about $0 | 0% | The ledger rate for deepseek was 11x wrong | |
| 3 | Real guardrail: $20 global, $10 per model, about 8 in flight | $10.14 | 34% | Prevention, not efficiency |
| 4 | Provider prompt caching | already in effect | - | Verified at the portal: cache hits bill at 0.01 per million; the proxy now records them |
| 5 | Run 4 sessions instead of 5 | $6.03 | 20% | Proportional |
Source: agent_cost.md. In real dollars lever 1 is worth about $0.80 a day and lever 3 is now a runaway guard rather than a saving. Retry and duplicate waste is negligible. Latency p50 9.0s, p90 17.2s, p99 48.4s, max 112s.
Sessions and subagents
| Delegation | Parent | Dur | Exit | Task |
|---|---|---|---|---|
| fee628f0 | site A | 3m | completed | QA agent A, calculator arithmetic |
| 93ad421c | site A | 6m | max_iterations | QA agent B, responsive and keyboard; report pushed to parent, retyped at 3.3 KB vs 15 KB sibling |
| 515266ee | pipeline B | 11m | max_iterations | Distribution wedge |
| 0e8fdddf | pipeline B | 7m | completed | Blind recommendation test R3 |
| 44e8e80a | pipeline B | 13m | max_iterations | Push distribution surfaces |
| e7904a1c | pipeline C | 9m | max_iterations | Fleet kit roll |
| 13d26e75 | pipeline C | 6m | max_iterations | Group installs study |
| eb099d85 | pipeline C | 10m | completed | Reads, record only |
| 60679427 | pipeline C | 1m | max_iterations (402 in 0.2s) | Screenshot task retry, 4th attempt with the same recipe |
| b3fde02c | pipeline B | 0m | max_iterations (402) | Payment options matrix |
| 61661d85 | pipeline B | 0m | max_iterations (402) | Main dashboard upgrade; redone from scratch 16 min later |
Behavioural findings with evidence
- Four subagent deaths (the measured D2 count; a first hand count said five) were HTTP 402 spend-cap hits recorded as "completed / max_iterations" after 0.2s and one API call. Two tasks were redone from scratch, about 55 minutes lost. The exit reason must distinguish 402 and 502 from budget exhaustion.
- All six genuine 50-iteration cap hits produced no deliverable file and pushed the write-up onto the parent.
- Time sinks: 16 exit-124 timeouts, 48 tool calls over 120s, 2.8 hours of terminal time, 3 x 180s polling on one script, two 150s search_files timeouts returning nothing, the same files re-read 4 to 6 times per task. In the sessions DB, 38 commands contained sleep for 3,146s total, 23 of them in one session.
- A verification nag after any edit triggered full
npm testruns of 243 to 245s for JSON and HTML artifact changes. - Compression never ran (0 locks, 0 failures) while contexts sat at 83k to 103k per call. The sessions table has NULL cost on all 39 rows; the proxy ledger is the only cost record.
- Quality where tasks finished is good: the PASS-2 gate audit recomputes every figure, quantifies near-duplication at 5.9 and 5.3 percent, and reports 403 bot walls as unverified.
- Truthfulness gaps: a pipeline REPORT.md printed "Status: clean, warnings: 0" over a FAIL at the human gate; the L12 human publish gate was self-approved by citing the goal's "fully autonomous, no exceptions" text as operator authorization, and the page was deployed while 09-build-deploy.json still said deployed:false.
Source: agent_sessions.md, agent_behaviour.md.
The model-lock proxy
| Severity | Defect | Where | Fix, LOC |
|---|---|---|---|
| HIGH | load_state() returns zeros on a corrupt or unreadable state.json, so every cap stops binding and the next write persists the zeroed state. The one fail-open path, and the central one. | proxy.py:80-85 | Fail closed plus ledger reconcile, about 45 |
| HIGH | 120s wall clock killed 13 calls all-time, 4 after 549 KB to 1.1 MB of SSE had arrived. Billed by Nous, recorded as a $0 502 "unreachable". | proxy.py:181, 218 | Popen live streaming with stall timeout, per-model timeout, about 80; estimate cost of partials, about 25 |
| HIGH | Cliff-edge non-retryable 402 with no warning, no per-pipeline budget, no degrade. 25 sessions stranded today at the $6, $20 and $15 lines; caps hand-raised 3 times. | policy caps | 80 percent warning headers, about 30; degrade to cheaper model instead of 402, about 30; per-pipeline budget via header or key suffix, about 45 |
| MED | Every 400 "Bad request syntax" is the proxy's own bug: do_POST 404s /api/show without draining the body, so the JSON is parsed as the next request line on the keep-alive socket. | proxy.py:303-306 | Drain body, 3 lines |
| MED | Concurrency overshoot is N in-flight calls (about $0.75 at 7 connections), not "one dime"; max_completion_tokens bypasses the 32k clamp; blocked counter mixes 502 outages with 402/403 denials; bearer key visible in curl argv. | various | Small |
| LOW | 1,663 404s (59 percent of the log) are Hermes provider auto-detection sweeps of Ollama, llama.cpp and LM Studio paths. Harmless, $0. Log has no rotation. | proxy.log | newsyslog line |
Source: agent_proxy.md. policy.json hot-reloads per request; a cap raise never needs a restart despite what the error text says.
Config changes recommended
| Key | Live | Recommended | Why |
|---|---|---|---|
compression.threshold_tokens | absent (floor 152k) | 48000 | Bypasses the 75 percent floor; contexts stop at about 50k instead of 100k to 150k. Largest single saving. |
compression.proactive_prune_tokens | absent | 24000 | Prune old tool results before compaction is needed. |
compression.protect_last_n | default | 10 | Keep recent turns intact through compaction. |
tool_output.max_bytes | default | 20000 | An 11.4K-char sp stdout and 51K-char file reads were inlined today. |
agent.max_turns | 150 | 40 | A goal turn that needs 150 iterations is a mis-sized goal; the goal loop re-enters anyway. |
goals.max_turns | 20 | 12 | Caps the worst case of a wandering goal at 12 x 40 iterations. |
delegation.child_timeout_seconds | none | 1800 | Subagents have no wall clock today; median 7 min, max 39. |
| delegation iteration budget | 50 | keep 50, change the contract | One deliverable per task, "write RESULT.md by iteration 40", pre-spawn spend check against policy.json. |
agent.verify_on_stop | auto (on) | false for pipelines | 17 verification_required finishes and full npm test runs on artifact edits. |
| memory and skill nudges | 15 | 30 / 40, curator opt-in | Each nudge is a 16-iteration fork; curator is about 5 calls x 50k tokens per subagent. |
auxiliary.*.model, delegation.model | deepseek-v4-flash-0731 (restored 07:30 UTC) | deepseek for bulk, glm where quality needs it | The 12:52 switch to glm rested on an 11x wrong rate and was reverted. Real cost is dominated by cache reads priced equally on both. |
| toolsets per task | 33 to 37 tools | task-scoped allowlist | 12.7K to 16.8K tokens of schema per call; video, image, TTS, cron tools never used. |
| MEMORY block | 2 stale August notes | purge | 90 percent of the block is irrelevant to every task run today. |
hard_stop_enabled | off | true | Makes budget exhaustion a clean stop instead of a truncated response. |
Source: agent_capabilities.md section 3, agent_behaviour.md section 5. Compression and dead keys: compression.summary_model and memory.flush_min_turns in the live config do nothing.
Multi-pipeline framework
Operating model
Keep the TUI for a human at the keyboard. Run each pipeline as a shell supervisor that issues onehermes -z oneshot per step with --usage-file, a per-pipeline model, toolset and skills, and a per-pipeline budget enforced at the proxy. Resume non-interactively with hermes chat -q ... --resume <session_id> --max-turns N, the only non-interactive resume path (cli.py:18417). Use cron only for delayed re-verification. Migrate to kanban lanes later: the dispatcher, rate-limit release and breaker already exist in kanban_db.py.| Mode | Cost visibility | Restartable | Parallelism | Failure isolation | Verdict |
|---|---|---|---|---|---|
| Interactive TUI tabs (today) | None inside Hermes; shared proxy cap | Manual resume, keystroke risk | One tab per goal | One 402 stops every tab | Humans only |
| Oneshot per step, supervised | --usage-file per step plus tagged ledger | Resume same session by id | One process per pipeline | Per-pipeline key and budget | Recommended now |
| Cron in gateway | Per job model and toolset, no turn cap or timeout | Reruns on schedule | Scheduler-bound | Shared | Delayed re-verification only |
| Kanban lanes | Per-card runtime and goal budgets, no USD cap | Built-in retries and breaker | Concurrency caps | Per lane | Target after the runner works |
Per-pipeline budgets without touching Hermes
Authorization: Bearer $NOUS_API_KEY. The proxy ignores that header today. A 30-line change makes the bearer value the budget bucket: launch pipeline A with NOUS_API_KEY=pl-siteA, pipeline B with pl-siteB, map each to a USD ceiling in policy.json, and the shared cap stops taking every tab down at once (25 denies in one hour today).Runner skeleton
pipelines/<name>/steps.txt one prompt per line, in order
pipelines/<name>/state.json session_id, step index, spend
runner.sh <name>:
for step in steps:
NOUS_API_KEY=pl-<name> hermes -z "$step" -m z-ai/glm-5.3-flash \
--provider custom:nous-api -t terminal,read_file,write_file,patch \
--usage-file pipelines/<name>/usage-$i.json --yolo
classify from the ledger, not the exit code (-z exits 0 on a partial run, oneshot.py:283):
402 -> poll state.json until headroom, resume same session_id
502/503 -> jittered exponential backoff, cap 10 min, 6 tries
gate fail -> stop, write BLOCKED.md
launchd: one KeepAlive=false agent per pipeline, StartInterval or manual kick
Cost design
- Context hygiene:
compression.threshold_tokens: 48000 to 60000,proactive_prune_tokens: 24000 to 40000,tool_output.max_bytes: 20000,terminal.timeout: 120, fresh oneshot per step so no step inherits another's context. Compaction fired at most 4 times in 1,940 calls today because the threshold applies to a 1,048,576-token detected deepseek window and a guessed 202,752 glm window. - Routing: at catalog rates deepseek-v4-flash-0731 is the cheaper producer and glm the dearer one; pick per step on quality, since cache reads cost the same. No local model exists (ollama is not installed; the cache file lists Ollama Cloud names).
- Subagents:
delegation.max_iterations: 25with a deliverable-first contract,max_concurrent_children: 2,child_timeout_seconds: 900. - Caching: verified at the portal. 97.5 percent of prompt tokens are cache hits billed at $0.01 per million; the proxy records cached_tokens since 07:23 UTC.
- Projection in real dollars: about $0.40/h today to about $0.22/h with a 40k context ceiling (median prompt 67.6k to about 30k), five sessions running.
Quality design
- Gates between steps: deliverable manifest exists, verification_evidence.db shows a full build and test after the last edit, humanize scan with a known-bad control, served-vs-dist byte compare, per-step spend cap.
- Goal contract in Hermes's own /goal vocabulary: explicit deliverable paths, "done" defined as files plus passing gates, never as prose.
- Adversarial reviewer step on deepseek returning CONFIRMED, REFUTED or PLAUSIBLE per claim; the human publish gate cannot be satisfied by quoting the goal text.
Source: agent_framework.md, which contains the full runner and the verification_evidence.db schema. Framework rating: 34/100 as run today, about 75 after the eight S and M backlog items (roughly two working days), past 85 with kanban lanes. The capabilities lane rated the framework's fitness at 54 on configuration alone; the two figures bracket the same gap.
Settings review, 07.45 UTC
Every live key against the source default and today's log
~/hermes-reports/config.recommended.yaml. Nothing in this section has been applied.Change these
| Key | Live | Default | Recommended | Evidence today |
|---|---|---|---|---|
compression.threshold_tokens | absent | None | 48000 | Hermes hardcodes glm at 202,752 tokens (55 log lines "hardcoded context length") and floors compaction at 75 percent, so it fired once, at 153,095 tokens, and that compaction call itself died on the 402. Average context per call 83k to 103k. |
compression.proactive_prune_tokens | 0 (off) | 0 | 24000 | Deterministic prune of old tool results, no model call. The parent's three largest tool results were its own prompt files, 22.5k, 16.2k and 15.8k chars. |
compression.protect_last_n | 20 | 20 | 10 | Twenty protected messages at 2 to 3k tokens each is 50k that compaction can never touch. |
compression.summary_model | deepseek | - | delete | Dead key. The live setting is auxiliary.compression.model. |
agent.max_turns | 150 | 500 | 40 | One turn exhausted 150/150 today; the goal loop re-enters anyway, so a low per-turn cap costs nothing and bounds runaway turns. |
terminal.timeout | 180 | 180 | 120, plus background=true for long jobs | 25 commands ended in exit 124 today, three of them 180 s polls on one script and one 600 s replay. The tool already refuses nohup and ampersand wrappers (15 errors) and points to background=true. |
tool_output.max_bytes | absent | 50,000 chars | 20000 | An 11.4k-char sp stdout and a 51k-char file read were inlined into context. |
tool_loop_guardrails.hard_stop_enabled | false | false | true | Warnings only. The same screenshot task ran 4 times with the same recipe; files were re-read 4 to 6 times per task. |
delegation.max_iterations | absent (50) | 50 | 30, with a deliverable-first contract | 9 of 22 completed delegations ended on max_iterations; all 6 genuine cap hits left no file. |
delegation.max_concurrent_children | absent (3) | 3 | 2 | The 05:25 to 05:40 fan-out had 10 requests in flight and 59 calls a minute against a 400 rpm, 4M tpm subscription limit. |
delegation.child_timeout_seconds | absent (0) | 0 = none | 900 | No wall clock on subagents. Median 7 min, max 39; results then sat 5 to 38 min before delivery. |
memory.nudge_interval | 10 | - | 30 | Each nudge spawns a 16-iteration review fork billed like a turn. |
skills.creation_nudge_interval | 15 | - | 40 | 34 skill_manage errors today, most "description is 111 to 208 chars, budget 60". 12 of the 30 skills on disk were created today by the curator. |
agent.verify_on_stop | absent (auto = on) | auto | false in pipeline profiles | 17 verification_required finishes; full npm test runs of 243 to 245 s after JSON and HTML edits. |
platform_toolsets.cli | 17 toolsets | - | 10 | 12.7k to 16.8k tokens of tool schema per call. bfl, image_gen, tts, vision, cronjob and session_search were never called; the registry already reports computer_use and browser CDP unavailable (8 check_fn lines per start). |
agent.personalities | 13 personas, 40 lines | same | delete | Installer defaults including emoji personas. Never selected. Clutter in a file that is also the security policy. |
updates.pre_update_backup | false | - | true | hermes update would overwrite a hand-tuned config and 14 agent-written skills without a copy. |
~/.hermes/memories/MEMORY.md | 1,836 bytes | - | purge | Two notes from Aug 7 and Aug 26 (a Gumroad puller, an intel dashboard menu fix) injected into every prompt today; zero relevance to any task run. |
Verify these
| Key | Live | Question |
|---|---|---|
web.backend: firecrawl, browser.cloud_provider: browser-use, image_gen.use_gateway | gateway on | The portal account reports tool_access.enabled: false. 251 firecrawl log lines today show searches being issued; confirm they succeed on a direct key and are not silently empty, or the research steps are running blind. |
fallback_providers: [], fallback_model commented | none | Single upstream: a Nous 502 or 503 stalls every session (14 x 502 today). A fallback to the other allowed model on the same provider keeps the directive and survives one lane's outage. |
agent.relay_runtime | - | "Hermes Relay turn finalization failed" logged 5 times today across 4 sessions. Harmless if Relay is unused; otherwise a lost handoff. |
security.tirith_enabled | commented | tirith is not installed. With approvals off, the pre-exec scanner is the only remaining command gate besides the hardline list. |
Leave as they are
agent.reasoning_overrides: deepseek none, glm low. Measured earlier: deepseek returns empty content without it, glm refuses none.approvals.mode: offfor unattended pipelines; the hardline deny list (config, env, credentials) still applies and blocked 5 commands today.prompt_caching.cache_ttl: a no-op on the custom endpoint, but harmless. Caching is server-side at Nous and already gives the 97.5 percent hit rate.database.journal_mode: wal,code_execution.max_tool_calls: 50,auxiliary.free_only: true(only restricts an OpenRouter fallback that is not configured),session_reset.mode: none.streaming.enabled: falsewithdisplay.streaming: true: the proxy buffers streams anyway, so this only affects the TUI.
Sources: ~/.hermes/config.yaml (299 lines), hermes_cli/config_defaults.py, today's agent.log (warning classes: 34 skill_manage, 15 terminal wrapper refusals, 13 non-retryable 402s, 8 registry check_fn per start, 5 hardline blocks, 5 relay finalization), the sessions DB figures above, and the earlier request-dump measurements. The "429" count in the log is a false match on token totals, not rate limiting.
Implementation brief, 08.00 UTC
~/hermes-reports/HERMES-IMPLEMENTATION-PLAN.md, with five verified workstream files, two config helpers and a read-only measure.py under ~/hermes-reports/plan/. Four waves, each with acceptance rows and a stop rule:
- Context hygiene (config only, live): compression threshold 48k, prune 24k, tool output 20k, terminal timeout 120, max_turns 40, memory purge. Gate: prompt p90 under 60k within 2 hours, cache share stays above 90 percent.
- Proxy hardening (one restart): body drain, sweep terminator on /version (never a models key on /api/tags, which would make Hermes treat the proxy as Ollama), fail-closed state with ledger anchors, 80 percent warning headers, degrade instead of 402 with reasoning override rewrite, /stats.
- Agent behaviour : delegation 30/2/900, curator opt-in, toolset trim, a pre-spawn budget hook on the proxy's existing /healthz, a delegation-contract skill, verify_on_stop off per profile, then vendor patches saved as diffs: true exit reasons, busy-parent result drain (the 15 minute delivery lag is the parent's turn length), and the pipeline's own gate truthfulness fixes.
- Budgets and runner : bearer-keyed buckets, portal-truth guard on total_usable_credits, live streaming relay, and a per-profile runner on
hermes -p X chat -q(oneshot ignores skills and exits 0 on partial runs), migrating one site pipeline first.
Before and after
How to measure
Wave 1 and wave 3 config keys were applied to ~/.hermes/config.yaml at 2026-09-17T08:20:44Z (recorded in plan/apply_time_wave1.txt; backup config.yaml.bak-tune-20260917-152059). Hermes copies most keys at process start, so a session that was running at 08:20:44Z keeps the old config until it is restarted; only sessions whose started_at is after the apply time, and their subagents, measure the change. measure.py --sessions-started-after applies exactly that filter to the sessions database and agent.log (ledger and proxy rows carry no session id and stay time-window only).
| Before, 2026-09-17 00:00 to 08:20:44 UTC (plan/before_full_day.txt) | Value | Target after |
|---|---|---|
| ledger calls | 2,777 | at least 500 in the after window before judging |
| prompt tokens p50 / p90 / max | 65,900 / 117,604 / 154,144 | p90 under 60,000 (C1) |
| calls over 60k / over 100k prompt | 1,557 (56.1 percent) / 636 | under 10 percent / 0 |
| cache hit share (rows with cached_tokens) | 95.85 percent | stays at or above 90 percent (C3, the stop rule for prune) |
| compression started per 1,000 calls | 2.2 (preflight threshold seen: 152,064) | 2 to 20 and the threshold seen must read 48,000 (C2, C1c) |
| real cost total / per call / per CLI session-hour | $2.3041 / $0.00083 / $0.1851 | under $0.00060 per call, under $0.12 per session-hour |
| cache re-reads share of real cost | 78.9 percent | under 60 percent |
| delegation tasks / max_iterations / share | 32 / 13 / 40.6 percent | under 15 percent over at least 20 tasks (D1) |
| 402 deaths labelled max_iterations | 4 | 0 (D2) |
| delivery lag p50 / max, min | 15.2 / 85.0 | p50 under 3 (D3) |
| exit-124 terminal results / per CLI session-hour | 25 / 2.01 | under 3 per hour (C4); sleep seconds 5,009 should fall |
| skill_manage errors (agent.log) | 50 (34 description too long, 9 curator refused) | 0 (S1) |
| first API call prompt tokens p50 (n=76) | 19,433 | under 12,000 after the toolset trim (T1) |
| proxy: 400 bad-syntax / sweep 404 per hour / 402 denies | 129 / 306 / 25 | 0 / under 5 / 0 while credits exceed the reserve (P-series, wave 2) |
| MEMORY.md bytes | 2,079 | under 400 (purged to 0 at 08:21Z) |
After new sessions have worked for at least two hours (or 500 calls), run one command:
bash ~/hermes-reports/plan/after.sh
It reads the apply time, runs measure.py --json for the before window (00:00 UTC to the apply time) and for the after window (apply time to now, sessions started after the apply only), runs compare.py, writes plan/after_<ts>.txt and plan/after_<ts>.json, and appends a dated row to plan/RESULTS.md. The first lines state how many post-apply sessions and ledger calls it found; with zero it says NO DATA YET instead of scoring. Nothing is written under ~/.hermes; the sessions database is opened read-only.
What PASS looks like. The compare table has twelve rows with a weight each (C1 15, C3 12, D1 12, T1 10, C2 8, C4 8, P3 8, D2 6, D3 6, S1 6, P1 5, P2 4). PASS is a weighted score of 80 or more with no FAIL row, over at least half the weight evaluable: p90 prompt under 60k, 2 to 20 compactions per 1,000 calls with the preflight threshold reading 48,000, cache hit share at or above 90 percent, fewer than 3 exit-124 per session-hour, max_iterations under 15 percent over at least 20 tasks, zero 402 deaths labelled max_iterations, delivery lag p50 under 3 minutes, zero skill_manage errors, first-call prompt under 12k, and on the proxy zero bad-syntax 400s, under 5 sweep 404s an hour and zero 402s while credits exceed the reserve. The cost block should show real cost per CLI session-hour moving from $0.1851 towards $0.12 and the cache re-read share falling below 60 percent. Wave 2 (proxy) and wave 4 (buckets) rows stay FAIL until those waves ship; that is expected and does not block wave 1 and 3 sign-off.
Restart note: the five TUI sessions live at apply time do not pick up compression.*, terminal.timeout, tool_output.max_bytes or the toolset trim until they exit and start again (/new does not reload config). Until then the ledger mixes old and new sessions; after.sh prints how many pre-apply sessions were still active. Kit: plan/measure.py, plan/compare.py, plan/after.sh, snapshots plan/before_full_day.txt and plan/before_full_day.json.
Implementation results, 08.45 UTC
All four waves shipped
What the operator does now
Every Hermes session from before 08:20 UTC has finished its goal and sits idle; all five carry the old config in memory. Start new sessions (hermes, or hermes -p pl-site-a for the first pipeline) and let them work for two hours. Then run:
bash ~/hermes-reports/plan/after.shIt prints the before-and-after table with PASS or FAIL per criterion and a score. The apply time it compares against is 2026-09-17T08:20:44Z. Until then this section is the delivery record, not the outcome.
Shipped, by wave
| Wave | Shipped | Evidence | Rollback |
|---|---|---|---|
| 1 Context hygiene | compression threshold 48k, prune 24k at 4k chars, protect_last_n 10, dead summary_model removed, tool output 20k, terminal timeout 120, max_turns 40, personas removed, pre-update backup on, approvals mode quoted; MEMORY.md purged (5 stale entries) | Loader readback of every key; purge under the memory lock | config.yaml.bak-tune-20260917-152059; memories/MEMORY.md.bak.* |
| 2 Proxy | proxy.py 571 to 1,485 lines: body drain and sweep terminator, fail-closed state with ledger anchors, 80 percent warning headers and warn.log, degrade to the other model on per-model caps with reasoning rewrite, bearer-keyed buckets (pl-smoke 0.50, pl-site-a 3.00), portal-truth guard with 1.10 reserve, live streaming with stall cut and fail-high partial pricing, --refresh-rates, /stats, SIGHUP log reopen, newsyslog conf | 58 zero-spend checks pass; 3 real 5-token probes; restart gap 0.27 s; first portal read: drift 0.011 vs ledger | proxy.py / policy.json / state.json .bak-impl-20260917-083237 plus kickstart |
| 3 Behaviour | delegation 30 / 2 / 900 s, curator off, memory nudge 30, toolsets trimmed to 10, budget-gate hook on the proxy health endpoint, delegation-contract skill plus SOUL.md rule; vendor patches 01-05: true exit reasons, child deliverable contract, busy-parent result drain through the steer rail, curator 60-char rule, verify-on-stop scoped to code files | Hook blocks at under $1 headroom in a dry test; delegate tests 70 pass; drain tests 9 pass; patches re-apply byte-identical onto a clean export | plan/patches/*.diff and REAPPLY.md; git checkout of the vendor tree |
| 3 Pipeline gates | Cached-stage issues counted, any ok:false stage forces FAIL, evidence counts from real stage outputs; L12 sign-off needs a person, an operator token in a file the pipeline only reads, and no autonomy wording; deployed:true only from a recorded wrangler output hash plus served-vs-dist compare; PHASE-4 docs rewritten to stop at the gate | 197 of 197 existing tests plus 18 gate and 14 status checks; today's real approval.json is now rejected | *.bak-gate-* beside each file |
| 4 Runner | Profiles pl-smoke and pl-site-a (own state.db, config, bearer key, verify-on-stop off); ~/hermes-pipelines with runner.sh, cost.py, classify.py, prompts, launchd template (not loaded), README and MIGRATION | Smoke: 2 steps, 47 s, $0.001211 for 5 calls, root config sha unchanged | Delete the two profile dirs and ~/hermes-pipelines |
| Measure | measure.py with sessions-started-after and bucket split, compare.py with 12 weighted criteria, after.sh one-command readout, before snapshots saved | Dry run on an empty after window exits 0 and says so; synthetic split proves the scoring path | - |
Skipped or deferred, stated plainly
- newsyslog rotation needs one
sudo cpof~/.hermes/modellock/newsyslog.d-hermes-modellock.confinto /etc/newsyslog.d; not run. - Per-task iteration budget and a toolsets argument on delegate_task (plan 1d, 3b) were not built; the config-level caps and the trimmed toolset cover the measured problem.
- Kanban lanes are documented in MIGRATION.md but not enabled; the runner comes first by design.
- The first site pipeline's step file was written from the plan because every phase document on the iCloud Desktop was dataless during the build; run its first gate by hand once before trusting it.
--usage-fileturned out to be oneshot-only, so chat-mode steps get their usage synthesized from the profile's session row plus the ledger window; it matched the ledger exactly on the smoke run.- Catalog rate refresh is a manual command, not scheduled.
Why 88 and not 95
Sources: plan/impl/impl_proxy.md, impl_vendor.md, impl_pipeline.md, impl_runner.md, impl_measure.md; backups named per row. Live proxy stats at http://127.0.0.1:8788/stats.
First run on the new setup
Deepvalueradar.com, 08.57 to 10.49 UTC
What ran
Session 20260917_155714_88e201, in a fresh Terminal window, goal./sp https://deepvalueradar.com/ run and finish, started 37 minutes after the config apply in a fresh process. Its log shows the new setup in force: first call 11,812 prompt tokens (16k to 20k before the toolset trim), budget hook registered at start, a hung ./sp status cut at exactly 120 s, and preflight compression firing twice at 50,949 and 51,784 tokens against the 48,000 threshold, compacting 172 messages to 96 in 14 s. Before today compaction fired only at 152k. The site was not registered in the pipeline; the agent registered it (site key deepvalueradar, project ~/deepvalueradar) and drove all 15 orchestrator stages.Timeline
Status: BLOCKED (1 critical) · FAIL: 10-human-check · warnings: 4.Stage outcomes
| Stage | Result | Note |
|---|---|---|
| 01 intake | ok | 8 brief files, vertical Finance, deep-value equity research |
| 02 deep research | ok after re-run | 3 sources, 3 claims; first pass was empty by refusal, not fabrication |
| 03 DataForSEO validate | ok | 1 buildable: "margin of safety formula for stocks", volume 10, difficulty 37. Thin target. |
| 04 source validate | ok | 3 live fields validated, e.g. OTEX price from the keyless Yahoo chart API, 0.00 percent spread, 0.0 h old |
| 05 generate | ok after re-run | 1 page from 3 claims and 3 fields |
| 06 factcheck | ok with warning | 3 citations checked; 2 SEC XBRL fetches failed; worker circuit opened after 2 charged failures; L8 linkage audit NOT performed |
| 07 QA, 07b helpfulness, 07c freshness, 07d footprint | ok | 1 page QA'd; H1 gain score 0.65 (pass mark 0.5), unique element: interactive calculator; footprint clean against 23 siblings |
| 08 harden | ok | L9, L10, L11 plus humanize gate on 1 page; independently reviewed: 0 |
| 09 build-deploy | dry run | 1,334 tokens resolved, 0 unresolved; built false, deployed false, placeholder staging URL |
| 10 human check | FAIL by design | awaiting human sign-off; approved false; the agent never wrote approval.json |
| 11 index-monitor, 12 report | skipped / ok | Report status BLOCKED, failed layers L7 and L12, header agrees with the stage list |
The page it built
~/hermes-reports/plan/impl/dvr-page-draft-2026-09-17.html. Not deployed anywhere.The agent's own report, checked
Cost and efficiency, this session against the morning
| Metric | Another site in the same portfolio, 05:38, old config | deepvalueradar, 08:57, new config |
|---|---|---|
| Orchestration calls | 315 | 160 |
| Wall time | 135 min | 112 min |
| Prompt tokens p50 / p90 / max | 89,965 / 126,091 / 146,711 | 32,707 / 40,254 / 43,230 |
| Cache hit share | 98.3% | 90.5% |
| Real Nous cost | $0.311 | $0.087 |
| Delegations | 5 | 0 |
| Worker (OpenRouter broker) jobs / cost | 26 / $0.0299 | 25 / $0.0203 |
| Compression events | 1, at 153k | 2, at 51k and 52k |
| End state | self-approved L12, deployed on 1 source | BLOCKED at L12, nothing deployed |
Rubric score
| Dimension | Weight | Score | Why |
|---|---|---|---|
| Evidence quality | 25 | 14 | Real SEC URLs, and the adversarial pass caught two wrong CIKs from research, but one of them (0000832941) is still cited in the final page while the agent report says both were replaced (5/8); target qualified on real DataForSEO calls with the balance quoted, volume 10 (5/6); H1 0.65 and H4 0.75 quoted (4/6); no three-validator consensus, independently reviewed 0 (0/5) |
| Page build | 20 | 13 | Tool-led with a live first-party field and 0 unresolved tokens (6/6); humanize checked but no known-bad control shown (3/5); QA once, no parity diff evidenced (2/5); no visible date or byline (2/4) |
| Gate truthfulness | 25 | 19 | Stage JSON matches REPORT.md, human check ok false, L12 NOT approved stated, no approval.json, production NOT DEPLOYED, token file untouched (8/8); state steps carry evidence paths that exist and mostly support their notes, minus the half-true CIK claim (7/10); no gate weakened, but the TUI summary says "staged deploy done" and the report says "staged URL emitted" for a dry run whose 09 JSON records built false and deployed false and whose URL is a placeholder answering 302, and the dwell-floor shortfall is not declared (4/7) |
| Efficiency | 15 | 11 | 160 calls against 315 and no delegations (4/6, 4/4); 10 minutes lost to the evicted-library grep loop, three early 120 s timeouts and one 40-iteration cap hit; time mostly in orchestrator and broker, not retries (3/5) |
| Completeness | 15 | 9 | PAGE_URL and TARGET_QUERY recorded, but PAGE_URL is the placeholder (2/5); the phase 1 HTML report exists with real gate figures, the second report does not (3/5); operator received the staged URL, gate scores, the blocker and a numeric summary (4/5) |
| Total | 100 | 66 | No hard cap triggered: no agent-authored approval, no deploy, no override, header agrees with stages |
Before and after, measured
after.sh at 10:36 UTC scored the after window 55 of 100 over 76 percent evaluable weight, with C1 (p90 prompt) and T1 (first-call prompt) failing. That result is contaminated: the window's two "post-apply sessions" are this one and a second session, which started at 09:03 inside a Hermes process launched at 06:57 and therefore runs the old config; its 119k prompts are what fail C1 and T1. On the deepvalueradar session alone, the only fresh-process session so far, C1 passes (p90 38.9k under 60k), T1 passes (first call 11.8k under 12k), C2 passes (2 compactions in 133 calls), C3 passes (97 percent cache), S1 passes (0 skill errors) and P3 passes (0 stranded 402s). The proxy rows P1 and P2 improved, 129 to 4 bad-syntax 400s counted over the 8.35 h and 2.26 h windows, and 306 to 23 sweep 404s an hour, but did not reach zero and five. Follow-up for measure.py: classify sessions by process start, not session start; the "shell hook registered" log line marks a fresh process.Biggest insights from this run
- The tuned config works exactly as designed. Compression fired at 51k instead of 153k, the median prompt fell from 89,965 to 32,707, and the session cost $0.087 to the human gate against $0.311 for the morning's run, on a different site.
- The integrity gate held under pressure. The goal text says "run until you can deploy, no exceptions". The run ended BLOCKED at the human check, no approval file was written, and the token file is still zero bytes. The morning run had self-approved and deployed.
- The pipeline's biggest defect is an unset environment variable. Without PSEO_PROJECT the live research and generate workers refuse every call, the stages pass as ok with a warning, and the report says clean over an empty page. Both runs today hit it. Export it in the sp launcher and make an empty bundle a FAIL.
- iCloud eviction cost the agent ten minutes and would have cost more. grep over an evicted tree returns nothing without an error. The pipeline should not live on the Desktop, or the launcher should hydrate it first.
- Evidence is the weak dimension, not tooling. A 10-search-a-month keyword, three trivially true SEC claims, two unfetchable citations and no independent review produced a page that passes every automated gate. The gates measure process, not value.
- Steer is the right lever. Two non-interrupting steers moved the run past a stall and away from the token file without breaking its turn; earlier today a typed message killed a session.
Sources: agent.log session lines, runs/deepvalueradar/2026-09-17/*.json and REPORT.md, broker /status, proxy /stats, plan/after_20260917T103627Z.txt, dvr_pipeline_doc.md rubric. The two steers sent are quoted in plan/impl/dvr_monitor_notes.md.
Publication, 11.30 to 11.45 UTC
What changed on the live site after the autonomous run, and how it got there
Sequence
Before and after on the live site
| Item | 09.01, before the run | 11.45, after |
|---|---|---|
| New page | none | Margin of safety formula for stocks, with a working calculator. v2 in the site layout: 853 words, 2 calculator inputs, 3 SEC citations, live OTEX price field, byline, date, canonical, index robots |
| Sitemap URLs | 49 | 49 |
| Homepage bytes | 1,125,585 | 1,125,589 |
| Deploy record | none | 13-prod-deploy.json with tool output hash and bytesMatch true |
| Pipeline report | - | Status clean, warnings 2, and still "production NOT DEPLOYED by this pipeline", which is accurate because stage 13 is a by-hand step |
Final state at 11.55 UTC, terminal idle
The agent rebuilt the page as v2 inside the site layout and redeployed. Live now: title with the site suffix, canonical to itself, robots index and follow, Open Graph tags, byline By Michael Lip, Updated 2026-09-17, 853 words, the two-input calculator, and citations to CIKs 1002638, 1429937 and 832988. The flagged CIK 0000832941 is gone. Two independent validators ran on the page and passed load, arithmetic (22.0 percent, $6.60 discount, 28 percent markup recomputed) and factual checks, with OTEX 23.40 confirmed through a second source after Yahoo rate-limited. The page is still absent from the sitemap, which is the one edit left before indexing.
Whole session including publication and both validators: 286 calls over 188 minutes, prompt p50 34,815 and p90 42,441 tokens, cache share 90.4 percent, 6 compactions, real Nous cost $0.168. The morning's session on another site in the same portfolio, which published nothing usable, cost $0.311 over 315 calls.
Two things this proved
The gate that was added this morning is instruction-level, not system-level. It held through two refusals while the agent was unprompted, and it opened the moment a human said deploy. The record it produced is honest about that, since the reason field quotes your words. If you want the token to be something only a person can create, the token file has to live under a different user or a keychain, because today the agent runs as the same uid and can write it.
The pipeline had a real gap that only a live attempt could expose. It qualifies, drafts, hardens and approves a page but never writes it into the site before deploying. The agent diagnosed that in one pass and finished by hand using the pipeline's own stage 13 procedure, which is the right way to close it. The permanent fix is a materialise step between 08 and 09.
The page is live but unlisted. After the v2 rebuild it has its byline, date and corrected citations; adding it to the sitemap is the one edit left before it should be indexed.
Backlog, ranked
| # | Item | Effort | Gain |
|---|---|---|---|
| 0 | DONE 07:23 UTC: cache-aware pricing in the proxy, catalog rates, state rebased, caps in real dollars, aux models restored | S | Ledger now tracks the portal within about 1 percent; no more stops on phantom spend |
| 0b | Proxy polls /api/oauth/account every few minutes and caps on total_usable_credits, with 80 percent warning and degrade instead of 402 | S | The guard follows the real balance, including top-ups |
| 1 | Set compression threshold 48k, prune 24k, tool output 20k in config.yaml | S | About $13.80/day measured, more with fewer compaction misses |
| 2 | Distinct exit reasons for 402 and 502 in delegate_tool, plus a pre-spawn read of policy.json remaining budget | S | Stops mislabelled deaths and redo work (55 min lost today) |
| 3 | Proxy: fail-closed state, drain-body fix, 80 percent warning header, degrade-to-glm instead of 402 | M | No more stranded sessions; the central fail-open path closed |
| 4 | Proxy: per-pipeline budgets keyed by API key suffix or header, /stats endpoint | M | Each pipeline has its own ceiling and a live readout |
| 5 | Delegation contract: one deliverable path, write by iteration 40, task-scoped toolsets | S | Cuts the 41 percent max_iterations rate and 16.8K schema per call |
| 6 | Pipeline runner: per-profile hermes -p X -z ... --usage-file under launchd with resume on 402 and backoff on 502 | M | Restartable, cost-visible, isolated pipelines instead of TUI tabs |
| 7 | Proxy: live streaming with stall timeout and per-model timeout; price partial responses | M | Ends the 120s kills and the $0 records for billed calls |
| 8 | Truthful gates: REPORT status must reflect the human gate; publish gate cannot be self-approved from goal text; deploy JSON updated before "deployed" | S | Removes the one integrity failure found |
| 9 | Tool layer timeout policy: no command over 120s without detaching; search_files bounded; cache file reads per task | M | Recovers most of the 2.8 hours of terminal time |
| 10 | Populate session cost from the proxy ledger; purge MEMORY; make curator and verify-on-stop opt-in per profile | S | Cost visibility inside Hermes; fewer hidden turns |
Sources
- agent_cost.md, agent_sessions.md, agent_behaviour.md, agent_capabilities.md, agent_proxy.md, agent_framework.md in ~/hermes-reports (copied from the audit scratchpad, with the SQL and python used).
- Live data: ~/.hermes/modellock/ledger.jsonl, state.json, proxy.log, policy.json; ~/.hermes/state.db (read-only); ~/.hermes/sessions/request_dump_*.json; ~/.hermes/cache/delegation; Terminal window contents via AppleScript.
- Portal truth:
https://portal.nousresearch.com/api/oauth/account(Hermes's own account client, an undocumented endpoint that may change) with the Hermes OAuth token from auth.json, andhttps://inference-api.nousresearch.com/v1/modelspricing. Two probe calls per model confirmed the usage fields. - Changes made today: caps raised three times on the mispriced ledger, approvals.mode off, auxiliary models moved to glm at 12:52 and restored to deepseek at 07:30 UTC, proxy repriced and restarted at 07:23 UTC, state rebased, caps set in real dollars. Backups sit beside each file.
~/hermes-reports/HERMES-IMPLEMENTATION-PLAN.md, the instruction for a future Fable session. Workstream detail, helpers and measure.py are under ~/hermes-reports/plan/. Nothing here has been applied.Hermes tuning
Implementation brief for a Fable session
Written 2026-09-17 08:00 UTC from the audit at ~/hermes-reports/hermes-audit-2026-09-17.html.
This file is the instruction. The five workstream files under ~/hermes-reports/plan/ hold the
verified line references, code and tests; read the one for the wave you are executing before touching anything.
0. Ground truth you must not re-derive
- Nous bills through the Portal at catalog rates: glm-5.3-flash 0.075 / 0.25 per M, deepseek-v4-flash-0731 0.04 / 0.10 per M, cache read 0.01 on both. The old ledger was 12x to 17x high. Real spend today: $2.11 for 2,549 calls. Real burn: $0.40 per wall-clock hour, $0.18 per session-hour.
- 96.5 percent of prompt tokens are cache hits. 79.8 percent of the real bill is cache re-reads of context (p50 prompt 66k, p90 118k). Context size is the cost lever; model choice is not.
- Every 402 today came from the local proxy
~/.hermes/modellockon phantom spend, never from Nous. The proxy was repriced at 07:23 UTC (cache-awareprice(),cached_tokensin the ledger, caps rebased to real dollars: total 34, daily 10). Policy hot-reloads per request; proxy.py changes needlaunchctl kickstart -k gui/$(id -u)/com.<user>.hermes-modellock. - Truth endpoint for the credit balance:
GET https://portal.nousresearch.com/api/oauth/account, the undocumented endpoint Hermes's own account client uses, so it may change. Send the bearer from~/.hermes/auth.jsonproviders.nous.access_token. Hermes rotates it hourly; never call the refresh endpoint yourself, because the comments in Hermes's own auth code say refresh tokens are single-use and revoke the session. - Hermes config:
~/.hermes/config.yaml(299 lines,_config_version: 33). Most keys are copied at process start; onlydelegation.*,agent.verify_on_stop,updates.pre_update_backupand the approval policy are hot-read./newdoes not reload config. - Five TUI sessions may be live in Terminal.app. Never send keystrokes to them (a typed string arrived as Ctrl-C once and killed a session). Never drive Chrome by "front window": another automation navigates it; use DevTools targets on the dedicated remote-debugging port, addressed by target id.
1. Rules for the implementing agent
- Back up before every write:
cp <file> <file>.bak-<tag>-$(date +%Y%m%d-%H%M%S). Vendor files under~/.hermes/hermes-agentget a.diffsaved to~/hermes-reports/plan/patches/becausehermes updatestashes local edits. - Read-only on
~/.hermes/state.db(?mode=ro). Never runhermesinteractively; the only permitted Hermes invocations are the smoke tests named in wave 4, which cost under $0.01. - No paid API beyond: the two 5-token proxy probes in the proxy acceptance script, the portal account GET, and the catalog GET. State the call budget before any subagent is spawned; forbid paid APIs in its brief.
- Measure before and after every wave with
python3 ~/hermes-reports/plan/measure.py --since <apply-time-ISO>; the before snapshot isplan/baseline_run2.txt. A wave passes only when its acceptance rows pass; on the stop rule, roll back that wave and record why. - Apply waves in order. Waves 1 and 3 are config-only and are what stretch the credit runway; do not let the proxy or runner builds delay them.
- Report in the format of section 6, with numbers from measure.py, never from memory.
2. Wave 1
Context hygiene (config.yaml, 10 min apply, 2 h measure, runs with goals live)
Detail: plan/plan_config.md steps 1 to 8; helper plan/apply_tuning.py (ruamel round-trip, keeps comments) or plan/setkey.py key value.
| Key | Set to | Why (measured) |
|---|---|---|
| compression.threshold_tokens | 48000 | floor put compaction at 152,064 on the hardcoded 202,752 window; fired 5 times all day |
| compression.proactive_prune_tokens | 24000 | deterministic prune, no model call |
| compression.proactive_prune_min_result_chars | 4000 | 51k-char reads inlined |
| compression.protect_last_n | 10 | 20 protected messages is 50k untouchable |
| compression.summary_model | delete | dead key since migration v17 |
| tool_output.max_bytes | 20000 | default 50,000 chars |
| terminal.timeout | 120 | 23 exit-124 today; reaches the tool only via TERMINAL_TIMEOUT env bridge at process start |
| agent.max_turns | 40 | 150/150 exhausted once; goal loop re-enters anyway |
| updates.pre_update_backup | true | 14 agent-written skills at stake |
| approvals.mode | 'off' quoted | bare off parses as boolean False; works by accident today |
Then purge ~/.hermes/memories/MEMORY.md under its flock (procedure in plan_config.md step 8; 1,836 bytes of August notes injected into every prompt).
Takes effect per session at its next start; do not restart live goals, let them finish. Acceptance (plan_acceptance.md C1 to C4, T3): prompt p90 under 60k within 2 h of the first new session, compaction 2 to 20 per 1,000 calls, cache-hit share stays above 90 percent (prune can break the prefix cache; that is the stop rule), exit-124 under 3 per hour.
3. Wave 2
Proxy hardening (proxy.py + policy.json, one restart in a 10 min quiet window, 4 h soak)
Detail: plan/plan_proxy.md, in this order, shipped as one restart.
- Step 0: env overrides MODELLOCK_ROOT / MODELLOCK_PORT / MODELLOCK_AUTH_JSON so the acceptance script runs a second instance on 8799 against a scratch copy.
- Drain the request body on unrecognised POSTs (the 94 "Bad request syntax" 400s a day) and terminate Hermes's provider sweep with a 200 on
/versioncarrying aversionkey. Do NOT answer/api/tagswith amodelskey: Hermes would classify the proxy as Ollama and start/api/showprobes. - Fail-closed
load_state(): write a ledgeranchorevent at startup and on SIGHUP, rebuild from the last anchor on a corrupt state.json, 503 when no anchor exists, never zeros. Run--anchoronce at deploy because the 07:30 rebase left no anchor row. - Warning headers
X-ModelLock-Remaining-USDandX-ModelLock-Warningat 80 percent of any cap, plus a WARN line to~/.hermes/modellock/warn.log. - Degrade instead of 402 when only a per-model cap is hit: rewrite to the other allowed model and overwrite the
reasoningfield with the target's override (glm rejectseffort: nonewith HTTP 400). Global and bucket caps still 402. /statsJSON endpoint and SIGHUP log reopen (launchd holds the fd; a plain newsyslog rename leaves an empty file).
Acceptance (P1 to P7): zero bad-syntax 400s, sweep 404s per hour under 5 excluding /api/v1/models, state.json survives a deliberate corruption test on the scratch instance, two real 5-token calls cost under $0.0005 each with cached_tokens present. Rollback: restore proxy.py.bak-* and kickstart.
4. Wave 3
Agent behaviour (config plus one hook, one working day to measure 20 tasks)
Detail: plan/plan_delegation.md. Config and hook first, vendor patches last.
delegation.max_iterations: 30,max_concurrent_children: 2,child_timeout_seconds: 900(hot-read).skills.creation_nudge_interval: 0(curator becomes opt-in; 48 skill_manage errors today were 60-char description refusals) andmemory.nudge_interval: 30.platform_toolsets.cli: drop bfl, image_gen, tts, computer_use, session_search (about 6k tokens per call;delegate_taskhas no toolsets argument, so this is the only lever). Needs per-tab restarts at goal boundaries.- Pre-spawn budget hook: a
pre_tool_callshell hook in config.yamlhooks:matched todelegate_taskthat GETshttp://127.0.0.1:8788/healthz(already servesspent_today_usdandcaps) and blocks the spawn under a reserve. Script text is in plan_delegation.md step 2. - Delegation contract as a skill named
delegation-contractthe parent pastes intocontext: deliverable path written by iteration N-10, budget stated every 10 iterations, no sleep polling. Surviveshermes update. agent.verify_on_stop: falsein pipeline profiles only (orHERMES_VERIFY_ON_STOP=0); 18 verification_required finishes and 243 s npm test runs after JSON edits.- Vendor patches, each saved as a .diff: exit reasons
billing_blocked/provider_error/max_iterations/incompleteintools/delegate_tool.py(the child already carriesfailure_reason: billing, never read); busy-parent drain of completed delegations through the steer rail (delivery lag p50 15 min is the parent's turn length); curator prompt states the 60-char limit. - Operator pipeline fixes in the operator's SEO pipeline repo: orchestrator.mjs must push cached-stage issues into
allIssues(REPORT.md said clean over anok:falsehuman check); L12 sign-off requires a person inbyand an operator-minted token, rejects autonomy wording;deployed:trueonly from a13-prod-deploy.jsonwritten from real wrangler output.
Acceptance (D1 to D4, S1, T1): max_iterations share under 15 percent over 20 tasks, zero 402 deaths labelled max_iterations, delivery lag p50 under 3 min, first-call prompt under 12k tokens, skill_manage errors zero.
5. Wave 4
Budgets, portal guard, runner (live, one pipeline per quiet hour)
Detail: plan/plan_proxy.md steps 5 to 9 and plan/plan_runner.md.
- Proxy buckets: the inbound bearer value (
NOUS_API_KEY=pl-<name>per profile) becomes the budget key;buckets:section in policy.json;bucketfield in the ledger. Unknown bearer falls intodefault. - Portal guard: every 50 calls or 5 minutes, curl
-K -(token off argv) to/api/oauth/account, store credits and member spend instate["portal"], cap ontotal_usable_credits - spent_since_read <= 1.10. On 401 or expiry fall back to local caps, log drift, never rebase automatically. - Live streaming relay: keep curl,
Popenwith-N --speed-limit 1 --speed-time <stall> --max-time <per-model>, price partials fail-high aspartial_estimate. - Runner at
~/hermes-pipelines/: usehermes -p pl-<name> chat -q "<step>" -Q --yolo --max-turns N --skills ... --usage-file ..., not-z(-zignores--skills, has a fixed 90-iteration cap and exits 0 on partial runs). Profiles viahermes profile create pl-<name> --clone --no-alias; each profile.envcarries its ownNOUS_API_KEY=pl-<name>andHERMES_VERIFY_ON_STOP=0. Classify outcomes from usage.json plus the ledger bucket, resume on 402 with--resume<session_id>, back off 30 to 600 s on 502, exit 75 for the launchd tick. Smoke test: the 2-step /tmp write-and-verify, about $0.002. - Migration order: the first site pipeline (4 phases map to steps), then the other site pipelines, then the two non-site pipelines as single-step pipelines. Kanban lanes afterwards:
hermes kanban create --assignee pl-<name> --max-runtime --max-retries, dispatcher in the gateway.
Acceptance (P4 to P9, R1 to R3): zero 402 while usable credits exceed the reserve, per-bucket cost visible in /stats, smoke pipeline completes twice with cost under $0.006, no session ever needs a keystroke to resume.
6. Report format when done (per wave)
- Applied: file, key or function, backup name.
- measure.py before and after: the ACCEPTANCE table rows for this wave, pass or fail each.
- Real spend during the wave from state.json and the portal account GET (two numbers, and the drift between them).
- Anything skipped and why. Anything that regressed and what was rolled back.
Projection to hold yourself to (plan_acceptance.md section 5): $2.11 a day today; $1.33 to $1.64 after waves 1 and 2; about $1.12 after waves 3 and 4. Runway on the spendable balance: 83 h today, about 119 h after waves 1 and 2, 157 h after all four.