Hermes tuning

Implementation brief for a Fable session

Written 2026-09-17 08:00 UTC from the audit at ~/hermes-reports/hermes-audit-2026-09-17.html. This file is the instruction. The five workstream files under ~/hermes-reports/plan/ hold the verified line references, code and tests; read the one for the wave you are executing before touching anything.

0. Ground truth you must not re-derive

• Nous bills through the Portal at catalog rates: glm-5.3-flash 0.075 / 0.25 per M, deepseek-v4-flash-0731 0.04 / 0.10 per M, cache read 0.01 on both. The old ledger was 12x to 17x high. Real spend today: $2.11 for 2,549 calls. Real burn: $0.40 per wall-clock hour, $0.18 per session-hour.
• 96.5 percent of prompt tokens are cache hits. 79.8 percent of the real bill is cache re-reads of context (p50 prompt 66k, p90 118k). Context size is the cost lever; model choice is not.
• Every 402 today came from the local proxy ~/.hermes/modellock on phantom spend, never from Nous. The proxy was repriced at 07:23 UTC (cache-aware price(), cached_tokens in the ledger, caps rebased to real dollars: total 34, daily 10). Policy hot-reloads per request; proxy.py changes need launchctl kickstart -k gui/$(id -u)/com.<user>.hermes-modellock.
• Truth endpoint for the credit balance: GET https://portal.nousresearch.com/api/oauth/account, the undocumented endpoint Hermes's own account client uses, so it may change. Send the bearer from ~/.hermes/auth.json providers.nous.access_token. Hermes rotates it hourly; never call the refresh endpoint yourself, because the comments in Hermes's own auth code say refresh tokens are single-use and revoke the session.
• Hermes config: ~/.hermes/config.yaml (299 lines, _config_version: 33). Most keys are copied at process start; only delegation.*, agent.verify_on_stop, updates.pre_update_backup and the approval policy are hot-read. /new does not reload config.
• Five TUI sessions may be live in Terminal.app. Never send keystrokes to them (a typed string arrived as Ctrl-C once and killed a session). Never drive Chrome by "front window": another automation navigates it; use DevTools targets on the dedicated remote-debugging port, addressed by target id.

1. Rules for the implementing agent

1. Back up before every write: cp <file> <file>.bak-<tag>-$(date +%Y%m%d-%H%M%S). Vendor files under ~/.hermes/hermes-agent get a .diff saved to ~/hermes-reports/plan/patches/ because hermes update stashes local edits.
2. Read-only on ~/.hermes/state.db (?mode=ro). Never run hermes interactively; the only permitted Hermes invocations are the smoke tests named in wave 4, which cost under $0.01.
3. No paid API beyond: the two 5-token proxy probes in the proxy acceptance script, the portal account GET, and the catalog GET. State the call budget before any subagent is spawned; forbid paid APIs in its brief.
4. Measure before and after every wave with python3 ~/hermes-reports/plan/measure.py --since <apply-time-ISO>; the before snapshot is plan/baseline_run2.txt. A wave passes only when its acceptance rows pass; on the stop rule, roll back that wave and record why.
5. Apply waves in order. Waves 1 and 3 are config-only and are what stretch the credit runway; do not let the proxy or runner builds delay them.
6. Report in the format of section 6, with numbers from measure.py, never from memory.

2. Wave 1

Context hygiene (config.yaml, 10 min apply, 2 h measure, runs with goals live)

Detail: plan/plan_config.md steps 1 to 8; helper plan/apply_tuning.py (ruamel round-trip, keeps comments) or plan/setkey.py key value.

Then purge ~/.hermes/memories/MEMORY.md under its flock (procedure in plan_config.md step 8; 1,836 bytes of August notes injected into every prompt). Takes effect per session at its next start; do not restart live goals, let them finish. Acceptance (plan_acceptance.md C1 to C4, T3): prompt p90 under 60k within 2 h of the first new session, compaction 2 to 20 per 1,000 calls, cache-hit share stays above 90 percent (prune can break the prefix cache; that is the stop rule), exit-124 under 3 per hour.

3. Wave 2

Proxy hardening (proxy.py + policy.json, one restart in a 10 min quiet window, 4 h soak)

Detail: plan/plan_proxy.md, in this order, shipped as one restart.

1. Step 0: env overrides MODELLOCK_ROOT / MODELLOCK_PORT / MODELLOCK_AUTH_JSON so the acceptance script runs a second instance on 8799 against a scratch copy.
2. Drain the request body on unrecognised POSTs (the 94 "Bad request syntax" 400s a day) and terminate Hermes's provider sweep with a 200 on /version carrying a version key. Do NOT answer /api/tags with a models key: Hermes would classify the proxy as Ollama and start /api/show probes.
3. Fail-closed load_state(): write a ledger anchor event at startup and on SIGHUP, rebuild from the last anchor on a corrupt state.json, 503 when no anchor exists, never zeros. Run --anchor once at deploy because the 07:30 rebase left no anchor row.
4. Warning headers X-ModelLock-Remaining-USD and X-ModelLock-Warning at 80 percent of any cap, plus a WARN line to ~/.hermes/modellock/warn.log.
5. Degrade instead of 402 when only a per-model cap is hit: rewrite to the other allowed model and overwrite the reasoning field with the target's override (glm rejects effort: none with HTTP 400). Global and bucket caps still 402.
6. /stats JSON endpoint and SIGHUP log reopen (launchd holds the fd; a plain newsyslog rename leaves an empty file).

Acceptance (P1 to P7): zero bad-syntax 400s, sweep 404s per hour under 5 excluding /api/v1/models, state.json survives a deliberate corruption test on the scratch instance, two real 5-token calls cost under $0.0005 each with cached_tokens present. Rollback: restore proxy.py.bak-* and kickstart.

4. Wave 3

Agent behaviour (config plus one hook, one working day to measure 20 tasks)

Detail: plan/plan_delegation.md. Config and hook first, vendor patches last.

1. delegation.max_iterations: 30, max_concurrent_children: 2, child_timeout_seconds: 900 (hot-read).
2. skills.creation_nudge_interval: 0 (curator becomes opt-in; 48 skill_manage errors today were 60-char description refusals) and memory.nudge_interval: 30.
3. platform_toolsets.cli: drop bfl, image_gen, tts, computer_use, session_search (about 6k tokens per call; delegate_task has no toolsets argument, so this is the only lever). Needs per-tab restarts at goal boundaries.
4. Pre-spawn budget hook: a pre_tool_call shell hook in config.yaml hooks: matched to delegate_task that GETs http://127.0.0.1:8788/healthz (already serves spent_today_usd and caps) and blocks the spawn under a reserve. Script text is in plan_delegation.md step 2.
5. Delegation contract as a skill named delegation-contract the parent pastes into context: deliverable path written by iteration N-10, budget stated every 10 iterations, no sleep polling. Survives hermes update.
6. agent.verify_on_stop: false in pipeline profiles only (or HERMES_VERIFY_ON_STOP=0); 18 verification_required finishes and 243 s npm test runs after JSON edits.
7. Vendor patches, each saved as a .diff: exit reasons billing_blocked / provider_error / max_iterations / incomplete in tools/delegate_tool.py (the child already carries failure_reason: billing, never read); busy-parent drain of completed delegations through the steer rail (delivery lag p50 15 min is the parent's turn length); curator prompt states the 60-char limit.
8. Operator pipeline fixes in the operator's SEO pipeline repo: orchestrator.mjs must push cached-stage issues into allIssues (REPORT.md said clean over an ok:false human check); L12 sign-off requires a person in by and an operator-minted token, rejects autonomy wording; deployed:true only from a 13-prod-deploy.json written from real wrangler output.

Acceptance (D1 to D4, S1, T1): max_iterations share under 15 percent over 20 tasks, zero 402 deaths labelled max_iterations, delivery lag p50 under 3 min, first-call prompt under 12k tokens, skill_manage errors zero.

5. Wave 4

Budgets, portal guard, runner (live, one pipeline per quiet hour)

Detail: plan/plan_proxy.md steps 5 to 9 and plan/plan_runner.md.

1. Proxy buckets: the inbound bearer value (NOUS_API_KEY=pl-<name> per profile) becomes the budget key; buckets: section in policy.json; bucket field in the ledger. Unknown bearer falls into default.
2. Portal guard: every 50 calls or 5 minutes, curl -K - (token off argv) to /api/oauth/account, store credits and member spend in state["portal"], cap on total_usable_credits - spent_since_read <= 1.10. On 401 or expiry fall back to local caps, log drift, never rebase automatically.
3. Live streaming relay: keep curl, Popen with -N --speed-limit 1 --speed-time <stall> --max-time <per-model>, price partials fail-high as partial_estimate.
4. Runner at ~/hermes-pipelines/: use hermes -p pl-<name> chat -q "<step>" -Q --yolo --max-turns N --skills ... --usage-file ..., not -z (-z ignores --skills, has a fixed 90-iteration cap and exits 0 on partial runs). Profiles via hermes profile create pl-<name> --clone --no-alias; each profile .env carries its own NOUS_API_KEY=pl-<name> and HERMES_VERIFY_ON_STOP=0. Classify outcomes from usage.json plus the ledger bucket, resume on 402 with --resume <session_id>, back off 30 to 600 s on 502, exit 75 for the launchd tick. Smoke test: the 2-step /tmp write-and-verify, about $0.002.
5. Migration order: the first site pipeline (4 phases map to steps), then the other site pipelines, then the two non-site pipelines as single-step pipelines. Kanban lanes afterwards: hermes kanban create --assignee pl-<name> --max-runtime --max-retries, dispatcher in the gateway.

Acceptance (P4 to P9, R1 to R3): zero 402 while usable credits exceed the reserve, per-bucket cost visible in /stats, smoke pipeline completes twice with cost under $0.006, no session ever needs a keystroke to resume.

6. Report format when done (per wave)

• Applied: file, key or function, backup name.
• measure.py before and after: the ACCEPTANCE table rows for this wave, pass or fail each.
• Real spend during the wave from state.json and the portal account GET (two numbers, and the drift between them).
• Anything skipped and why. Anything that regressed and what was rolled back.

Projection to hold yourself to (plan_acceptance.md section 5): $2.11 a day today; $1.33 to $1.64 after waves 1 and 2; about $1.12 after waves 3 and 4. Runway on the spendable balance: 83 h today, about 119 h after waves 1 and 2, 157 h after all four.

Implementation results, 08.45 UTC

All four waves shipped

bash ~/hermes-reports/plan/after.sh

Shipped, by wave

Skipped or deferred, stated plainly

• newsyslog rotation needs one sudo cp of ~/.hermes/modellock/newsyslog.d-hermes-modellock.conf into /etc/newsyslog.d; not run.
• Per-task iteration budget and a toolsets argument on delegate_task (plan 1d, 3b) were not built; the config-level caps and the trimmed toolset cover the measured problem.
• Kanban lanes are documented in MIGRATION.md but not enabled; the runner comes first by design.
• The first site pipeline's step file was written from the plan because every phase document on the iCloud Desktop was dataless during the build; run its first gate by hand once before trusting it.
• --usage-file turned out to be oneshot-only, so chat-mode steps get their usage synthesized from the profile's session row plus the ledger window; it matched the ledger exactly on the smoke run.
• Catalog rate refresh is a manual command, not scheduled.

Why 88 and not 95

Sources: plan/impl/impl_proxy.md, impl_vendor.md, impl_pipeline.md, impl_runner.md, impl_measure.md; backups named per row. Live proxy stats at http://127.0.0.1:8788/stats.