When a high-risk side effect (e.g. host restart via shutdown.exe) runs,
the user's plain-text confirmation phrase is persisted in the conversation
transcript. If the host restart killed the gateway process before the
assistant's tool result was written, the transcript tail ends on the
assistant's text response - and the dangerous confirmation text remains
in the user role.
On the next inbound message - possibly a casual 'are you there?' from
the user minutes later - the LLM sees the stale confirmation and may
interpret the new turn as a fresh re-confirmation, re-executing the
destructive action. This is the failure mode reported in #59607.
Fix:
- Add strip_stale_dangerous_confirmations() in agent/replay_cleanup.py
that removes user messages whose content matches a known dangerous
confirmation pattern AND whose timestamp is older than 60 seconds.
- Add is_dangerous_confirmation() helper with the matched patterns
(i18n-aware: covers 確認強制重開機 from the original incident).
- Wire the stripper into _build_gateway_agent_history() right after the
existing 75ed07ace strippers, so the strip chain is:
strip_interrupted_tool_tails -> strip_dangling_tool_call_tail ->
strip_stale_dangerous_confirmations.
- Update _build_replay_entry() to preserve the timestamp on user
messages (it was previously dropped), since the new stripper needs it.
Complements 75ed07ace (which strips the assistant side of the broken
tail) by handling the user side: a stale plain-text confirmation that
the assistant has not yet responded to in a way the resume logic
recognises.
Failing-test-first discipline: the bug-detection test
test_stale_confirmation_text_is_stripped_on_resume fails on unfixed
code (proves the test catches the bug) and passes after the fix.
Five additional safety tests confirm no regression on:
- fresh confirmations (within expiry) are preserved
- non-confirmation text is preserved
- non-matching histories are untouched
- dangerous-pattern detection works in all cases (case, i18n, None)
- direct unit test of the strip helper
Refs: #59607
* fix(auth): resolve Anthropic OAuth file per-profile + close port-binding platform gaps
Two focused pieces salvaged from PR #57563:
1. _HERMES_OAUTH_FILE was computed at module import time — frozen before
HERMES_HOME/profile overrides, so multiplexed profile turns read and
wrote the DEFAULT profile's .anthropic_oauth.json (OAuth path hijack).
Replaced with a lazy _get_hermes_oauth_file(); all web_server.py call
sites updated.
2. _PORT_BINDING_PLATFORM_VALUES was missing whatsapp_cloud and line —
both bind aiohttp TCP listeners, so a secondary multiplex profile
enabling them would collide with the primary's listener instead of
failing fast at startup.
Original work by @austinlaw076. The rest of #57563 was redundant on
main (adapter routing sweep superseded by #56854's salvage; cron secret
scope landed in fdab380a1; nested-config fallback in from_dict).
* chore(release): map austinlaw076 author email for PR #57563 salvage
* test(hermes_cli): patch _get_hermes_oauth_file instead of removed _HERMES_OAUTH_FILE constant
---------
Co-authored-by: Austin <austin@openvm067.space>
Co-authored-by: Ben <ben@nousresearch.com>
When skill_view loads a supporting file (references/, scripts/,
templates/) instead of the main SKILL.md, the CLI quiet-mode line and
the friendly tool labels now show 'name → file_path' so it's clear
which file was actually read.
Follow-up hardening on the cherry-picked pool-fallback fix. The original
_resolve_codex_usage_credentials wrapped BOTH resolve_codex_runtime_credentials()
and the separate _read_codex_tokens() account_id read in one broad
'except Exception: pass', which had three problems:
1. A transient refresh/network failure (non-AuthError) from the resolver was
silently swallowed and downgraded to pool.select(), which could report
/usage limits for a DIFFERENT pool account than the one actually running.
On main that error surfaced. This is a real behavior regression for the
multi-account/pool case.
2. If the resolver succeeded but only the account_id read raised, the whole
singleton tier was abandoned in favor of a pool token that carries no
ChatGPT-Account-Id header (PooledCredential has no account_id concept),
risking a wrong-account read or 401.
3. 'except Exception' masked genuine programming errors.
Fix: narrow the outer catch to AuthError (the documented 'no creds' failure
mode of both functions), and read account_id in a best-effort inner try so a
partial/missing singleton store can't sink an otherwise-usable credential.
Transient errors now propagate and fail open via the outer fetch_account_usage
guard rather than mis-routing to the wrong account. Adds debug breadcrumbs and
a comment characterizing when the tier-3 pool path actually fires.
Guard tests: a non-AuthError resolver failure must NOT swap to the pool
(fail-open, no snapshot); an account_id read failure keeps the singleton token.
Updated the existing pool-fallback test to use AuthError (the real failure
mode) instead of a generic RuntimeError.
Follow-up on the salvage of #59523. Two low-risk cleanups surfaced by review:
- Extract _ZAI_CODING_OVERLOAD_SHORT_ATTEMPTS as a module constant so
adaptive_rate_limit_backoff() and zai_coding_overload_retry_ceiling()
share one source of truth. Previously both hardcoded short_attempts=3
independently; tuning one without the other would silently desync the
retry ceiling from the backoff schedule.
- Replace the tautological formula-mirroring assert in
test_zai_overload_retry_ceiling_exceeds_short_attempts with a behavior
invariant (ceiling leaves headroom for every long-backoff entry), per the
repo's contracts-over-snapshots testing rule.
Z.AI Coding Plan GLM-5.2 reports server overload as HTTP 429 code 1305
("temporarily overloaded"). classify_api_error routes that to
FailoverReason.overloaded (so a valid credential pool isn't burned), but
the adaptive Z.AI backoff was gated on is_rate_limited — which excludes
overloaded — so it never ran (policy=default) and the request failed after
a few quick short retries.
Two compounding causes, both fixed here:
1. Detect the Z.AI overload 429 directly and let its adaptive backoff run
on the overloaded path, not only the rate_limit path.
2. Raise the retry ceiling for this narrow case via
zai_coding_overload_retry_ceiling(). The long-backoff tier
(30/60/90/120s) starts after short_attempts (3) retries, but the default
api_max_retries is also 3, so the loop always gave up before the long
tier could run — leaving the whole long-backoff schedule as dead code.
Scope is limited to the existing narrow is_zai_coding_overload_error match,
so other providers' 429/503/529 handling is unchanged.
A fallback candidate can itself carry a stale credential (e.g. an
expired ANTHROPIC_TOKEN picked up by _try_anthropic). Its 401 previously
propagated out of the fallback call site and aborted the auxiliary task
— for compression: a 60s cooldown + context marker while the session
kept growing past the context cap. Live case: mattalachia debug dump
(Jul 2026), Codex timeout → Anthropic 401 x5 → 296K 'Cannot compress
further'.
Now each fallback candidate call is wrapped: on auth error, refresh the
candidate's provider credentials and retry once; if unrefreshable, mark
the provider unhealthy and walk the discovery chain again so the next
viable candidate serves. Sync + async paths. Non-auth errors still
raise unchanged.
Infer the concrete auxiliary auth provider from the selected client base
URL so provider:auto routes can refresh Copilot/Codex/Anthropic/Nous
credentials after auth errors, instead of skipping refresh because
resolved_provider stayed 'auto'. Adds the copilot branch to
_refresh_provider_credentials and evicts the stale auto-route cache
before retrying.
Fixes#20832. Salvaged from PR #20837, reapplied surgically onto current
main (branch predated the _retry_same_provider_sync/async extraction).
When context.engine selects a plugin engine (e.g. LCM), the host
compression threshold — including the Codex gpt-5.5 50% -> 85%
autoraise — only configures the built-in ContextCompressor and never
reaches the plugin. The autoraise notice still fired, telling the user
auto-compaction was raised when nothing actually changed, and the
startup context-limit line printed the host percent next to the
engine's own threshold_tokens, contradicting itself.
- Clear _compression_threshold_autoraised when a plugin engine is
selected, suppressing both the CLI startup notice and the gateway
turn-1 replay via _compression_warning.
- Print the active engine's own threshold_percent in the startup
context-limit line so percent and token count agree.
- Built-in behavior is preserved, including the fallback path where a
configured engine fails to load and the built-in compressor takes
over.
Fixes#44439
The Codex gpt-5.5 compaction-threshold autoraise notice re-fired on every
agent init. Because the gateway rebuilds the agent per inbound message, the
notice spammed long-running Discord/Telegram/etc. sessions, and the only
documented remedy (`compression.codex_gpt55_autoraise false`) disables the
useful autoraise behavior itself.
Gate both emission surfaces — the CLI startup print and the gateway
`_compression_warning` replay — on a persisted per-profile marker under
`$HERMES_HOME` (`.codex_gpt55_autoraise_notice`), keyed on the from→to
percentages the notice displays. The notice now shows at most once per
profile; the autoraise still fires and `codex_gpt55_autoraise: false` still
disables it; and a later change to the raised threshold re-notifies once.
Docs updated to match.
The Codex gpt-5.5 compaction autoraise (#40957) overrode the effective
threshold unconditionally. If a user had set compression.threshold above
0.85, agent_init dropped them down to 0.85. That wastes usable window and
contradicts the feature's whole point: use more of the context, not less.
It happened silently too, since the one-time notice is suppressed when the
override doesn't raise.
The override is an autoraise. It must only raise. Pulled the apply logic
into a small pure helper that clamps the Codex case to never lower a
higher-or-equal user threshold, and emits the notice only when it actually
fires. Other overrides (Arcee Trinity) keep their existing unconditional
behavior.
Fixes the Codex gpt-5.5 compaction autoraise lowering a user's higher
configured threshold. A user on the Codex OAuth route with
compression.threshold > 0.85 was silently clamped to 0.85, compacting
earlier than they asked and using less of the 272K window the feature was
meant to unlock. The autoraise now only ever raises.
N/A
- [x] 🐛 Bug fix (non-breaking change that fixes an issue)
- [ ] ✨ New feature (non-breaking change that adds functionality)
- [ ] 🔒 Security fix
- [ ] 📝 Documentation update
- [ ] ✅ Tests (adding or improving test coverage)
- [ ] ♻️ Refactor (no behavior change)
- [ ] 🎯 New skill (bundled or hub)
- `agent/agent_init.py`: added `_resolve_compression_threshold()`, a pure
helper that combines the global threshold with a per-model override. The
Codex gpt-5.5 autoraise never lowers a higher-or-equal user threshold;
the notice is returned only when it actually raises. Rewired `init_agent`
to call it, replacing the unconditional `compression_threshold = _model_cthresh`.
- `tests/agent/test_arcee_trinity_overrides.py`: added 5 cases for the
helper — raise from default, never-lower regression, equal-is-noop,
no-override passthrough, and non-codex (Trinity) unconditional apply.
1. Set `compression.threshold: 0.90` and run gpt-5.5 on provider `openai-codex`.
2. Before: effective threshold drops to 0.85, no notice. After: stays 0.90.
3. Run `scripts/run_tests.sh tests/agent/test_arcee_trinity_overrides.py`.
Stash `agent/agent_init.py` and the new cases fail; restore and they pass.
- [x] I've read the [Contributing Guide](https://github.com/NousResearch/hermes-agent/blob/main/CONTRIBUTING.md)
- [x] My commit messages follow [Conventional Commits](https://www.conventionalcommits.org/) (`fix(scope):`, `feat(scope):`, etc.)
- [x] I searched for [existing PRs](https://github.com/NousResearch/hermes-agent/pulls) to make sure this isn't a duplicate
- [x] My PR contains **only** changes related to this fix/feature (no unrelated commits)
- [x] I've run `pytest tests/ -q` and all tests pass
- [x] I've added tests for my changes (required for bug fixes, strongly encouraged for features)
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)
- [x] I've updated relevant documentation (README, `docs/`, docstrings) — or N/A
- [x] I've updated `cli-config.yaml.example` if I added/changed config keys — or N/A
- [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — or N/A
- [x] I've considered cross-platform impact (Windows, macOS) per the [compatibility guide](https://github.com/NousResearch/hermes-agent/blob/main/CONTRIBUTING.md#cross-platform-compatibility) — or N/A
- [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A
gpt-5.3-codex-spark has a native 128K context window but the default
50% compaction trigger fires at ~64K, wasting half the usable window
before the session has accumulated enough turns to summarize
meaningfully. This raises the trigger to 70% (~90K) on the Codex OAuth
route only, leaving ~38K headroom for the summary and continued
conversation before the 128K hard limit.
The override is not gated by allow_codex_gpt55_autoraise because 128K
is the model's native window (unlike gpt-5.5's artificial 272K Codex
cap). Non-Codex routes are unaffected.
Also adds a boundary regression test verifying the short-session
scenario from the issue always yields a non-empty compressible window
(no silent context wipe).
The ChatGPT Codex OAuth backend caps both gpt-5.4 and gpt-5.5 at a 272K
context window, but the autoraise that lifts the compaction trigger to 85%
only matched gpt-5.5. On gpt-5.4 the global 50% threshold fired at ~136K —
half the usable window — compacting far earlier than necessary.
Rename _is_codex_gpt55 -> _is_codex_gpt54_or_gpt55 and match both families.
The one-time user notice is now model-aware (shows the actual slug). The
config key codex_gpt55_autoraise is kept as-is for backward compatibility.
Adds gpt-5.4 coverage to the autoraise tests.
Follow-up on the cherry-picked #36896 commits, wiring 1Password into
the new registry as the reference *mapped* source:
- OnePasswordSource adapter (shape=mapped, scheme=op): fetch-only —
precedence, override semantics, conflict warnings, and env writes
move to the orchestrator; apply_onepassword_secrets kept as legacy
shim like Bitwarden's.
- Registered in _ensure_builtin_sources; mapped op:// bindings now
outrank bulk Bitwarden project dumps on contested vars.
- _cache.py FetchResult/is_valid_env_name re-exported from base so
there is exactly one canonical definition; bitwarden.py re-adapted
onto the contributor's DiskCache substrate.
- ErrorKind classification for op failures (auth/binary/empty/network).
- Registry + conformance coverage for OnePasswordSource, incl. the
headline multi-source test: both vaults claim the same var, mapped
1Password wins, conflict surfaced, provenance correct.
- env_loader tests migrated off the legacy apply_* mocks onto the
fetch layer; AUTHOR_MAP entry for @hwrdprkns.
The 1Password secret source resolves op:// references using
OP_SERVICE_ACCOUNT_TOKEN read from os.environ. Under systemd the gateway
gets that token via EnvironmentFile, but cron jobs, subprocesses, CLI
runs, macOS launchd, and Docker containers spawn fresh interpreters with
no inherited shell state — so they silently failed to resolve any
reference and fell back to empty strings.
Two patches close the gap, matching Bitwarden's reliability guarantees:
1. env_loader: auto-load ~/.hermes/.op.env after .env so the gitignored
bootstrap token is available everywhere. override=False plus an
explicit guard ensure it never clobbers a token already in env (e.g.
from a systemd EnvironmentFile, which keeps precedence).
2. credential_pool: _get_env_prefer_dotenv() now prefers the resolved
value in os.environ when .env still holds a raw op:// reference,
instead of handing a URL to provider auth. Non-op:// values keep the
existing .env-takes-precedence behaviour.
Also gitignore .op.env, document the three bootstrap-token options, and
add tests covering auto-load, no-override, and the resolved-vs-raw
precedence (plus regression guards).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve provider credentials from 1Password op://vault/item/field references
at startup via the official `op` CLI, alongside the existing Bitwarden source.
Users map env-var names to references in secrets.onepassword.env; after .env
loads, each is resolved with `op read` and injected into os.environ. Auth is
whatever `op` already uses (service-account token or desktop/interactive
session) — Hermes never authenticates or installs `op` itself.
Startup-safe and fail-open: a missing binary, expired auth, a bad reference,
or an empty value each warn and fall back to existing credentials, never
blocking startup. Successful, complete pulls are cached in-process and on disk
(<hermes_home>/cache/op_cache.json, 0600) via the shared DiskCache; only
secret values are stored, never the token (auth is fingerprinted into the
key). Adds `hermes secrets onepassword {setup,status,set,remove,sync,disable}`
(aliases op/1password), config defaults, the cli-config example, docs, and
hermetic tests.
Hardening applied across both backends in env_loader: each source runs in its
own guard, config sections are coerced to dict, and cache_ttl_seconds is
coerced defensively — so a malformed secrets: section can't abort startup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pull the disk-cache + FetchResult substrate out of bitwarden.py into a new
agent/secret_sources/_cache.py: FetchResult, CachedFetch, is_valid_env_name,
and a generic DiskCache (atomic mkstemp -> chmod 0600 -> os.replace write,
0700 cache dir, TTL-gated read AND write). Bitwarden now consumes it via a
module-level DiskCache instance and thin wrappers, so the security-sensitive
atomic-write/0600/TTL logic lives in exactly one place instead of being
copy-pasted per backend (and drifting). Behavior is unchanged — the full
Bitwarden suite passes untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When a bundled web provider (firecrawl, tavily, exa, ...) is listed in
plugins.disabled, its provider never registers and the web_search/
web_extract dispatchers emitted the misleading "No web extract provider
configured. Set web.extract_backend to ..." — even though the backend was
configured correctly. The real fix is to re-enable the plugin.
- web_tools.py + web_search_registry.py: when the configured backend names
a disabled bundled web plugin, both dispatchers now point the user at the
actual cause (re-enable the plugin) instead of a wrong config hint.
- plugins_cmd.py cmd_enable: enabling by canonical key now also clears the
manifest-name alias (web-firecrawl) from plugins.disabled, so the
suggested command actually re-enables the plugin ('explicit disable wins'
matches on the name too).
- plugins_cmd.py cmd_toggle / _run_composite_ui / _run_composite_fallback:
the interactive 'hermes plugins' menu now persists the canonical key
(web/firecrawl), never the bare manifest name — the drift that put the
offending entry in plugins.disabled in the first place.
Follow-up to #59518 (which fixed web credential resolution, a different
cause). Fixes the disabled-plugin symptom reported after that PR.
`summarize_background_review_actions` was structured on the assumption
that every parsed tool response is a fully-typed dict-of-fields. In
practice the memory/skill tools — and their wrappers over Mem0 OSS and
the skill_manage MCP server — sometimes serialize `_change` as a list
or scalar, and clamp `operations` to a single string when the field
came in via a partial JSON bridge.
The original code did the equivalent of:
change = data.get("_change", {})
change.get("description", "")
so when `_change` was a list the inner .get crashed with
`AttributeError: 'list' object has no attribute 'get'`, every ~10
turns the user saw the entire background review collapse.
Three defensive guards in summarize_background_review_actions:
- `call_details.get(tcid, {})` → `call_details.get(tcid) or {}` plus
`isinstance(detail, dict)` coercion. Catches stale scalar/None
values when a fork inherits partial state from a stale tool_call_id.
- `operations = detail.get("operations") or []` → `isinstance(ops_raw, list)`
coerce, then per-entry isinstance check before `.get()`. Skips
non-dict items without raising; an entire surrounding review no
longer goes down because one entry was malformed.
- `change = data.get("_change", {})` → `isinstance(change_raw, dict)`
coerce. The originally-reported crash class for skill_manage with
list-shaped _change now falls through to the generic summary path.
And the caller in `_run_review_in_thread` is wrapped in a try/except
that maps any residual summarize exception to `actions = []` and
emits a 'partial results' warning, so even an entirely unanticipated
shape won't take down the outer review — the user only sees
'Background memory/skill review failed' instead of the prior hard
crash that lost every successful action the fork had completed.
Tests: tests/test_background_review_list_shapes.py — standalone
pytest-free runner, 7/7 PASS:
a_change_as_list_does_not_crash (originally-reported shape)
a_change_as_int_does_not_crash (scalar fallback)
b_operations_as_string_treated_as_empty
b_operations_as_none_treated_as_empty
c_operations_contains_non_dict_entries (verbose-mode per-entry filter)
d_detail_non_dict_replaced_with_empty
e_call_defends_via_try_except (structural anchor)
Refs NousResearch/hermes-agent#59437
build_preloaded_skills_prompt() (hermes -s <skill>, and tui_gateway's
HERMES_TUI_SKILLS deployment env var) loads skills via _load_skill_payload()
with a raw identifier, bypassing get_skill_commands()' scan-time disabled
filter entirely. Result: a skill an operator disabled via skills.disabled
still gets force-loaded and injected into every session — including every
session on a shared tui_gateway deployment where the operator set
HERMES_TUI_SKILLS.
The bundle-invocation path (#59156) already re-checks get_disabled_skill_names()
for exactly this reason; preloaded-skill loading was the other _load_skill_payload
call site still missing it.
Fix: check each resolved skill's name (and raw identifier) against
get_disabled_skill_names() before injecting it. A disabled skill is now
reported the same way an unknown one already is (skipped, listed in the
returned missing_identifiers) — no return-shape or caller changes needed.
No behavior change when no skill is disabled.
Same bug class as #40190: these providers read credentials via bare
os.getenv(), so keys stored in ~/.hermes/.env (hermes config layer)
were invisible in execution paths that never exported them into the
process environment. Add get_provider_env() on the WebSearchProvider
module as the shared config-aware lookup (get_env_value with os.getenv
fallback) and route all credential reads through it. SearXNG already
did this (#34290); Firecrawl fixed in the preceding cherry-picked
commit by @liuhao1024.
_classify_by_status() routes every other transient HTTP status to a retryable
reason (500/502 -> server_error, 503/529 -> overloaded, 429 -> rate_limit,
413 -> payload_too_large), but 408 Request Timeout fell through to the generic
`400 <= status < 500` branch and was classified as a non-retryable
format_error -- the same bucket as a 400 Bad Request.
A 408 is a transient timing failure the server itself flags as safe to retry
(RFC 9110 15.5.9), not a malformed request, so the retry loop aborted the turn
when a simple retry would recover. Common trigger: a reverse proxy in front of
a self-hosted backend (llama.cpp / Ollama / vLLM) returns 408 when a long
generation outruns the proxy's request-read window.
Route 408 to the existing FailoverReason.timeout (rebuild client + retry).
Add a regression test plus a boundary test asserting 400 stays non-retryable.
Four independent pre-request stalls sat on the critical path between
prompt submission and the first streamed token, measured with cProfile
against a live process:
1. Discord capability detection (~2.0s, worst 5s): get_tool_definitions
-> _get_dynamic_schema made a BLOCKING https call to discord.com
inside AIAgent.__init__ for any user with DISCORD_BOT_TOKEN set, on
every platform, every cold process. Now non-blocking: memory cache ->
24h disk cache -> permissive default + one background detection that
seeds the disk cache for the next process. The permissive default is
pinned per-process so tool schemas never flip mid-conversation
(prompt-cache safety); it mirrors the existing detection-failure
fallback (all actions exposed, 403s enriched at call time).
2. Ollama /api/show probe (~0.3s): get_model_context_length step 5e
POSTed to <base_url>/api/show for KNOWN providers (openrouter etc.),
got a 404, and never cached the miss - so every fresh process paid a
full HTTP round-trip. Known non-Ollama providers now skip the probe;
local/custom/unknown endpoints keep the exact previous behavior.
3. env_probe subprocess sweep (~0.5s): the Python-toolchain probe ran
4-8 subprocess calls inside the FIRST system prompt build. Now warmed
off-thread during agent init; the prompt build hits the cache (same
lock, so a mid-flight warm just joins instead of recomputing).
4. tools.mcp_tool import (~0.4s): the between-turns MCP refresh in
build_turn_context imported the whole mcp package even with zero MCP
servers configured. MCP tools can only exist if tools.mcp_tool was
already imported (discovery/reload paths), so gate the import on
sys.modules membership - no behavior change for MCP users.
CLI additionally pre-imports run_agent + openai off-thread during the
idle banner window (same pattern as the /model picker prewarm), hiding
the remaining ~1.5s of module imports while the user types. Fixes 1-4
apply to every interaction layer (CLI, gateway, TUI, desktop, cron).
Measured cold first turn (submit -> request dispatched, openrouter,
discord token set): 4.3s before -> 0.9s after CLI prewarm (~80%); the
agent-side non-import cost drops 2.9s -> 0.36s (init) + 0.27s (turn
prologue).
DeepSeek v4 (and other strict OpenAI-compatible providers) reject an
assistant message carrying ``tool_calls: []`` with HTTP 400 "Invalid
'messages[N].tool_calls': empty array. Expected an array with minimum
length 1, but got an empty array instead." Once it hits, every retry on
the session returns the same 400 and the conversation is stuck.
An empty array is semantically identical to "no tool calls", but it
reaches the wire from session resume, host-fed histories, and the
consecutive-assistant merge in repair_message_sequence (which preserves
a pre-existing []). None of the existing sanitizer passes touch it —
they all short-circuit on ``if not tcs`` / ``if msg.get("tool_calls")``.
Fix it at sanitize_api_messages, the final pre-API chokepoint, per the
#56980 review guidance: normalize on the per-call copy (shallow-copy the
message, drop the key) rather than in repair_message_sequence, which
would destructively rewrite the persisted trajectory and prompt cache.
Follow-up to the #45545 salvage: the cherry-picked fix duplicated the
~35-line header-shaping block (kimi UA, copilot headers, nvidia NIM,
provider-profile defaults, query params) from the explicit_base_url
branch. Route the main_runtime case through the same block instead —
one client-build path, no drift risk. Also uses _create_openai_client
like the sibling branch instead of constructing OpenAI directly.
When the main agent uses a named custom provider (custom:<name>),
resolve_runtime_provider correctly resolves the base_url and api_key.
But the auxiliary client re-resolves from the bare 'custom' provider
name, losing the provider identity. The bare 'custom' falls back to
OpenRouter, which _resolve_custom_runtime() then rejects — leaving all
auxiliary tasks (title gen, compression, vision, session search, etc.)
with no credentials.
Fix: when resolve_provider_client receives a main_runtime dict
containing concrete base_url + api_key, use it directly instead of
re-resolving. The main agent already solved provider resolution;
the auxiliary client just needs to reuse its answer.
Closes#45472
Follow-up to the #55911 salvage: inherit model.api_key only when the aux
base_url resolves to the same hostname as the main model's base_url
(runtime override or config). A misconfigured aux endpoint on a different
host keeps the fail-safe no-key-required placeholder instead of leaking
the main credential cross-host.
When an auxiliary task is configured with provider=custom and an explicit
base_url but an empty api_key, the custom_key fallback chain in
resolve_provider_client() jumped straight to the no-key-required
placeholder without consulting model.api_key from config.yaml. Users
on self-hosted gateways who share the same endpoint and credentials for
both the main model and auxiliary tasks got 401 auth errors.
Add _read_main_api_key() following the same pattern as _read_main_model()
and _read_main_provider(): checks _RUNTIME_MAIN_API_KEY (runtime override)
first, then config.yaml model.api_key. Insert it into the fallback chain
before no-key-required so real credentials are used when available, while
local servers without auth still get the placeholder.
Skill bundles load their member skills via _load_skill_payload directly,
bypassing the scan-time disabled filter in get_skill_commands(). PR #58888
closed this gap for stacked slash-skill invocations, but /<bundle> dispatch
in the gateway had the same class of bypass: a skill an operator disabled
for a platform via skills.platform_disabled still got its full content
injected when referenced by a bundle.
build_bundle_invocation_message() now accepts a platform kwarg, filters
members against get_disabled_skill_names(platform=...), and reports skipped
skills in the bundle header. Gateway dispatch passes the event's platform
explicitly (env-var resolution can't be trusted in the multi-platform
gateway process, same reasoning as the #58888 gate).
_resolve_task_provider_model returns early on an explicit provider arg,
which skips the config block that consults auxiliary.<task>.base_url /
api_key. Any caller passing provider explicitly (e.g.
resolve_vision_provider_client(provider="custom", ...)) bypasses the
configured custom endpoint and falls through to main-runtime resolution,
silently routing the task to the wrong backend.
Adopt the task's configured base_url/api_key before the early returns,
but only when no explicit base_url was given and the config targets the
same provider (or names none) — a caller forcing a *different* provider
keeps full explicit-arg priority, and an explicit base_url still wins
over config.
Fixes#58515
Port from openclaw/openclaw#95108: an unbounded response.read() on a
non-OK *streaming* response can balloon memory (huge body) or hang the
agent forever (body opens then stalls with no further bytes). The
diagnostic body is only ever shown truncated, so reading megabytes or
blocking indefinitely buys nothing.
Add agent/bounded_response.read_streaming_error_body() which caps the
read at a byte limit and enforces a hard wall-clock deadline (run on a
worker thread so it can interrupt a socket read that stalls mid-chunk,
which a between-chunk wall-clock check cannot). Wire it into all three
streaming error-body sites that previously did a bare response.read():
native Gemini, Gemini Cloud Code, and Antigravity Cloud Code. The
existing error builders now accept an optional pre-read body_text so
classification (status code, RESOURCE_EXHAUSTED, free-tier guidance,
Retry-After) is preserved unchanged.
Tests use a real in-process socket server (no mocks): oversize body is
capped, stalled body hits the deadline with partial text preserved,
normal error envelope reads intact and parses.
_redact_env already skips redaction when a KEY=value assignment's value is
a programmatic env lookup (os.getenv(...), os.environ[...], process.env.X)
per issue #2852 — masking it would corrupt a code snippet, not redact a
secret. _redact_json (JSON "key": "value" syntax) and _redact_yaml
(unquoted key: value syntax) are separate closures in the same function
and never got the same check, so the identical code-snippet-in-config-
syntax case still gets mangled:
{"apiKey": "os.getenv('OPENAI_API_KEY')"} -> {"apiKey": "os.get...EY')"}
api_key: os.getenv("OPENAI_API_KEY") -> api_key: os.get...EY")
Fix: apply the same _ENV_LOOKUP_VALUE_RE.match(value) check in both
closures before masking, mirroring _redact_env exactly. Real secret
values in JSON/YAML syntax are still redacted (verified live and via new
tests) — this only skips the case where the "value" already look like a
code snippet.
Port from openai/codex#21069 ("Spill large hook outputs from context").
Both shell hooks and Python plugins can return {"context": "..."} from
pre_llm_call, which gets appended to the current turn's user message on
every subsequent API call. A plugin that emits a large blob inflates
every turn and blows out the prompt cache prefix.
- tools/hook_output_spill.py: shared helper that writes oversized
context to $HERMES_HOME/hook_outputs/<session_id>/<uuid>.txt and
returns a head/tail preview plus the saved path. Never raises.
- agent/turn_context.py: apply the cap at the pre_llm_call aggregation
site (moved here from run_agent.py since the original PR), covering
both Python plugins and shell hooks.
- agent/shell_hooks.py: reserve output_spill as a sub-key under hooks:
so the config block doesn't emit unknown-hook-event warnings.
- Docs: document the cap + config in build-a-hermes-plugin.md.
Config (behaviour-preserving when absent):
hooks.output_spill: enabled/max_chars/preview_head/preview_tail/directory
Tests: 14 unit tests; shell_hooks (56) and plugins (100) suites green.
E2E validated with isolated HERMES_HOME (spill, passthrough, traversal
sanitisation, reserved-key skip).
The google-antigravity and google-gemini-cli OAuth providers were removed
in #50492. They were the only producers of a cloudcode-pa:// base_url, so
the account-level-quota early-returns in _pool_may_recover_from_rate_limit
and _credential_pool_may_recover_rate_limit are now unreachable.
- Drop the dead cloudcode-pa:// checks and the now-unused provider/base_url
params on _pool_may_recover_from_rate_limit (only caller updated).
- Prune the obsolete CloudCode-specific regression tests; keep the live
single/multi-entry pool-rotation invariants (#11314).
ruff check --fix --select F541 . on current main. Pure prefix removals;
adjacent-string concatenations keep the f only on interpolating fragments.
No string content or live placeholder altered.
Port from anomalyco/opencode#33533. Native MCP tools now register as
mcp__<server>__<tool> (double-underscore delimiter) instead of
mcp_<server>_<tool>, aligning with the convention used by Claude Code,
Codex, and OpenCode.
The double-underscore delimiter disambiguates the server/tool boundary
even when either component contains underscores (the single-underscore
form was ambiguous, which is why is_mcp_tool_parallel_safe already had to
track provenance in a side-map). It also unifies native registration with
the Anthropic-OAuth wire form (_MCP_TOOL_PREFIX = 'mcp__'), so the
single->double promotion that path performed is now a no-op for native
tools while still handling legacy replayed names.
- tools/mcp_tool.py: add MCP_TOOL_NAME_PREFIX + mcp_prefixed_tool_name()
helper; route _convert_mcp_schema, utility schemas, refresh stale-set,
and the parallel-safe prefix gate through it
- agent/transports/codex_event_projector.py: mirror convention in the
deterministic call_id input for MCP server-executed tool calls
- tests: update produced-name assertions to the new convention
The unreadable-config-overwrite bug (an existing config.yaml that reads as
{} on a permission/IO error gets replaced with only defaults or the edited
section) is not limited to save_config / config set / auth. The same
read-then-atomic_yaml_write pattern lives at ~7 other independent write
sites that don't route through those functions:
- gateway/slash_commands.py: _save_config_key, memory/skills write_approval
toggles, tool_progress toggle, runtime_footer toggle, personality set
- hermes_cli/doctor.py --fix (stale root-key migration)
- gateway/platforms/yuanbao.py auto-sethome
- plugins/platforms/telegram/adapter.py topic thread_id persistence
- tui_gateway/server.py _save_cfg
- agent/onboarding.py mark_seen
Rather than sprinkle require_readable_config_before_write() at each site,
add a single fail-closed chokepoint, atomic_config_write(), that runs the
guard then delegates to atomic_yaml_write, and route every config.yaml
write through it. Root cause remains that read_raw_config() can't tell an
absent file from an unreadable one (returns {} for both) — read-only
callers correctly stay fail-open, but any full-file replacement now fails
closed in one enforced place instead of relying on each caller to remember
the guard.
save_config / set_config_value / auth keep the contributor's original
guard calls (their commit); this commit widens the fix to the sibling
call paths and adds a regression test on the chokepoint (fails closed on
unreadable existing file + still creates a genuinely absent file).
Compression could produce a transcript with ZERO user-role messages,
which OpenAI-compatible backends (vLLM/Qwen) reject with a non-retryable
`400 No user query found in messages`. This crashes `hermes kanban`
workers unrecoverably: every resume replays the same poisoned history and
fails on the very first request after a successful compaction.
The existing #52160 guard pins the handoff summary to role="user" only
when `last_head_role == "system"` — i.e. when the system prompt sits
inside `messages` (the gateway `/compress` path). The main
auto-compression path prepends the system prompt at request-build time,
so the list handed to `compress()` starts with a user/assistant turn,
`last_head_role` defaults to "user", and the summary is emitted as
role="assistant". A kanban worker seeded with a single short
`"work kanban task <id>"` prompt followed by nothing but assistant/tool
turns therefore ends up user-less once that early turn is summarised.
Generalise the guard: when no user-role message survives in the protected
head or the preserved tail, force the summary to carry role="user" so the
request always has at least one user turn. When a user does survive
(e.g. in the tail), the guard does not fire, so alternation is preserved.
Fixes#58753.
Follow-up to the salvaged #58696 (devatnull) + #41343 (annguyenNous)
commits: instead of fully suppressing commentary/analysis-phase stream
deltas, fire on_reasoning_delta so the CLI/gateway display them like
thinking text. Matches Codex CLI semantics where commentary is never
the turn's final answer, while keeping the narration visible in the
reasoning display. Adds devatnull to AUTHOR_MAP.
GPT-5.x models on the Codex Responses API emit short pre-tool-call
"preamble" text as message items with phase="commentary". Previously,
_normalize_codex_response() added ALL message items to content_parts
regardless of phase, causing commentary text to leak as visible
assistant content on chat gateways.
Fix: when normalized_phase is "commentary" or "analysis", route the
message text to reasoning_parts instead of content_parts. This keeps
preamble/internal planning in the reasoning channel where it belongs.
FixesNousResearch/hermes-agent#41293
The salvaged #54550 converted AIAgent._build_keepalive_http_client but the
near-identical build_keepalive_http_client in agent/process_bootstrap.py
(used by auxiliary clients: compression, vision, web_extract, titles) kept
the socket_options transport and the api.githubcopilot.com bypass. Same
conversion: httpx.Limits(keepalive_expiry=20) + pool timeouts, verify
forwarded on client and no-proxy mounts, copilot hardcode removed.
Inspired by Claude Code v2.1.199 (July 2, 2026): stacked slash-skill
invocations load all leading skills (up to 5), not just the first.
- agent/skill_commands.py: split_stacked_skill_commands() consumes leading
/skill tokens (stops at the first non-skill token so slash-path arguments
are never swallowed); build_stacked_skill_invocation_message() composes
the multi-skill turn reusing the existing bundle scaffolding markers so
extract_user_instruction_from_skill_message() keeps memory providers
storing the user's instruction, not N skill bodies.
- cli.py + gateway/run.py: dispatch the stacked path on both surfaces.
- 11 new tests + docs section in skills.md.
Inspired by Claude Code v2.1.199 (July 2, 2026): SSL certificate errors
(TLS-inspecting proxies, missing CA bundles, expired certs) no longer
burn retries before showing actionable guidance — they fail immediately
with the fix hint.
- agent/error_classifier.py: new FailoverReason.ssl_cert_verification +
_SSL_CERT_VERIFY_PATTERNS, checked BEFORE the transient-SSL patterns
(cert-verify messages also contain '[SSL:' and previously retried
forever as timeout). Non-retryable, no compression, no fallback churn.
- agent/conversation_loop.py: dedicated status line + per-cause fix
hints (corporate proxy CA bundle, certifi refresh, self-signed local
endpoints) on the non-retryable abort path.
- 7 new tests incl. regression guards (transient alerts still retry,
large-session cert failure doesn't trigger compression).
format_date() at line 76 still used %-d, which raises ValueError on
Windows strftime. Same class as the axis-label site fixed by #56640;
use dt.day directly. Credit also to @x7peeps (#58480) who flagged both
sites.
`_period_label()` in `learning_graph_render.py` used `%-d %b`
strftime, which is a Linux-only format — the `%-` prefix for
zero-padding suppression doesn't exist on Windows `strftime`, causing
`ValueError: Invalid format string` on `hermes journey`.
Fix by using `dt.day` directly (an integer, no zero-padding by
default) combined with `strftime('%b')` for the month. This is
cross-platform and produces identical output.
Reproduced and tested on Windows 10 with Hermes v0.18.0.
* fix(cli): set correct x-initiator header per Copilot turn
copilot_default_headers() always hardcoded x-initiator: agent, but
GitHub Copilot billing requires "user" for user-initiated prompts and
"agent" for tool/follow-up calls. This caused premium requests to never
be consumed correctly, risking billing issues or account bans.
Adds is_agent_turn param to copilot_default_headers() and injects
extra_headers={"x-initiator": "user"} on the first API call of each
user turn when targeting Copilot URLs. The flag flips to False after
injection so subsequent calls (tool use, streaming fallback) default
back to "agent".
Fixes#3040
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore(release): add AUTHOR_MAP entry for @tjp2021 (PR #4097 salvage)
---------
Co-authored-by: Tim <tim@iteachyouai.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Extend the pre_tool_call plugin hook return contract with a new directive:
{"action": "approve", "message": "why this needs human confirmation"}
Previously a pre_tool_call hook could only veto a tool call (action: block)
or allow it silently. It could not escalate to the existing human-approval
flow. This unlocks user-defined runtime approval rules on ANY tool (HTTP
writes, file writes to sensitive paths, email sends), enforced at runtime —
resolving #51221 as a pure plugin, with no core approval.py rule schema.
Mechanism:
- get_pre_tool_call_directive() returns (action, message) for block|approve;
get_pre_tool_call_block_message() kept as a block-only back-compat shim.
- resolve_pre_tool_block() is the single dispatch-site chokepoint: fetches
the directive and, for approve, invokes the human gate; fail-closed to a
block on denial, timeout, or gate exception. ALL FOUR tool-dispatch sites
now call it: tool_executor (concurrent + sequential), agent_runtime_helpers,
and model_tools.handle_function_call.
- request_tool_approval() escalates via the SAME machinery as Tier-2
dangerous commands: session/permanent allowlist, prompt_dangerous_approval
(CLI) / submit_pending (gateway), [o]nce/[s]ession/[a]lways/[d]eny,
timeout fail-closed, approvals.cron_mode for cron contexts.
Architecture: extracted the shared decision core into _run_approval_gate(),
called by BOTH check_dangerous_command() and request_tool_approval() so the
fail-closed / cron / gateway / yolo / persist policy lives in ONE place and
cannot drift. Fixed a latent divergence — the plugin path now honors --yolo.
Approval grain: [a]lways is keyed on tool_name + a hash of the reason (an
explicit plugin rule_key overrides), so distinct reasons on the same tool
persist independently instead of one 'always' blanketing the whole tool.
Non-interactive: cron honors approvals.cron_mode (parity with commands); any
other non-interactive non-gateway context fails CLOSED for the plugin path
(the command path keeps its historical fail-open default, unchanged).
No new config schema, no new env vars, no new hook events.
When OpenRouter routes to an endpoint that does not support tool/function
calling, it returns HTTP 404 with the message 'No endpoints found that
support tool use. Try disabling "browser_back".'
The raw error body does not contain 'model not found' or any other
_MODEL_NOT_FOUND_PATTERNS entry, so it falls through to FailoverReason.unknown
with retryable=True. The retry loop wastes 3-5 attempts on the same
deterministic rejection, then surfaces a confusing generic error instead of
automatically failing over to a fallback model or provider.
Adding the OpenRouter phrase to _MODEL_NOT_FOUND_PATTERNS classifies it as
model_not_found (retryable=False, should_fallback=True), which triggers the
client-error fast-fallback path in conversation_loop.py: the agent switches
to a configured fallback model/provider before the user sees the error.
Existing buffered guidance in conversation_loop.py (the 'support tool use'
hint at line ~2967) remains intact and surfaces only if every fallback
exhausts.
_try_anthropic() hard-failed (return None, None) when the anthropic
credential pool was present but had no selectable entry — e.g. the pooled
OAuth token expired and its refresh_token had gone stale, so
_select_pool_entry("anthropic") returned (True, None). This wedged every
auxiliary task routed to Anthropic (goal judge surfaced "no auxiliary
client configured") even when a perfectly valid ANTHROPIC_TOKEN /
credentials-file token was available. The main session stayed healthy
because it resolves the env token directly.
The openrouter path (_try_openrouter) and codex path already fall through
to their standalone credential on (True, None); anthropic was the only
provider that hard-failed. Make _try_anthropic fall through to
resolve_anthropic_token() on that branch so the three paths are symmetric:
a temporarily dead pool entry must not block auxiliary tasks when a valid
standalone credential exists.
Adds a regression test covering: (1) pool present + no entry + valid env
token -> client built from the env token, (2) pool present + no entry + no
resolvable token -> clean (None, None), (3) base_url defaults correctly
when falling through with pool_present=True.
'KEY=os.getenv(...)' / 'os.environ[...]' / 'process.env.X' values are
variable-name references in code snippets, not leaked secrets. Masking
them corrupted pasted code in prose/log contexts (issue #2852):
ha_token=os.getenv('HOMEASSISTANT_TOKEN') -> ha_token=os.get...EN').
Skip these values inside _redact_env, which covers all three passes that
share the closure (_ENV_ASSIGN_RE, _CFG_DOTTED_RE, _CFG_ANCHORED_RE).
Real secret values are still masked.
Salvage of PR #2852-fix #2854 — the PR's own placement (an unconditional
pass before the code_file gate) would have reintroduced the code-file
false-positive class; the skip is applied inside the existing gated pass
instead. Tests adapted from the PR.
Co-authored-by: crazywriter1 <sampiyonyus@gmail.com>
Strict providers (DeepSeek) reject a payload where the same tool_call_id
appears more than once with HTTP 400 'Duplicate value for tool_call_id'.
The issue was filed as an 'orphaned tool message' compression bug, but the
pasted error is a DUPLICATE tool_call_id — orphans are already handled on
main; duplicates were not. Reproduced live on main: both shapes leaked
through repair_message_sequence and sanitize_api_messages.
Two chokepoints, two shapes:
- repair_message_sequence: consume the id from known_tool_ids on first
match so a SECOND tool result reusing it falls into the drop branch
(duplicate tool-result shape). This is @Robinlovelace's kernel from
#55436 (applied manually — that PR was ~800 commits stale and bundled
an unrelated duplicate-DB-write change for #860, which is dropped here).
- sanitize_api_messages (final pre-API pass): add a dedup pass covering
BOTH (a) duplicate tool_calls sharing an id WITHIN one assistant message
(the message[6] shape) and (b) later tool result messages reusing an
already-seen id. #55436 covered neither of these at this chokepoint.
Tests: duplicate-tool-result dedup at both functions, duplicate-assistant-
tool_call-id collapse, and a negative control proving distinct ids are
never dropped (no over-dedup).
Credit: @Robinlovelace (#55436) for the repair_message_sequence dedup kernel.
Closes#58327.
Replace the inline dict-copy + _db_persisted pop in
_ensure_compressed_has_user_turn with the canonical
_fresh_compaction_message_copy helper (the same primitive the compressor's
own protected-head/tail assembly uses), so the persistence-marker strip
stays consistent across all compaction copy sites (#57491). Expand the
docstring to record the alternation-safety and end-placement rationale.
22c5048d9 restored Anthropic-style cache_control for two of MoA's three
call paths: the acting aggregator (MoAChatCompletions.create, the
persistent `provider: moa` model) and the advisor fan-out (_run_reference).
aggregate_moa_context() -- the /moa <prompt> one-shot command's synthesis
call -- is the third, independent call path and was never covered: its
call_llm(task="moa_aggregator", ...) sent a single undecorated user message
containing the full joined reference output, re-billing the entire input on
every invocation even when the resolved aggregator slot is a cache-honoring
route (Claude on OpenRouter/native Anthropic, MiniMax, Qwen/DashScope).
- Generalize _maybe_apply_advisor_cache_control to
_maybe_apply_moa_cache_control (it never had advisor-specific logic --
same policy function, same breakpoint layout as the main loop, judged
purely on the passed-in runtime) and reuse it in aggregate_moa_context
the same way _run_reference already does.
- Compute _slot_runtime(aggregator) once and reuse it for both the
decoration call and the call_llm kwargs, instead of calling it twice.
Mutation-verified: reverting the moa_loop.py change makes the new
regression test fail by asserting a plain string aggregator-message
content where the cache-honoring case expects native cache_control
content blocks.
Both wired against features the iron-proxy author (@mslipper) confirmed on
PR #30179 — and both verified present in the pinned v0.39.0 source.
Header-auth providers (match_headers):
- New _HEADER_AUTH_PROVIDERS: Anthropic native (x-api-key), Azure OpenAI
(api-key on *.openai.azure.com / *.cognitiveservices / *.services.ai),
Gemini (x-goog-api-key + ?key= query param via match_query).
- TokenMapping grows match_headers + alias_env_names; per-provider header
sets flow into the secrets rules; mappings.json roundtrips them
(legacy files load with the Authorization default).
- GEMINI_API_KEY / GOOGLE_API_KEY collapse into ONE mapping (two
require-rules on the same host would reject each other); the sandbox
gets the token under both names, and the proxy child env mirrors the
alias into the canonical name when only the alias is set.
- Docker backend injects alias env names alongside canonical ones.
- The fail-closed tier is now empty, so fail_on_uncovered_providers and
discover_blocked_providers are deleted (dead toggle otherwise);
_NON_BEARER_PROVIDERS shrinks to genuinely-unswappable signature auth
(AWS SigV4, GCP service-account OAuth) — warn-only, as before.
Management API (hot reload):
- Generated proxy.yaml enables the v0.39 management listener: loopback
only at tunnel_port+2, bearer key from HERMES_IRON_PROXY_MGMT_KEY.
- Key minted at setup (management.token, 0600); start_proxy injects it
(v0.39 refuses to start when api_key_env is empty).
- hermes egress reload -> POST /v1/reload: re-reads proxy.yaml and
atomically swaps the pipeline; 422 leaves the running ruleset
untouched; actionable errors for not-running / pre-management config /
key mismatch. Secrets changes still require restart (daemon env is
read at spawn) — the CLI says so.
Validation: 218/218 unit+CLI+docker tests; 3/3 gated live E2E against the
real v0.39.0 binary (Authorization swap, x-api-key swap, live reload with
token rotation on the same pid). Docs updated.
Two review findings on the #57922 salvage:
1. Stale inline comment at the login-exchange site still claimed the token
endpoint uses the claude-code/ UA prefix and 404s claude-cli/ — now
contradicts the axios/ fix. Repointed it at _OAUTH_TOKEN_USER_AGENT.
2. The inherited Path.home test isolation on the three TestRefreshOauthToken
tests only stubbed the ~/.claude *file* source, not the macOS Keychain.
_refresh_oauth_token re-reads read_claude_code_credentials() (keychain
first) in its adopt-already-refreshed branch, so on any macOS dev/CI runner
with real Claude Code creds the branch short-circuits and the 3 tests fail.
Stub read_claude_code_credentials -> None so the tests are hermetic.
(The remaining TestResolveAnthropicToken/TestResolveWithRefresh/TestRunOauthSetupToken
failures on macOS are the same pre-existing keychain-leak class on origin/main,
unrelated to this OAuth-UA fix, and pass in CI — left out of scope.)
hermes auth add anthropic fails 100% at token exchange with HTTP 429 while
Claude Code /login succeeds through the same client_id/redirect/scope. The
discriminator is the User-Agent on the /v1/oauth/token request.
Verified live against platform.claude.com (throwaway code, nothing burned):
claude-code/2.1.200 (external, cli) -> 429 rate_limit (Hermes, blocked)
Mozilla/5.0 -> 429 rate_limit
axios/1.7.9 -> 400 invalid_grant (reached validation)
node / empty / SDK-style UAs -> 400 invalid_grant
Anthropic now rate-limits token-endpoint requests whose UA starts with
claude-code/ (the anti-abuse net for Max-sub-as-API-key). This is the same
prefix-block shape that #48534 first hit on claude-cli/, then #56263 dodged by
switching to claude-code/ — which held ~2 weeks and is now blocked too. Bumping
_CLAUDE_CODE_VERSION_FALLBACK cannot help; the gate is prefix-based.
Fix: shared _OAUTH_TOKEN_USER_AGENT (axios/) on the token endpoint only — the
two refresh POSTs (refresh_anthropic_oauth_pure) and the login exchange POST
(run_hermes_oauth_login_pure). The real Claude Code CLI exchanges the auth code
with a bare axios client, NOT its claude-code/ inference UA.
The INFERENCE client (build_anthropic_kwargs, /v1/messages) is deliberately left
on claude-code/ + x-app: cli — that fingerprint is required there and is NOT
throttled on the messages API. Two endpoints, opposite UA requirements.
Also isolate two _refresh_oauth_token tests from live ~/.claude creds and update
the UA regression tests to assert the split (token endpoint uses a
non-claude-code UA while inference keeps claude-code/).
Verified E2E: Hermes' own login path now returns 400 (past the 429 wall)
instead of 429, using the real _OAUTH_TOKEN_USER_AGENT constant against the live
platform.claude.com token endpoint.
Salvaged from #57922 (authorize-host + scope changes dropped as non-load-bearing;
they only add a redirect hop back to claude.ai and the UA fix alone clears 429).
repair_message_sequence Pass 1 registered only tc.get("id") when building
the set of known assistant tool_call ids, then matched tool results against
it by tool_call_id. In the Codex Responses format an assistant tool_call
carries both id (fc_...) and a distinct call_id (call_...); a tool result's
tool_call_id may be keyed on either depending on which builder produced it.
Registering only id made a valid tool result whose tool_call_id matched
call_id look orphaned, so the pass dropped it and left the assistant
tool_call unanswered -- producing HTTP 400 on strict providers (DeepSeek,
Kimi): 'Messages with role tool must be a response to a preceding message
with tool_calls'. Long-running sessions that persisted such a sequence were
permanently broken, re-sending the orphan every turn.
Register both id and call_id for each assistant tool_call so a result
matching either key is recognized, consistent with
AIAgent._get_tool_call_id_static and the compressor's _sanitize_tool_pairs.
Apply the same call_id||id precedence to the corrupted-args sanitizer's
existing-result scan / stub insertion, which had the identical mismatch.
Adds 3 regression tests covering the codex id!=call_id case (match on
call_id, match on only call_id, match on id when both present).
PR #57601's original branch added a top-level reasoning_effort emit to the
LEGACY build_kwargs path (agent/transports/chat_completions.py), but
provider=custom resolves to CustomProfile (plugins/model-providers/custom/),
so chat_completion_helpers takes the profile path and returns early — the
added branch was unreachable dead code for every custom endpoint.
Move the fix to its real site, CustomProfile.build_api_kwargs_extras(), and
follow the DeepSeek/Zai profile precedent:
- disabled -> extra_body.think = False (unchanged)
- enabled + effort -> TOP-LEVEL reasoning_effort (the OpenAI-compatible
format GLM-5.2/ARK expect), passed through verbatim
incl. max/xhigh
- enabled + no effort -> omit, so the endpoint's server default applies
(avoids silently forcing 'medium' as the original
branch did)
Deliberately does NOT force think=True on enable — that flag is Ollama-only
and risks a 400 on GLM/vLLM endpoints that don't recognize it; thinking is
already server-default-on for these backends.
Verified end-to-end through the real profile dispatch (temp HERMES_HOME):
custom+high -> reasoning_effort=high; custom+max -> reasoning_effort=max;
custom+none -> think=False; custom+unset -> nothing; num_ctx composes.
Adds tests/plugins/model_providers/test_custom_profile.py (13 cases).
Addresses the custom-provider half of #55276.
Co-authored-by: huanshan5195 <huanshan5195@users.noreply.github.com>
- Add 'max' to VALID_REASONING_EFFORTS (GLM-5.2 native parameter)
- Emit top-level reasoning_effort string for custom providers
- Stop hardcoding 'medium' in legacy extra_body.reasoning, use actual effort
Custom providers (e.g. GLM-5.2 on Volcengine ARK) silently dropped
reasoning_effort — the value never reached the upstream API. Kimi,
TokenHub, and LM Studio all had dedicated branches for this, but
custom providers had none.
Self-review (hermes-pr-review Phase 2) flagged the mid-turn pre-API
compaction for reuse/duplication (W1/W2); fixing that surfaced a
regression against a deliberate existing feature, now also fixed.
The block now mirrors the turn-prologue preflight's guard chain exactly
(agent/turn_context.py) instead of a hand-rolled pressure limit:
1. should_defer_preflight_to_real_usage(rough) — defer when the rough
estimate is known-noisy vs a recent real provider prompt that fit
under threshold (schema overhead / post-compaction over-count, #36718).
2. get_active_compression_failure_cooldown() — skip during a same-session
compression-failure cooldown.
3. should_compress(rough) — reuses the canonical threshold_tokens (output
room already reserved by _compute_threshold_tokens) plus its summary-LLM
cooldown + anti-thrash guards (#11529).
Dropped the seven inline _reserve/_output_pressure locals (W2: they
re-derived _compute_threshold_tokens and omitted its 85% degenerate-window
fallback). compression_attempts stays as the hard per-turn backstop.
Without guard (1) the block fired a compaction the preflight deliberately
defers, breaking test_413_compression::test_preflight_defers_when_recent_
real_usage_fit (ValueError from the mocked _compress_context). Verified:
test_413_compression 26/26, codex 82/82, and anti-thrash engaged
(_ineffective_compression_count=2) still suppresses the block (0 calls).
The salvaged max-output/pressure fix set conversation_history=None after
the new pre-API compaction. That is only correct for legacy session-
rotation. Under the default in-place compaction (compression.in_place:
True), archive_and_compact inserts the compacted rows into the session DB
directly without stamping them with the intrinsic persisted-marker, so a
subsequent flush with conversation_history=None re-appends them — doubling
the active context and retriggering compression (the early-persist
duplicate-row trap).
Use conversation_history_after_compression(agent, messages), matching the
two existing compaction sites (post-response should_compress and the
turn-prologue preflight), which returns None for rotation and
list(messages) for in-place so the compacted dicts are skipped by identity.
Adds a regression test with a real SessionDB + real archive_and_compact
that asserts the compacted summary row is persisted exactly once (fails
with 2 copies on the None variant).
Follow-up to the salvaged #57845 fix. _can_carry_marker used
any(isinstance(part, dict)) but _apply_cache_marker only marks the LAST
content part, so a list whose last element is a non-dict passed the carrier
gate yet received no marker — wasting one of the four breakpoints. Tighten
the predicate to require content[-1] to be a dict (mirroring the apply
logic) and add a regression test. Flagged by a 3-agent review.
Follow-up to the per-provider guards. Three improvements from review:
1. Extract agent.file_safety.raise_if_read_blocked() as a single shared
chokepoint and route the OpenAI, OpenRouter, and (newly) xAI image
providers through it, replacing the 3x-duplicated inline try/except.
Fixes the whole bug class: xai/_xai_image_field read a model-supplied
local path via open() with no guard — the same vulnerability the PR
fixed for OpenAI/OpenRouter, in a sibling provider it missed.
2. Strengthen the regression tests from pass-on-any-ValueError to true
security invariants: spy open()/read_bytes() and assert the blocked
credential is NEVER read; add negative controls (legit local image
still loads; remote/data: URIs pass through unguarded) so a
block-everything regression can't pass.
3. Guard is best-effort by design (defense-in-depth, not a security
boundary) — documented on the shared helper.
- agent/file_safety.py: raise_if_read_blocked()
- plugins/image_gen/{openai,openrouter,xai}: route through helper
- tests: no-read spies + negative controls across all three providers
Two caching holes made MoA re-bill essentially its entire input stream:
1. AGGREGATOR: anthropic_prompt_cache_policy() judged the agent's own
model/provider — on the MoA path those are the virtual preset name and
'moa', which match no caching branch, so _use_prompt_caching was False
and the acting aggregator (Claude on OpenRouter) ran with ZERO
cache_control breakpoints. Measured on identical opus-4.8 sessions:
85% cache share solo vs 2% via MoA — ~30M re-billed input tokens on one
132-task benchmark run. Fix: when provider == 'moa', resolve the policy
from the preset's real aggregator slot (provider/model/base_url/api_mode
via resolve_runtime_provider).
2. ADVISORS: _run_reference never applied cache_control at all, and
Anthropic caching is opt-in per request — Claude advisors served 0
cache reads across 1,227 benchmark calls (11.5M re-billed input tokens)
even though the advisory view is append-only across iterations (stable
prefix; the synthetic end marker is last so it never pollutes it). Fix:
_maybe_apply_advisor_cache_control() reuses the SAME policy function and
SAME system_and_3 layout as the main loop, judged on the advisor slot's
own resolved runtime — advisor requests are now decorated exactly like
an acting agent on that provider. Auto-caching routes (OpenAI-family)
are left untouched by policy.
Live-verified on the wire (per-iteration opus+gpt5.5 preset, 4 fan-outs):
claude advisor fan-out 2-3 cache_write=2161/2344, fan-out 4
cache_read=2206 / fresh_in=2; aggregator session cache share 84%/77%
(vs 2%/0% before). Sub-1024-token prompts correctly stay uncached
(Anthropic minimum).
A custom:<name> main provider resolves at runtime to the bare provider id
"custom". In the vision auto-detect chain, the main-provider branch called
resolve_provider_client("custom", ...) WITHOUT explicit_base_url/api_key,
so it returned (None, None) ("no endpoint credentials found") and the whole
chain fell through to OpenRouter/Nous. A user on a custom endpoint with no
aggregator configured then got "No LLM provider configured for task=vision
provider=auto" on every image, even though their main model fully supports
vision.
Recover the live endpoint that set_runtime_main() records each turn
(_RUNTIME_MAIN_BASE_URL/_API_KEY/_API_MODE) and forward it to Step 1, with
a fallback to _resolve_custom_runtime() for non-gateway callers. Mirrors the
existing explicit-base_url branch directly above.
Adds TestResolveVisionCustomProvider covering custom, custom:<name>, and the
no-runtime fallback path.
When model.provider is set to custom:<name>, _supports_vision_override()
previously tried only the runtime provider key ('custom') and the raw
config value ('custom:my-proxy'). It did not try the stripped name
('my-proxy'), which is the actual key under providers: in config.yaml.
This caused native image routing to fall back to text mode even when the
user explicitly declared supports_vision: true on the named provider's
model entry.
Fixes#39963
The advisory view appends a synthetic user marker when it ends on an
assistant turn (Anthropic end-on-user rule) — i.e. on every tool iteration
after the first. The user_turn prefix hash treated that marker as the last
user message, so the hashed prefix included the grown mid-turn context and
the signature changed every iteration: advisors re-ran per iteration,
silently defeating the once-per-turn cadence (live smoke test: 2 fan-outs
for a 2-iteration task; expected 1). Hoist the marker to a module constant
and skip it when locating the last REAL user message. Verified: iteration-2
signature now equals iteration-1 (cache HIT); a new real user message still
re-triggers the fan-out.
New preset key 'fanout': 'per_iteration' (default, unchanged behavior)
re-runs the reference fan-out whenever the advisory view changes — every
tool iteration. 'user_turn' runs the advisors ONCE per user turn and lets
the aggregator act alone for the rest of the tool loop — the original MoA
shape (upfront multi-model synthesis, then a single acting model), and the
obvious lever on MoA's wall/cost multiplier (advisor generation dominates
per-turn latency).
Implementation reuses the existing turn-scoped reference cache: in
user_turn mode the cache signature hashes only the prefix up to the LAST
user message, so mid-turn advisory-view growth doesn't change the key and
iteration 2+ is a cache HIT (advice reused, zero advisor spend, no
re-trace). A new user message changes the prefix and re-triggers the
fan-out. Unknown fanout values normalize to per_iteration.
A single-model Hermes agent never sends temperature; the provider default
applies. MoA hardcoded reference_temperature=0.6 / aggregator_temperature=0.4,
and the coercion float(preset.get(key, 0.6) or 0.6) made unset IMPOSSIBLE to
express: absent, null, empty, and even an explicit 0 all collapsed to the
baked-in default. Every MoA advisor and aggregator therefore ran at 0.6/0.4
while the same model running solo used the provider default — silently
skewing solo-vs-MoA comparisons and overriding provider-tuned defaults.
- moa_config normalization: temperatures coerce to None when absent/blank/
invalid (new _coerce_float_or_none); explicit values incl. 0 honored.
- moa_loop: _preset_temperature() resolves preset values; None flows to
call_llm, which already omits the parameter when None (same contract as
max_tokens). Aggregator still inherits the acting agent's own configured
temperature when the preset doesn't pin one.
- conversation_loop (context-mode MoA): same resolution, no more hardcoded
0.6/0.4 at the call site.
- DEFAULT_CONFIG preset + web_server payload models + docs updated: unset
is the default, pinning stays available.
Follow-up to the per-site strips from the review gate. The two copy-site
strips are correct but positional — a copy site added after the assembly
loops would re-leak _db_persisted into the child-session flush. Add a single
terminal sweep (_strip_persistence_markers) run once on the fully-assembled
compressed list so the invariant 'no compacted message leaves compress()
carrying a persistence marker' is structural, not dependent on copy-site order.
- agent/context_compressor.py: _strip_persistence_markers() called before
compress() returns; helper docstring notes the sweep is the authoritative guard
- tests/agent/test_context_compressor.py: structural regression — neuter the
per-site helper to a leaking copy, assert the terminal sweep still strips
- tests/run_agent/test_compression_persistence.py: pin the fixture assumption
behind the exact-equality row-count assertion
Shallow messages[i].copy() during context compression propagated the
_db_persisted marker from cached gateway incremental flushes into the
post-rotation compressed list. _flush_messages_to_session_db then skipped
every row when writing to the new child session, so gateway restarts
lost the compacted transcript (severe amnesia).
Strip the marker in _fresh_compaction_message_copy() and add regression
tests for rotation flush + compressor assembly.
Fixes#57491
Self-review (ruff+ty lint diff = 0 net-new; 2-agent deep review) surfaced one
Warning + comment-accuracy nits; no Critical:
- W1: the local-probe TTL cache memoized None (probe failure) for 30s, so a
probe that failed during a startup race would suppress a legit retry once
the server came up. Cache only positive results — still fully bounds the
hot-path probe rate (reachable servers cache their value) while an
unreachable one re-probes on the next call. Add a regression test asserting
a None result is NOT cached (retry re-probes); mutation-verified.
- Tighten the platform-guard comment: gateway/TUI/cron already construct with
quiet_mode=True (gated by `not agent.quiet_mode`), so the guard's active job
is CLI dedup vs show_banner, not "filling the gateway/TUI gap" as originally
worded.
Verified not-issues (per review): positive-value 30s cache does not break the
reconcile-after-restart freshness contract (restart = fresh process, empty
cache); cache key is collision-safe; platform guard is correct in both
directions (no runtime path leaves platform None on a non-CLI surface).
Tests: 149 passed. ruff clean; ty 0 net-new vs base.
Salvage review of #56431 surfaced one Critical + two Warning issues; fix
them on top of the contributor's cherry-picked commits:
1. Critical — duplicate non-agentic warning on the interactive CLI. The new
agent_init warning fires on every platform, but cli.py show_banner()
already warns on CLI (richer output + /model hint), so a CLI user saw the
warning twice per startup. Guard the agent_init emit to skip platform=="cli"
— it now fills exactly the gateway/TUI gap the PR intended, no duplication.
2. Warning — vLLM error-parse regex under-matched. The patterns required a
literal space before the number, so "max_model_len: 32768", "=32768",
"(32768)", and "... is 32768" all returned None. Broaden both patterns to
accept :/=/(/ 'is' delimiters. Add a parametrized test over all delimiter
variants.
3. Warning — per-call live probe latency on local endpoints. The new
reconcile-on-hit + pre-defaults step-7 probe made every local resolution
fire a synchronous network probe (banner + /model switch + compressor
update_model each within one startup). Add a 30s in-process TTL cache
keyed by (model, base_url) around _query_local_context_length so back-to-
back resolutions reuse one round-trip; not persisted to disk, so the
reconcile freshness contract (re-probe after restart) is preserved. Add an
autouse fixture clearing the cache between tests + TTL coverage.
Tests: 148 passed (was 138). ruff clean.
Reconcile stale local disk cache against live vLLM/Ollama max_model_len
probes, probe local servers before the llama hardcoded default, parse
vLLM max_model_len overflow errors, and surface the non-agentic Hermes 3/4
warning at agent init on gateway/TUI.
Sub-64K live probes are returned for startup rejection but are not
persisted to the context cache — preserving the 64K minimum-context
contract instead of normalizing undersized windows as valid config.
(cherry picked from commit c3a02db4fd9d57b7b0eb2732de91f8334d311aa5)
normalize_usage only read output_tokens_details.reasoning_tokens (the
Responses API shape). Chat Completions providers — OpenAI, OpenRouter,
DeepSeek, and every OpenAI-compatible proxy — report it under
completion_tokens_details.reasoning_tokens, so reasoning_tokens was 0 for
every chat_completions reasoning model: hidden thinking was invisible in
session accounting, MoA traces, and the eval's per-task token columns.
Measured impact (HermesBench MoA run on deepseek-v4-flash, 4,828 advisor
calls): reasoning_tokens showed 0 everywhere while individual calls burned
up to 21.5K hidden thinking tokens to emit ~500 visible tokens. Verified
live against OpenRouter: deepseek-v4-flash returns
completion_tokens_details.reasoning_tokens=61 for a 74-completion-token
call; the field was simply never read.
Responses-shape reads are unchanged; the new read only fires when the
Responses shape yielded nothing.
The terminal-refresh quarantine filtered in-memory entries on
source == "device_code" but built removed_ids from the deleted
"loopback_pkce" source name, so the revoked device-code entry was
never pruned from the persisted pool in auth.json. Also restores the
_print_loopback_ssh_hint test suite scoped to Spotify (the helper's
remaining caller) instead of deleting it wholesale.
Replace the loopback/PKCE-callback server and manual-paste fallback with
the RFC 8628 device-code flow as the only xAI Grok OAuth login path. The
flow works in headless/SSH/container sessions with no 127.0.0.1 listener,
shrinking the local attack surface.
- Poll the token endpoint with server-provided interval, honoring
slow_down and expires_in; store tokens with auth_mode
oauth_device_code.
- Adaptive proactive refresh skew for short-lived device-code JWTs;
rotated tokens sync back to auth.json, the global root store, and the
credential pool (no refresh-token replay).
- Clear source suppression on successful re-login (CLI + dashboard) and
drop the duplicate dashboard pool entry so exactly one seeded
device_code entry exists.
- Use the shared device_code source name for consistency with the
nous/codex device-code providers.
- Desktop: remove the loopback OAuth flow states and dead type variants;
pkce providers' sign-in URL selection is unchanged.
- Docs (EN + zh-Hans) rewritten for device-code login; drop the deleted
--manual-paste flag from documented commands.
Named providers / custom_providers entries in config.yaml now accept an
extra_headers dict scoped to that endpoint — for reverse proxies, API
gateways, and custom auth schemes (e.g. Cloudflare Access service tokens).
- hermes_cli/config.py: normalize extra_headers on provider entries
(_normalize_custom_provider_entry + providers-dict translation), add
get_custom_provider_extra_headers /
apply_custom_provider_extra_headers_to_client_kwargs helpers keyed on
base_url (case/trailing-slash insensitive, no substring bypass —
mirrors the TLS helpers)
- hermes_cli/runtime_provider.py: surface extra_headers in the resolved
runtime for named custom providers (providers dict, legacy
custom_providers list, and the credential-pool path)
- run_agent.py / agent/agent_init.py: merge per-provider extra_headers
onto the OpenAI client default_headers at construction and on every
_apply_client_headers_for_base_url re-application (credential swaps,
rebuilds), most-specific level wins; OpenAI-wire only (native
Anthropic/Bedrock scoped out)
- agent/auxiliary_client.py: accept model.extra_headers as an alias of
model.default_headers for the global variant
- cli-config.yaml.example: documented commented example
- Header values are treated as secrets and never logged
Salvaged from PR #3526 by @jneeee, reimplemented against current main.
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
The helper docstring described the typical ~15-25k gateway payload but
read as if that were the trigger range; the floor actually engages above
10k tokens. Clarify the prose to match the gate.
Lower the openai-codex stale-timeout floor from 25k to 10k estimated
tokens so Telegram/gateway sessions (~20k tools+instructions) are not
aborted at the generic 90s cutoff while Codex is still prefilling.
Three CLI reliability fixes:
1. Interrupt reliability: chat() only re-queued the user's interrupt
message when the turn result carried interrupted=True. When the agent
thread raced past its last interrupt check (or finished) before the
interrupt landed, the message was silently dropped — and the stale
_interrupt_requested flag left on the agent instantly aborted the
NEXT turn. Un-acknowledged interrupt messages are now re-queued as
the next turn and the stale flag is cleared (only when the agent
thread actually exited). The clarify-race path also parks the message
in _pending_input instead of dropping it.
2. Slow exit (5+ min): stdlib ThreadPoolExecutor workers are non-daemon
and joined unconditionally by concurrent.futures' atexit hook — even
after shutdown(wait=False). One wedged tool worker (abandoned after
interrupt/timeout) held the process open forever. Promoted
async_delegation's daemon executor to a shared tools/daemon_pool
module and adopted it in tool_executor (concurrent tool batches),
memory_manager (background sync), delegate_tool (child timeout wrapper
+ batch fan-out), and skills_hub (source fan-out). Added a 30s exit
watchdog (HERMES_EXIT_WATCHDOG_S) armed at _run_cleanup start as a
backstop for wedged cleanup steps.
3. Exit jank: after prompt_toolkit tears down the input/status bars the
terminal sat silent for the whole cleanup window, looking hung. Print
'Shutting down… (finalizing session)' immediately at exit start.
E2E: live PTY interrupt of a foreground 'sleep 120' terminal tool now
aborts in ~1s and the typed message runs as the next turn; wedged-worker
+ wedged-cleanup subprocess exits in 5.8s (watchdog) instead of hanging.
Follow-up on the salvaged #56392 guard. The cherry-picked change matched
custom:<name> pool entries against the primary by raw base_url string
equality, which (a) can't disambiguate two named custom providers sharing
one gateway base_url and (b) left a latent bare-"custom" entry bypass.
Route the match through get_custom_provider_pool_key(rt[base_url]) compared
against the entry's custom:<name> key, mirroring the sibling guard in
recover_with_credential_pool. Use CUSTOM_POOL_PREFIX instead of the literal.
Add regression tests for the custom same-endpoint (swap) and cross-endpoint
(skip) branches, plus the plain-provider fallback-pool case from #56885.
Two related hardening fixes for auxiliary calls (which include MoA reference
advisors — a pinned-model path where provider fallback is not a meaningful
recovery):
1. Transient-transport retries: the same-provider retry on a connection reset /
timeout / 5xx / 408 was a single attempt, then fallback. For a pinned aux
call a second blip silently loses the call (root of the run2 double-advisor
'Connection error' collapse — a genuine upstream blip). Now retries N times
with exponential backoff, N = auxiliary.transient_retries (default 2 -> 3
total attempts, clamped [0,6]). Compression-on-timeout fast-fail carve-out
preserved.
2. Per-model client-cache isolation: _client_cache_key excluded the model, so
two concurrent auxiliary calls to the same provider/base_url/key but
different models (e.g. an opus + gpt-5.5 MoA fan-out) shared one cache entry
and could race each other's client lifecycle. Model now participates in the
key -> distinct clients, no cross-call races. Same-model reuse unchanged.
- agent/auxiliary_client.py: _transient_retry_count() + backoff loop; model in
_client_cache_key and both call sites.
- hermes_cli/config.py: auxiliary.transient_retries default (2).
- tests: new retry/isolation tests; updated 2 stale-expectation tests to the
corrected behavior (per-model resolve; N-retry escalation).
Backoff base is overridable (_TRANSIENT_RETRY_BACKOFF_BASE) so tests don't sleep.
MoA per-turn latency is dominated by advisor GENERATION: turn wall time
correlates ~0.88 with output tokens and ~-0.03 with input tokens (measured over
52 turns). Each turn waits for the slowest advisor to finish writing, and
advisors were uncapped — writing multi-thousand-token essays the aggregator
only needs the gist of.
Add an opt-in per-preset reference_max_tokens knob (mirrors reference_temperature)
that caps ADVISOR output only; the acting aggregator is never capped. Default
None = uncapped, so existing presets are byte-for-byte unchanged (no regression).
Wired through both MoA execution paths (MoAChatCompletions.create and
aggregate_moa_context).
E2E: same task, closed preset uncapped vs reference_max_tokens=600 -> 59s to 33s
(~44% faster), final answer identical/correct.
- hermes_cli/moa_config.py: _coerce_int_or_none helper + reference_max_tokens
in _normalize_preset/_default_preset/flattened view
- agent/moa_loop.py: read preset.reference_max_tokens, pass to reference fan-out
- agent/conversation_loop.py: pass reference_max_tokens on the per-turn path
- tests + docs
* fix(streaming): handle completed responses with empty/None choices
The streaming fallback guard added in #55932 recognized a completed
response object only when its `choices` was a non-empty list. But an
adapter can return a completed response whose `choices` is `None` or an
empty list (an error / content-filter / terminal frame) — still a whole,
non-iterable response, not a token stream. Those shapes fell through to
`for chunk in stream` and crashed with
'types.SimpleNamespace' object is not iterable
which is exactly issue #55933 (MoA `openai-codex` aggregator on
TUI/Desktop, where a stream consumer forces the streaming path).
Broaden the guard to discriminate on the PRESENCE of a `choices`
attribute (a genuine provider Stream object exposes none), disable
streaming for the session, and return the completed object so the outer
loop's normal invalid-response validation handles empty/None choices via
its retry path instead of iterating.
Based on the diagnosis in #56525 by @spiky02plateau (that PR normalized
the MoA aggregator return with a one-shot chunk iterator; the common
text/tool-call crash was already fixed at this seam by #55932, so this
extends the existing guard to cover only the remaining empty/None-choices
gap).
Fixes#55933
* refactor(streaming): simplify empty-choices guard body and parametrize tests
Post-review cleanup (no behavior change):
- Inline the single-use `response_choices` local and drop the redundant
`if first_choice is not None else None` guard (getattr(None, ...) already
returns the default safely).
- Collapse the two near-identical empty/None-choices regression tests into
one `@pytest.mark.parametrize` case.
Mutation-verified: reverting the guard to the old non-empty-list condition
still makes both parametrized cases fail with the historical
'types.SimpleNamespace' object is not iterable.
---------
Co-authored-by: spiky02plateau <155588579+spiky02plateau@users.noreply.github.com>
agent/vertex_adapter.py resolved VERTEX_CREDENTIALS_PATH,
GOOGLE_APPLICATION_CREDENTIALS, VERTEX_PROJECT_ID, and VERTEX_REGION via raw
os.environ.get() instead of the profile-scoped get_secret() every other
credential lookup in hermes_cli/runtime_provider.py uses. In a multiplex
gateway serving several profiles from one process, os.environ still holds
whichever profile's .env python-dotenv loaded at boot — so a raw read here
let one profile's turn silently mint a Vertex OAuth2 token from, and get
billed against, a different profile's GCP service account. No error, no
fail-closed guard: the multiplex UnscopedSecretError protection was bypassed
entirely because these reads never went through get_secret().
- _resolve_credentials_path/_resolve_project_override/_resolve_region now
call agent.secret_scope.get_secret(), matching the _getenv() pattern
already used for every other provider's credentials.
- get_vertex_credentials()'s ADC fallback (google.auth.default()) reads
GOOGLE_APPLICATION_CREDENTIALS from os.environ internally, bypassing
get_secret() entirely — closed with a narrow guard: when multiplexing is
active and this profile's scope has no Vertex credentials of its own, but
os.environ still carries a value (left by a different profile's boot-time
dotenv load), refuse ADC rather than silently authenticate as a stranger.
- Zero behavior change for single-profile installs: get_secret() falls
through to os.environ transparently whenever multiplexing is off.
Same bug class as the already-fixed _HERMES_OAUTH_FILE/_AUTH_JSON_PATH/
HOOKS_DIR cross-profile leaks, now closed for Vertex's OAuth2 credential
path.
The salvaged fix wired per-provider ssl_ca_cert / ssl_verify (and
HERMES_CA_BUNDLE) into the MAIN OpenAI client. This follow-up:
- Auxiliary client parity: process_bootstrap.build_keepalive_http_client
accepts and forwards verify; auxiliary_client._resolve_aux_verify mirrors
the main-client TLS resolution (via load_config_readonly, the read-only
fast path) so compression/vision/web_extract/title-gen/session_search
honor the same per-provider CA. Without this, chat worked against a
private-CA endpoint but every auxiliary call still failed APIConnectionError.
- switch_model now reads custom_providers from live config (load_config_readonly)
instead of the init-time agent._custom_providers snapshot, so ssl_ca_cert /
ssl_verify edits are honored on mid-session model switch — matching the
context-length reload (#15779).
- Drop the dead client-level verify= where a custom httpx transport is used
(httpx ignores it there); verify lives on the transport. Fix docstrings.
Applies to both run_agent._build_keepalive_http_client and process_bootstrap.
- resolve_httpx_verify: add CURL_CA_BUNDLE to the env chain (consistency with
agent/ssl_guard._CA_BUNDLE_ENV_VARS) and emit a loud logger.warning naming
the endpoint whenever ssl_verify:false disables verification.
- get_custom_provider_tls_settings: case-insensitive base_url match (config
dedup already lowercases; scheme/host are case-insensitive) so a mixed-case
entry doesn't silently drop its CA. Exact match preserved — no prefix bypass.
- Demote best-effort except Exception: pass in agent_init/switch_model to
logger.debug(exc_info=True).
- Tests for aux verify forwarding, _resolve_aux_verify, case-insensitive
match, and prefix-bypass rejection.
Wire ssl_ca_cert and ssl_verify through custom_providers config and env
vars into the keepalive httpx client, fixing APIConnectionError against
mkcert/self-signed Ollama proxies behind HTTPS.
parse_frontmatter's malformed-YAML fallback stores every value as a string,
so a skill's `metadata` can be a str. `_category`/`_related` chained
`.get("metadata", {}).get("hermes", {})` and blew up with `'str' object has
no attribute 'get'`, taking down `build_learning_graph()` (and thus /journey
and `hermes journey`) whenever any installed skill had bad frontmatter.
Extract a `_hermes_meta()` helper that returns the nested dict only when it
really is one. Fixes the whole class, not just the two call sites.
Self-review follow-up on the salvaged approval-routing fix.
The initial adaptation re-read os.getenv("HERMES_YOLO_MODE") at session-build
time. That diverges from the repo's security invariant: HERMES_YOLO_MODE is
frozen into tools.approval._YOLO_MODE_FROZEN at import time precisely so a skill
running mid-process cannot set the env var and instantly flip the approval
bypass (a prompt-injection escalation path). A live re-read re-opened that hole
for the codex routing path.
- Add tools.approval.is_approval_bypass_active() — the canonical three-source
bypass check (frozen --yolo/HERMES_YOLO_MODE + session /yolo + approvals.mode
off) in one place. This is the 4th inline copy of that OR-chain (the three
sites in approval.py and tui_gateway/server.py:3121 all use the same idiom);
the helper is the shared chokepoint they can collapse onto.
- codex_runtime.py now calls is_approval_bypass_active() instead of the
hand-rolled mode-or-session check plus a runtime env re-read.
- Update the env-yolo test to patch _YOLO_MODE_FROZEN (the canonical test
pattern, e.g. tests/tools/test_yolo_mode.py) rather than setenv, which is
dead-on-arrival against the frozen constant.
Fail-closed default preserved on every branch; 28 integration + 77 session/yolo
tests pass; E2E confirms the real exec decision flips decline->accept only when
bypass is active.
On gateway/cron/non-CLI contexts the codex app-server runtime has no UI to
surface codex's exec/apply_patch approval requests, so they fail closed
(silently decline) — the bot appears responsive but cannot write files, with
no approval prompt anywhere ("patch rejected by user").
When the user has explicitly opted out of Hermes approvals (approvals.mode: off,
the /yolo session toggle, or HERMES_YOLO_MODE=1), collapse to codex's own
sandbox permission profile (~/.codex/config.toml) as the policy gate by passing
_ServerRequestRouting(auto_approve_exec=True, auto_approve_apply_patch=True) to
the session. Defaults (manual/smart/unset) preserve the current fail-closed
behavior — a no-op for users who have not opted out.
Reads the mode via the canonical tools.approval._get_approval_mode() (which
already normalizes the YAML-1.1 bare-'off'->False case) at session-build time,
so a mid-session /yolo toggle is honored too.
5 integration tests: each opt-out mechanism (config off, YAML False, env var,
session yolo) plus the default fail-closed regression guard.
Closes#26530
Co-authored-by: snav <jake@nousresearch.com>
Two independent MoA auxiliary-call fixes:
#53866 — auxiliary.moa_reference.timeout and auxiliary.moa_aggregator.timeout
were 600s while moa_agent was 120s. Raise both to 900s so a genuinely long
reference/aggregator turn (mixed providers, deep reasoning, long tool chains)
has headroom instead of being cut mid-generation.
#53735 — _CodexCompletionsAdapter (the Codex/Responses auxiliary path used by
the MoA acting-aggregator, compression, web_extract, session_search, etc.)
never set prompt_cache_key, so it stayed cache-cold while the MAIN Responses
transport (agent/transports/codex.py) was warm. Derive the same
content-addressed key via the shared _content_cache_key(instructions, tools)
helper and set it on the aux Responses request, with the same host guards the
main transport uses (xAI carries the key in extra_body; GitHub/Copilot opts out
of cache-key routing).
Tests: 5 new prompt_cache_key cases (set+prefixed, stable across identical
prefix, differs on different instructions, skipped for xai/github hosts).
tests/agent/test_auxiliary_client.py 279 pass; tests/hermes_cli/test_config.py
130 pass.
On the MoA path agent.model/provider are the virtual preset name (e.g.
"closed") and "moa", which have no pricing entry. estimate_usage_cost()
returned None for the aggregator turn, so the `if amount_usd is not None`
guard skipped it and the session's estimated_cost_usd reflected only the
advisor fan-out — a ~50% undercount when the aggregator does the full acting
loop (verified: $0.91 advisor-only vs $1.96 true, aggregator = 54%).
MoAChatCompletions.create() now stashes the resolved aggregator slot as
last_aggregator_slot (exposed via MoAClient); conversation_loop reads it to
price the aggregator turn at its real model/provider. cost_source flips from
'none' to 'provider_models_api'.
Follow-up to the END-MARKER reorder: moving the summary prefix after the
[PRIOR CONTEXT] wrapper meant _is_context_summary_content (prefix-at-start)
no longer recognized a merged-tail summary. That silently broke three
consumers — the last-real-user anchor (would pick the merged summary as a
real user turn, causing active-task loss), the carry-forward summary find,
and the auto-focus skip. _strip_summary_prefix would also carry the wrapper
+ stale tail content forward as the next summary body.
Extract the two delimiter strings into _MERGED_PRIOR_CONTEXT_HEADER /
_MERGED_SUMMARY_DELIMITER constants (writer + detector stay in sync), teach
_is_context_summary_content and _strip_summary_prefix to look past the
delimiter, and add a regression test. Standalone summaries unchanged.
When the compression summary is merged into the first tail message
(the alternation corner case where a standalone summary role would
collide with both head and tail), the old format was
SUMMARY + END_MARKER + OLD_TAIL_CONTENT — so the preserved tail content
appeared AFTER the end marker and the model could read it as a fresh
message to respond to.
Reorder so the END MARKER is always last: old tail content is wrapped in
[PRIOR CONTEXT ...][END OF PRIOR CONTEXT — COMPACTION SUMMARY BELOW]
delimiters, then the summary, then the END MARKER. _append_text_to_content
handles both string and multimodal-list content.
Salvaged from #56372 by @Gromykoss. Only the END-MARKER reorder half is
carried over. The PR's second change (a post-compaction pass that strips
user-role messages before the first summary marker on compression_count>=2)
was dropped: on 2nd+ compactions the protected head decays to system-only
(_effective_protect_first_n -> 0, #11996) so the targeted 'ghost head user'
does not occur, and where the strip does fire it deletes legitimate recent
tail user turns (data loss) and can leave consecutive assistant messages
(role-alternation violation).
Adds Vertex AI as a first-class provider for Gemini models via Vertex's
OpenAI-compatible endpoint. Vertex authenticates with short-lived OAuth2
access tokens (service-account JSON or ADC), not a static API key — the
missing piece behind the recurring requests (#13484, #12639, #56259).
- agent/vertex_adapter.py: OAuth2 token minting + refresh-on-expiry
(5-min margin), ADC->service-account fallback, global vs regional
endpoint URLs. Config precedence: env var > config.yaml > default.
- plugins/model-providers/vertex/: provider profile (auth_type=vertex),
reuses Gemini's extra_body.google.thinking_config translation.
- runtime_provider: vertex short-circuit BEFORE the credential pool so a
credentials-file path is never mistaken for a static API key; mints a
fresh token + computes base_url per resolve.
- run_agent + conversation_loop: _try_refresh_vertex_client_credentials()
re-mints the token and rebuilds the client on a mid-session 401, so a
long-lived gateway agent survives token expiry (~1h).
- auxiliary_client: vertex auth_type branch for side-LLM tasks.
- config.yaml: vertex.project_id / vertex.region (non-secret, bridged to
env); credential path stays in .env (VERTEX_CREDENTIALS_PATH).
- setup wizard + model picker: dedicated _model_flow_vertex; curated
google/gemini-* model list; --provider choices.
- pricing/metadata: Vertex prices off the gemini docs snapshot; endpoint
host auto-maps to the vertex provider (no probe spam).
- lazy_deps + pyproject [vertex] extra: google-auth, opt-in only.
- docs: guides/google-vertex.md + providers page; tests for adapter +
runtime resolution.
Salvages and modernizes #8427 by @slawt onto current main: rewired from
the legacy PROVIDER_REGISTRY path to the provider-profile architecture,
moved non-secret config out of .env into config.yaml, and added the
per-turn 401 token-refresh the original lacked.
- Track auth store source path on Nous state reads and write rotated
OAuth refresh tokens back to the same store, preventing stale-token
replays when Hermes falls back to a global/root auth.json.
- Skip Nous fallback entries locally when no access/refresh token is
present, suppressing repeated failed resolution attempts within a
session.
- Sync session model metadata after fallback switches so the gateway
DB reflects the backend that actually served the latest turn.
`pathlib.Path('~user').expanduser()` raises RuntimeError when the
tilde-expansion can't resolve the user (e.g. `~500-700` where the LLM
meant "approximately 500-700" rather than a path). The hint walker's
existing `except (OSError, ValueError):` clauses do not catch
RuntimeError, so it escapes through the tool dispatcher and surfaces
in the conversation loop as a misleading
Error during OpenAI-compatible API call #N:
Could not determine home directory.
Reproduced across three unrelated models (openai/gpt-5-mini,
openai/gpt-5.1-codex, deepseek/deepseek-v4-flash) on terminal-tool
commands containing literal tildes in non-path contexts — common in
LLM output ("~500 agencies", "~45,000 CVEs", "~80/hr blended rate").
Reproduction (one-liner):
>>> from pathlib import Path
>>> Path("~500-700").expanduser()
RuntimeError: Could not determine home directory.
Fix: extend the three `except` clauses in
agent/subdirectory_hints.py to also catch RuntimeError:
line 138 (_add_path_candidate's outer catch around the Path().expanduser() call)
lines 198+202 (_load_hints_for_directory's nested catches around hint_path.relative_to(Path.home()))
Tests: tests/agent/test_subdirectory_hints_tilde.py adds three cases
covering: tilde-as-approximately in heredoc commands, ~unknown_user paths,
and a regression guard that legitimate ~/path expansion still works.
Root cause: gateway spawns LSP servers (jdtls/pyright/yaml-ls) and
slash_worker without start_new_session=True, so they inherit the
gateway process group (= TUI parent PID). When mcp_tool
_snapshot_child_pids() races with these spawns during stdio MCP
server startup, non-MCP children leak into _stdio_pgids with the
TUI parent PGID. shutdown_mcp_servers() then killpg(tui_parent_pid,
SIGTERM), killing the TUI itself.
Evidence: tui_gateway_crash.log shows recurring SIGTERM stacks:
shutdown_mcp_servers -> _kill_orphaned_mcp_children ->
_send_signal -> killpg(pgid, sig) -> SIGTERM received
Fix (3 layers):
1. agent/lsp/client.py: add start_new_session=True to LSP server
spawn so each LSP server gets its own process group/session.
2. tui_gateway/server.py: same fix for slash_worker spawn, the
symmetric root-cause patch so no gateway direct child shares
the TUI parent pgid.
3. tools/mcp_tool.py: add _filter_mcp_children() defense-in-depth
that drops non-MCP children (slash_worker, jdtls/eclipse LSP)
from the PID delta before they can poison _stdio_pgids.
Follow-up correcting the salvaged fix's persistence approach to avoid a
duplicate user-message write (verified via E2E — the #860/#42039 bug class
the original diff aimed to avoid).
Root cause: in gateway mode the AIAgent is built WITH a session_db, so the
inbound user turn is already flushed at turn start (turn_context.
_persist_session). The original fix returned agent_persisted=False, making the
gateway re-write the whole new-message slice via append_to_transcript ->
append_message (a raw INSERT with no dedup), duplicating the already-flushed
user turn.
Corrected approach (single writer): run_codex_app_server_turn now flushes its
OWN projected assistant/tool messages via _flush_messages_to_session_db (which
dedups the already-persisted user turn through _DB_PERSISTED_MARKER) and
returns agent_persisted=True so the gateway skips its write. Net result:
session_search/distill see the full codex conversation, each message persisted
exactly once.
Adds regression coverage asserting exactly-once persistence on a real
SessionDB, agent_persisted=True, FTS visibility, and standard-runtime skip-db
behaviour preserved.
Co-authored-by: Lubos Buracinsky <lubos@komfi.health>
The codex_app_server runtime path (run_codex_app_server_turn in
agent/codex_runtime.py) is an early-return that bypasses
conversation_loop and never calls _flush_messages_to_session_db().
Meanwhile, gateway/run.py sets:
agent_persisted = self._session_db is not None # always True
and passes skip_db=agent_persisted to every append_to_transcript call,
assuming the agent self-persisted (correct for the standard runtime,
wrong for codex). The result: codex turn messages are persisted nowhere.
state.db accumulates only session_meta rows; session_search (full-text
search over state.db) and conversation-distill are blind to real gateway
conversations, causing 'the agent has no memory of what we discussed'.
Fix (three-part, all backward-compatible):
1. agent/codex_runtime.py — run_codex_app_server_turn success return
now includes 'agent_persisted': False, signalling that the codex path
did NOT self-persist its turn.
2. gateway/run.py — the agent_persisted assignment now reads:
agent_result.get('agent_persisted', self._session_db is not None)
For the standard runtime (which does not set the key) the default
(self._session_db is not None) preserves the existing skip-db
behaviour so no duplicate-write regression (#860 / #42039) occurs.
For the codex runtime the flag is False, so the gateway writes the
new turn's messages to state.db and FTS index.
3. gateway/run.py — the rebuilt result dict (run_agent return, which
becomes agent_result upstream) now includes agent_persisted passed
through from result_holder[0], with a safe True default. Without
this passthrough the flag set in step 1 was discarded when the result
was reconstructed, causing agent_result.get('agent_persisted', ...)
to always see the default True and never write codex turns.
Phase 2c review flagged that only 2 of the 4 structurally-identical
resolve_provider_client routing dead-ends were demoted. Complete the bug-class:
also demote+dedup the external-process ('not directly supported') and OAuth
('not directly supported, try auto') fall-throughs, keyed by provider name, so
none of the four dead-ends spam WARNING on a retry loop.
Add direct tests for the unhandled-auth_type and OAuth dedup paths via a
monkeypatched PROVIDER_REGISTRY (the review noted these were unverified).
Mutation-checked: reverting either sibling demotion fails its test.
The two fall-through branches in resolve_provider_client (unknown provider,
unhandled auth_type) logged at WARNING on every retry of a misconfigured
provider, spamming logs during retry loops. Demote both to logger.debug with
per-process dedup: the first occurrence still surfaces (a provider-name typo or
PROVIDER_REGISTRY/auth_type-drift bug is worth seeing once), while identical
repeats are suppressed for the process lifetime.
Salvaged from #56283 (extracting only the stated auxiliary_client fix; the
original PR also bundled ~2800 lines of unrelated changes across 10 other
files, which are dropped).
Think-enabled models (MiniMax M2.7, DeepSeek, etc.) emit inline
<think>...</think> reasoning even for simple prompts like title
generation, and the raw XML was leaking into session titles. Route the
title-model response through the canonical strip_think_blocks scrubber
before cleanup so every tag variant — closed pairs, unterminated blocks,
orphan closes, mixed case — is handled, not just a single literal
<think> pair.
- 2 regression tests: closed <think> pair stripped, unterminated block
at start yields no title.
Salvaged from PR #44126 by @shawchanshek.
MoA full-turn traces (moa.save_traces) recorded the aggregator's acting
output only on the non-streaming path, where it's captured inline at
call time. On the streaming path — which every hermes chat --query run
and every live gateway/CLI turn takes — the aggregator's raw token
stream is handed to the live consumer, so the trace left output=null and
only pointed at the session-db assistant row. An offline audit of a
benchmark run (HermesBench drives --query) then couldn't see what the
aggregator produced without hand-joining to state.db.
Capture the resolved streamed acting text at trace-flush time (the agent
already holds it in _current_streamed_assistant_text) and fold it into
the trace, so the record is self-contained in both modes. New
output_location value inline_from_stream marks a streamed turn whose text
was captured this way; a genuinely empty acting turn (pure tool call)
still points at the session db, matching state.db exactly.
Touches only the trace side-channel — no change to the acting path,
message history, role alternation, or prompt cache.
- agent/moa_loop.py: consume_and_save_trace(..., aggregator_output_fallback)
on both the facade and the MoAClient wrapper; prefer inline capture,
fall back to the resolved streamed text.
- agent/moa_trace.py: embed the fallback; add inline_from_stream location.
- agent/conversation_loop.py: pass _current_streamed_assistant_text at flush.
- tests: 5 cases across streaming / non-streaming / empty-fallback / no-double-write.
The forked skill/memory review agent shares the parent's session_id for
prompt-cache warmth. Without isolation it wrote its harness turn ('Review the
conversation above and update the skill library…') plus its curator-mode reply
straight into the user's REAL session in state.db; the next live turn re-read
that injected user message as a standing instruction and the agent 'became' the
curator, refusing the actual task.
Root fix: a _persist_disabled flag on the fork that hard-stops every DB write
and lazy-open path (_flush_messages_to_session_db, _ensure_db_session,
_get_session_db_for_recall) — the review writes only to the skill/memory stores
via its tools. Defense-in-depth: _strip_background_review_harness drops any
stray harness message (and the assistant reply that followed) at load time in
get_messages_as_conversation, so an already-polluted session resumes clean.
Salvaged from #50296.
Co-authored-by: arminanton <29869547+arminanton@users.noreply.github.com>
Local inference servers (llama.cpp/llama-server, vLLM/Ollama behind a
Cloudflare/Tailscale hop) report context overflow with HTTP 500/502/503/529
instead of 400/413. _classify_by_status returned server_error/overloaded and
retried blindly, then dropped the turn with no compaction. Route explicit
_CONTEXT_OVERFLOW_PATTERNS matches on those 5xx codes to context_overflow
(should_compress=True); plain 500 stays server_error, plain 503 overloaded.
Close a recovery/fallback final_response with an assistant transcript entry before session persistence so durable history cannot end at a tool/user message after the caller receives a final answer.
Adds a regression for a tool-tail transcript with a non-empty final_response. Related to #46071 / #46053, but covers the adjacent case where the assistant message was never appended before persistence.
When text compression can't reduce a 413 request further, evict base64
image parts from tool messages and retry once instead of dead-ending
with 'Payload too large and cannot compress further.'
A 413 is a request-body byte-size limit, not a token limit. browser_vision
screenshots (2-5MB base64 each) keep the HTTP body oversized even after
aggressive summarization. The strip pass passes remember_model=False so a
413 does not poison _no_list_tool_content_models — that set is for providers
that reject list-type tool content, a distinct failure mode.
Cherry-picked from #47397 by Tranquil-Flow; placed onto main's current
token-aware 413 recovery else branch.
vLLM (and other OpenAI-compatible servers) report context overflow with
both the window and the prompt in tokens:
"This model's maximum context length is 131072 tokens. However, you
requested 65536 output tokens and your prompt contains at least 65537
input tokens, for a total of at least 131073 tokens."
parse_available_output_tokens_from_error() already classified this as an
output-cap error (the "requested N output tokens" gate), but none of the
extraction patterns matched the "prompt contains [at least] N input
tokens" phrasing, so it returned None. The recovery path then
misclassified the failure as prompt-too-long and looped through
compression — which frees little while each retry keeps requesting the
same oversized max_tokens — terminating in "cannot compress further"
even though simply lowering the output cap would have succeeded.
Add an extraction branch for the token-based phrasing: available output
= window - reported input. When the input alone is at or over the
window it still returns None, so the caller correctly falls through to
compression.
Relates to #43547.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Anthropic's OAuth endpoints 404 for the claude-cli/ User-Agent prefix. Switch
all three OAuth UA sites (build_anthropic_client, refresh_anthropic_oauth_pure,
run_hermes_oauth_login_pure) to the claude-code/ prefix Anthropic expects.
Salvaged from #51948.
Co-authored-by: DhivinX <20087092+DhivinX@users.noreply.github.com>
_strip_orphaned_tool_blocks collected tool_result ids across ALL user messages
and kept any assistant tool_use whose id appeared anywhere, rather than
requiring the result to be in the immediately-following user message. A stale
match elsewhere in the transcript could keep a genuinely-orphaned tool_use,
which Anthropic rejects. Rewrite to adjacency-checked two-pass logic so a
tool_use is kept only when its result immediately follows.
Salvaged from #52145.
Co-authored-by: fsaad1984 <38867992+fsaad1984@users.noreply.github.com>
The original cross-session contamination fix (#38788) only cleared
_previous_summary in on_session_end(), but on_session_reset() clears
14+ per-session variables. When a session ends (cron exit, gateway
expiry, session-id rotation) and the compressor instance is reused,
the surviving stale state causes:
- _ineffective_compression_count surviving → next session skips
compression prematurely (anti-thrashing guard misfires)
- _summary_failure_cooldown_until surviving → next session blocks
summary generation for an unrelated transient error
- _last_compress_aborted surviving → callers think compression is
still aborted
- _last_aux_model_failure_* surviving → stale error warnings shown
- _last_summary_dropped_count / _last_summary_fallback_used
surviving → misleading user warnings
- _context_probed / _context_probe_persistable surviving → stale
context-probe state
Also fix on_session_reset() which was missing _last_compress_aborted
clearing — a /new or /reset would inherit the aborted flag from the
prior conversation.
Add 6 targeted tests covering the leak vectors and a parity test
ensuring on_session_end and on_session_reset always clear the same
surface.
The xapp-<num>-<hash> format used by Slack App-Level / Socket Mode
tokens was missing from both agent/redact.py prefix patterns and
gateway/run.py gateway secret patterns, so SLACK_APP_TOKEN values could
leak through to chat users even with security.redact_secrets enabled.
Adds an anchored xapp-\d+- pattern to both redaction paths.
The credential-pool Codex refresh path synced tokens from auth.json and
then POSTed the refresh_token to OpenAI's token endpoint without holding
the cross-process auth-store lock across the whole read->POST->write-back
sequence. Because Codex refresh tokens are single-use, two concurrent
Hermes processes could both adopt the same on-disk token and both POST
it; the loser got refresh_token_reused / invalid_grant.
Wrap the Codex OAuth branch of _refresh_entry in the existing shared
_auth_store_lock (reentrant, cross-process flock) using the same
extended-timeout pattern resolve_codex_runtime_credentials() already
uses. A waiting process now blocks on the lock and, once inside, the
in-lock re-sync picks up the rotated token the winner persisted and
skips its own POST. Also send User-Agent: hermes-cli/<version> on the
refresh request.
Credit @cooper-oai (#34820) for identifying the concurrent-refresh
reuse race; this ships the narrow lock-serialization fix without the
separate Codex auth-store partition.
_maybe_wrap_untrusted() only wrapped str-typed tool outputs. When a
high-risk tool (web_extract, browser_*) returns a multimodal content
list ([{type:text},{type:image_url}]) — which _tool_result_content_for
_active_model() produces by unwrapping the _multimodal envelope for
vision-capable providers — the text part reached the model completely
unguarded. An attacker page that ships one image bypassed the entire
untrusted-data wrapper.
Extend the wrapper to handle list content: each {type:text} part is run
through the same string-wrapping path (min-char threshold, delimiter
neutralization, one well-formed block), image/video parts pass through
untouched so the list stays valid for vision adapters. Recursing into
the existing string branch means the list path inherits the delimiter
defang and the no-forgeable-fast-path hardening from #56172 for free.
The outer list is rebuilt (not returned by identity), so callers compare
by value.
`@file` / `@folder` context-reference expansion enforced its own narrow
deny-list (`_ensure_reference_path_allowed` in `agent/context_references.py`)
that only covered `~/.ssh` keys, a handful of shell dotfiles, `~/.hermes/.env`,
and `skills/.hub`. It never blocked the credential stores that the canonical
read guard (`agent/file_safety.get_read_block_error`) protects: provider API
keys (`~/.hermes/auth.json`), Anthropic OAuth tokens
(`~/.hermes/.anthropic_oauth.json`), MCP OAuth material (`~/.hermes/mcp-tokens/`),
webhook HMAC secrets, and project-local `.env` files.
This matters because the messaging gateway feeds **untrusted** remote text
straight into reference expansion: `gateway/run.py` calls
`preprocess_context_references_async(..., allowed_root=_msg_cwd)` where
`_msg_cwd` defaults to the operator's HOME when `TERMINAL_CWD` is unset. A chat
peer (Telegram/Discord/Slack/...) could send `@file:~/.hermes/auth.json`, pass
the `allowed_root` check (it resolves under HOME), slip past the narrow list,
and have the operator's live keys read into the agent's context — where the
model would typically echo or act on them.
Rather than duplicate and re-sync a second secret list, this routes the guard
through the existing single source of truth. A reviewer might ask "why not just
add `auth.json` to the local list?" — because the local list has already drifted
once (a prior commit had to add `.config/gh`); anchoring to
`get_read_block_error` means every future addition there protects this path too.
The narrow checks are kept as a fallback since they also cover dirs that guard
does not (`.aws`, `.gnupg`, `.kube`, etc.), and the canonical lookup is wrapped
so it can never crash reference expansion.
N/A
- [x] 🔒 Security fix
- `agent/context_references.py`: `_ensure_reference_path_allowed` now also
consults `agent.file_safety.get_read_block_error` after its existing checks
and refuses the reference when that canonical guard flags the resolved path.
The lookup is wrapped so guard-resolution failures fall back to the explicit
checks instead of breaking expansion.
- `tests/agent/test_context_references.py`: added
`test_blocks_canonical_read_denylist_credential_stores`, asserting that
`@file` attaches for `auth.json`, `.anthropic_oauth.json`, `mcp-tokens/*`, and
a project-local `.env` are all refused and their secret bodies never reach the
expanded message.
- `scripts/release.py`: added the contributor email to `AUTHOR_MAP` (release
gate).
1. `scripts/run_tests.sh tests/agent/test_context_references.py` — all 15 tests
pass, including the new credential-store case.
2. Regression proof: stash `agent/context_references.py`, run the suite with
`-- -k canonical`, and confirm the new test fails (secrets leak into the
message) without the fix; restore and confirm it passes.
3. `ruff check agent/context_references.py tests/agent/test_context_references.py`
and `python scripts/check-windows-footguns.py agent/context_references.py
tests/agent/test_context_references.py` both pass.
- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits (`fix(scope):`, etc.)
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains **only** changes related to this fix (plus the AUTHOR_MAP release gate)
- [x] I've run the test suite for the touched area and all tests pass
- [x] I've added tests for my changes (required for bug fixes)
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)
- [x] I've updated relevant documentation (README, `docs/`, docstrings) — or N/A
- [x] I've updated `cli-config.yaml.example` if I added/changed config keys — or N/A
- [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — or N/A
- [x] I've considered cross-platform impact (Windows, macOS) — or N/A
- [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A
Review follow-up on the concurrent-tool deadline salvage. timed_out_indices is
snapshotted from not_done at the deadline; a worker can still finish and write
results[i] in the window before the post-execution result loop reads it. The
loop unconditionally replaced results[i] with a fabricated 'timed out' message
for any snapshotted index, discarding a genuinely-successful (just-late) result.
Gate the timeout message on 'and r is None' so a real result always wins. Add a
regression test that forces the snapshot-vs-result-loop race deterministically
(mutation-checked: reverting the guard fails it). Also document the intentional
detached-worker leak at the executor abandon site.
A tool with no internal interrupt check (read_file, web_search, or a wedged
terminal backend) that never returns keeps the concurrent-tool poll loop alive
forever: the loop only breaks when all futures finish or an interrupt is
requested, and the 30s heartbeat resets the gateway idle monitor so idle-kill
never fires. The ThreadPoolExecutor was also used as a context manager, so its
__exit__ joined the hung worker with wait=True.
Add a wall-clock batch deadline (HERMES_CONCURRENT_TOOL_TIMEOUT_S, default 420s
— above the 360s web_extract timeout; 0/negative disables). When it fires:
cancel pending futures, signal an interrupt to the worker threads, abandon the
executor (shutdown wait=False, cancel_futures=True) so hung threads aren't
joined, and return a per-tool 'timed out' result for the unfinished calls while
still surfacing the finished ones. Also fixes the latent futures.index(f)
lookup (ambiguous with duplicate futures) by tracking a future->index map.
Salvaged from #54562.
Co-authored-by: Gustavo Mendes <87918773+gustavosmendes@users.noreply.github.com>
verify_on_stop / pre_verify append a synthetic assistant "done" plus a
synthetic user nudge to keep the agent going one more turn before it can
claim completion. Both were flagged (_verification_stop_synthetic on the
nudge only), but the flags were never registered in
_EPHEMERAL_SCAFFOLDING_FLAGS, so the central _is_ephemeral_scaffolding()
filter that guards both persistence sinks (SQLite flush + JSON snapshot)
let them through. The resumed transcript then inherited loop-only
scaffolding, invalidating the prompt-prefix cache on later turns.
- add _verification_stop_synthetic and _pre_verify_synthetic to
_EPHEMERAL_SCAFFOLDING_FLAGS (the single chokepoint both sinks use)
- flag the blocked attempt assistant message too, not just the nudge, so
the whole synthetic pair drops together and persistence does not keep a
premature done with the nudge stripped (assistant to assistant adjacency)
The API-payload leak claimed in the report is already handled: the
chat_completions transport strips every underscore-prefixed message key
before the wire, so the marker never reaches strict providers.
Reported by patppham.
Widen the salvaged #32243 fix to the try_activate_fallback path: a custom
provider pointed at the native api.anthropic.com host (no /anthropic path
suffix, name != anthropic) fell through to chat_completions -> POST
/v1/chat/completions -> 404. Match the host the same way determine_api_mode()
and _detect_api_mode_for_url() now do. Absorbs #49247.
models_dev.py's fetch uses a synchronous requests.get(timeout=15). Called
from the async gateway message handlers, it blocked the event loop for up
to 15s, starving Discord heartbeats and causing ClientConnectionResetError
disconnects.
Adds get_model_context_length_async() which offloads the entire sync
resolution chain to a worker thread via asyncio.to_thread(), and switches
the two async gateway call sites (_prepare_inbound_message_text,
_handle_message_with_agent) to await it. The loop stays responsive; the
sync path remains the single source of truth for the cache.
Salvaged from PR #22753 by @itenev. Follow-up: dropped the unused
fetch_models_dev_async/lookup_models_dev_context_async aiohttp variants
from the original PR (dead code with zero callers that had drifted from
the sync cache logic) — the to_thread wrapper already runs the sync path
off-loop, so they were redundant.
knoal
Teknium
kshitij
spiky02plateau
xxxigm
Fan Yang
AIalliAI
Sanidhya Singh
sasquatch9818
Tranquil-Flow
Tanmay Choudhary
edgemass
Taylor H. Perkins
Ahmett101
pierrenode
Frowtek
Ray
falkoro
srojk34
HexLab98
annguyenNous
devatnull
dsad
wyuebei-cloud
luyifan
root
webtecnica
Jigoooo
kshitijk4poor
yoma
Shashwat Gokhe
Michael Steuer
huanshan5195
eliteworkstation94-ai
Lavya Tandel
Jacky Zeng
Jiahui-Gu
liuhao1024
nankingjing
infinitycrew39
Jaaneek
Jneeee
openhands
helix4u
Brooklyn Nicholson
snav
Gromykoss
Steve Lawton
HODLCLONE
ud
JabberELF
Lubos Buracinsky
Dutch Dim
shawchanshek
shandian64
arminanton
pefontana
WXBR
Tyler Merritt
DhivinX
fsaad1984
Ben Barclay
ArthurZhang
sprmn24
mrparker0980
Gustavo Mendes
itenev