emma-hermes/agent
teknium1 369075dc95 feat(tools): progressive tool disclosure for MCP and plugin tools
Adds Tool Search, a structured-tools progressive-disclosure layer that
replaces MCP and non-core plugin tools in the model-visible tools array
with three bridge tools (tool_search / tool_describe / tool_call) when
the deferrable surface would consume more than a configurable percentage
of the active model's context window. Core Hermes tools are never deferred.

Default mode is 'auto' with a 10% context threshold, so small toolsets
pay no overhead. Set tools.tool_search.enabled to 'on' to force or 'off'
to disable.

Design carefully reflects the OpenClaw production failure modes
documented in the openclaw-tool-search-report:

  - Core tools never defer (toolsets._HERMES_CORE_TOOLS). Addresses the
    'tools silently missing from isolated cron turns' regression class
    (openclaw#84141) by construction: there is no code path that can
    drop a core tool.
  - Catalog is stateless across turns — rebuilt from the live tool-defs
    list on every assembly. No session-keyed Map that can drift out of
    sync with the registry.
  - tool_call unwraps the bridge call before any hook fires, so plugin
    pre/post hooks, guardrails, approval flows, and the activity feed
    all see the underlying tool name, not the bridge (addresses
    openclaw#85588 and the verbose-mode complaint on openclaw#79823).
  - The unwrap happens in both the parallel and sequential paths of
    agent/tool_executor.py and also in handle_function_call, so direct
    callers (sandboxed code, eval harnesses) are covered too.
  - Bridge tools cannot invoke each other (recursion guard) and cannot
    invoke core tools (those must be called directly).
  - Tools mode only — no JS-sandbox code-mode. Keeps the surface small.
  - Token estimation via cheap char/4 heuristic; precision isn't needed
    for the threshold decision.

Files:
  - tools/tool_search.py — new module (BM25 retrieval, classification,
    threshold gate, bridge dispatch, unwrap helper).
  - tests/tools/test_tool_search.py — 35 tests including the OpenClaw
    #84141 regression guard.
  - model_tools.py — wires assembly into _compute_tool_definitions as the
    final step, adds skip_tool_search_assembly kwarg so the bridge can
    see the real catalog, dispatches the three bridge tools.
  - agent/tool_executor.py — unwraps tool_call in both parallel and
    sequential parsing loops so checkpointing, guardrails, plugin hooks,
    and tool-progress callbacks all observe the underlying tool name.
  - hermes_cli/config.py — DEFAULT_CONFIG['tools']['tool_search'] block.
  - website/docs/user-guide/features/tool-search.md — user docs.

Validation:
  - 35/35 new tests pass.
  - Existing tool/registry/model_tools/config/coercion/executor tests
    (82 + 74 + small adjacents) green.
  - Live E2E: 20 fake MCP tools registered, get_tool_definitions returns
    3 bridges, tool_search returns top 3 hits, tool_describe returns
    full schema, tool_call dispatches to the real underlying handler
    and the underlying result is what the model sees.
  - Reserved-name recursion guard verified live.
  - Core-tool refusal via tool_call verified live.
2026-05-29 02:04:12 -07:00
..
lsp chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
secret_sources chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
transports chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
__init__.py fix(agent): preload jiter native parser 2026-05-28 00:20:11 -07:00
account_usage.py
agent_init.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
agent_runtime_helpers.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
anthropic_adapter.py fix(auth): don't launch a text-mode browser inside the terminal for OAuth (#34479) 2026-05-29 01:23:06 -07:00
async_utils.py
auxiliary_client.py fix(auxiliary): detect xAI OAuth 403 bad-credentials as auth error 2026-05-29 00:28:02 -07:00
azure_identity_adapter.py
background_review.py test: cover ci-unblocker production regressions 2026-05-27 22:14:53 -07:00
bedrock_adapter.py
browser_provider.py
browser_registry.py
chat_completion_helpers.py fix(codex): relax no-byte TTFB watchdog default from 12s to 120s 2026-05-29 02:02:25 -07:00
codex_responses_adapter.py feat(prompt): universal task-completion guidance + local Python toolchain probe (#34340) 2026-05-28 22:26:09 -07:00
codex_runtime.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
context_compressor.py test: cover fallback dropped-turn handoff 2026-05-28 20:34:40 -07:00
context_engine.py feat(context-engine): host contract for external context engines 2026-05-28 01:45:30 -07:00
context_references.py
conversation_compression.py fix(compression): fail open when lock subsystem is missing (version skew) (#34475) 2026-05-29 01:32:32 -07:00
conversation_loop.py fix(kanban): close three blocked/iteration-exhausted handling gaps (#29747) 2026-05-29 00:13:29 -07:00
copilot_acp_client.py
credential_persistence.py
credential_pool.py fix(credential-pool): STATUS_DEAD for terminal OAuth failures (#32849) (#34412) 2026-05-28 23:45:42 -07:00
credential_sources.py
curator.py fix: preserve skill packages during curator consolidation 2026-05-27 13:39:58 -07:00
curator_backup.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
display.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00
error_classifier.py fix(agent): fallback immediately on provider content-policy blocks (#33883) 2026-05-28 07:28:24 -07:00
file_safety.py fix(security): add bws_cache.json to file_safety read guard 2026-05-28 23:31:20 -07:00
gemini_cloudcode_adapter.py
gemini_native_adapter.py
gemini_schema.py
google_code_assist.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
google_oauth.py fix(auth): don't launch a text-mode browser inside the terminal for OAuth (#34479) 2026-05-29 01:23:06 -07:00
i18n.py
image_gen_provider.py
image_gen_registry.py
image_routing.py feat(kanban): attach images referenced in task bodies to worker vision (#34210) 2026-05-28 17:50:42 -07:00
insights.py
iteration_budget.py
jiter_preload.py fix(agent): preload jiter native parser 2026-05-28 00:20:11 -07:00
lmstudio_reasoning.py
manual_compression_feedback.py
markdown_tables.py
memory_manager.py feat: expose completed-turn message context to memory providers 2026-05-29 02:16:43 +05:30
memory_provider.py feat: expose completed-turn message context to memory providers 2026-05-29 02:16:43 +05:30
message_sanitization.py
model_metadata.py fix: stop probe stepdown without provider context limit 2026-05-28 12:26:53 -07:00
models_dev.py remove Vercel AI Gateway and Vercel Sandbox (#33067) 2026-05-27 00:43:32 -07:00
moonshot_schema.py
nous_rate_guard.py
onboarding.py
plugin_llm.py
portal_tags.py
process_bootstrap.py
prompt_builder.py fix(kanban): tell workers not to use clarify; route to kanban_block instead (#32167) 2026-05-28 23:57:20 -07:00
prompt_caching.py
rate_limit_tracker.py
redact.py fix(redact): pass web URLs through unchanged (#34029) 2026-05-28 11:32:39 -07:00
retry_utils.py
shell_hooks.py
skill_bundles.py
skill_commands.py
skill_preprocessing.py
skill_utils.py
stream_diag.py feat(agent): buffer retry/fallback status, surface only on terminal failure (#33816) 2026-05-28 04:53:27 -07:00
subdirectory_hints.py
system_prompt.py feat(prompt): universal task-completion guidance + local Python toolchain probe (#34340) 2026-05-28 22:26:09 -07:00
think_scrubber.py
title_generator.py
tool_dispatch_helpers.py
tool_executor.py feat(tools): progressive tool disclosure for MCP and plugin tools 2026-05-29 02:04:12 -07:00
tool_guardrails.py
tool_result_classification.py
trajectory.py
transcription_provider.py
transcription_registry.py
tts_provider.py
tts_registry.py
usage_pricing.py feat: add claude-opus-4.8 and claude-opus-4.8-fast (#34003) 2026-05-28 10:31:59 -07:00
video_gen_provider.py
video_gen_registry.py
web_search_provider.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00
web_search_registry.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00