Bremen
/
emma-hermes
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
emma-hermes/skills/software-development/ego-browser-smoke-regression/SKILL.md

5.2 KiB
Raw Blame History

name description version author license platforms metadata
ego-browser-smoke-regression Use when tightening or verifying Hermes ego-browser routing, smoke scripts, and screenshot/form browser regressions. 1.0.0 Hermes Agent MIT
macos
linux
windows
hermes
tags related_skills
ego-browser
browser
smoke-test
regression
routing
screenshot
form
test-driven-development
systematic-debugging
spike

Ego-browser smoke regression

Overview

Use this skill when browser-task routing changed and you need a fast, repeatable way to prove Hermes still prefers ego-browser for live web work — especially screenshot and form tasks.

This workflow combines three layers:

  1. Prompt / routing regression tests — verify the instructions still bias toward ego-browser.
  2. Local smoke scripts — run real browser work against a tiny controlled form page and a stable public page (www.google.com).
  3. Opt-in CI smoke — wire the checks into a manual workflow instead of forcing every PR to pay for environment-sensitive browser runs.

When to Use

Use this skill when:

  • a PR changes browser routing, prompt text, screenshot fallback rules, or related skills
  • a user reports Hermes fell back to generic screenshot handling too early
  • form filling, page screenshots, or browser-native verification became flaky
  • you need a reproducible proof path before or after fixing an ego-browser regression

Do not use this skill for:

  • plain static web extraction that never needs a browser
  • desktop / OS screenshot work unrelated to webpage rendering
  • broad UI QA where dogfood is the better higher-level workflow

Repo assets

Current repo entry points:

  • tests/agent/test_prompt_builder.py
  • scripts/ego_browser_form_smoke.sh
  • scripts/ego_browser_screenshot_smoke.sh
  • tests/tools/test_ego_browser_smoke_scripts.py
  • tests/tools/test_ego_browser_smoke_utils.py
  • .github/workflows/ego-browser-smoke.yml
  • scripts/README.md

Procedure

1. Verify prompt-layer routing first

Run the focused prompt tests before touching smoke scripts:

python -m pytest tests/agent/test_prompt_builder.py -q

Completion criterion:

  • the ego-browser routing assertions pass
  • no new prompt regression appears

2. Run the form smoke on a controlled local page

Use the local form smoke because it is fast and deterministic:

./scripts/ego_browser_form_smoke.sh SmokeValue42

What success looks like:

  • JSON output with the local loopback URL
  • the returned value matches the injected text
  • status echoes value:<same text>

3. Run the screenshot smoke against a stable page

Use the screenshot smoke to validate the real fallback path while keeping the task centered on ego-browser:

./scripts/ego_browser_screenshot_smoke.sh

What success looks like:

  • used_ego_browser_skill: true
  • loaded_generic_screenshot_skill: false
  • a non-empty screenshot_path
  • ocr_text contains Google

4. Use pytest wrappers for repeatable opt-in runs

Form-only smoke via pytest:

RUN_EGO_BROWSER_SMOKE=1 python -m pytest tests/tools/test_ego_browser_smoke_scripts.py -q

Full screenshot smoke via pytest (explicit opt-in because it shells into a real hermes chat run):

RUN_EGO_BROWSER_SMOKE=1 RUN_EGO_BROWSER_SCREENSHOT_SMOKE=1 \
  python -m pytest tests/tools/test_ego_browser_smoke_scripts.py -q

Utility tests for OCR/parsing helpers:

python -m pytest tests/tools/test_ego_browser_smoke_utils.py -q

5. Keep CI manual unless the environment is guaranteed

The screenshot path depends on a real ego-browser installation plus tesseract. Do not bolt it onto the default PR lane. Use the manual workflow:

  • .github/workflows/ego-browser-smoke.yml
  • trigger via workflow_dispatch
  • run on a self-hosted macOS runner with ego-browser already onboarded

Completion criterion:

  • workflow file is valid YAML
  • prerequisites are checked before the expensive steps
  • screenshot smoke stays opt-in

Common Pitfalls

  1. Testing only the helper, not the routing. A direct helper call can pass while Hermes still routes real screenshot work incorrectly.
  2. Treating webpage screenshots as desktop screenshots. If the target is browser rendering, keep the task in the browser workflow first.
  3. Forgetting OCR verification. A screenshot file existing is not enough; confirm the expected page text is visible.
  4. Making screenshot smoke mandatory in default CI. That creates noise and false reds when the runner lacks ego-browser onboarding.
  5. Skipping retry / fallback behavior in the screenshot script. Real runs are environment-sensitive; keep the script resilient but strict about the success criteria.

Verification Checklist

  • tests/agent/test_prompt_builder.py passes
  • tests/tools/test_ego_browser_smoke_utils.py passes
  • ./scripts/ego_browser_form_smoke.sh returns the injected value and status
  • ./scripts/ego_browser_screenshot_smoke.sh returns used_ego_browser_skill: true
  • screenshot JSON includes ocr_text with Google
  • the opt-in pytest wrapper stays green for form smoke
  • the manual CI workflow remains opt-in and prerequisite-gated