emma-hermes/docs/profile-isolation/checklist.md

Profile Isolation Acceptance Checklist

Goal

Verify that global system rules, principal-specific profiles, and session-only context do not leak into each other.

Must-pass checks

Identity / principal resolution

• Every gateway request resolves to a canonical principal when possible
• Ambiguous identity fails closed
• A missing verified identity does not fall back to a random operator profile
• Email is treated as an attribute of the principal, not the primary key

System profile protection

• Only local admin workflows can modify the system profile
• Telegram / LINE / email / OpenWebUI users cannot edit the system profile
• System profile changes are audit-logged
• System profile remains stable across user sessions

Principal profile isolation

• Each canonical principal has its own profile bucket
• One principal cannot read or write another principal's profile
• The resolved principal always selects the matching profile
• Profile writes are restricted to allowed fields

Session context safety

• Session-only facts are not persisted unless explicitly promoted
• Temporary task notes do not end up in the durable profile
• A failed or ambiguous identity does not create a permanent profile entry

Shared knowledge layer

• Shared knowledge is clearly global and non-private
• Shared knowledge is sanitized before injection
• Shared knowledge does not absorb one user's personal facts
• Shared knowledge lives outside principal homes

Prompt assembly order

• System rules load before principal personalization
• Principal personalization loads before session context
• Lower-priority layers cannot override higher-priority rules
• Shared knowledge does not replace the system profile

Conversation isolation

• A cannot see B's conversation summary
• B cannot see A's conversation summary
• A cannot ask for B's memory or profile notes
• B cannot ask for A's memory or profile notes
• Existence leaks are rejected just like full content leaks

API-server session and response-chain isolation

• api_server session CRUD routes operate inside the resolved principal profile only
• Session message history resolves from the same principal-scoped state.db
• previous_response_id resolves from the active principal-scoped response_store.db only
• Conversation-name chaining does not cross principal boundaries
• GET /v1/responses/{id} and DELETE /v1/responses/{id} cannot touch another principal's response store

Managed download isolation

• Owner-bound managed downloads reject callers outside the matching request scope
• Public managed downloads strip owner_metadata instead of silently reusing owner-bound scope
• Public and private download modes preserve the same expiry semantics
• Callers can be prompted or guided to choose private vs public link mode when a managed download is generated
• Streaming terminal payloads rewrite managed-download URLs consistently with non-streaming payloads
• Any remaining token-level streaming leakage risk is either blocked or explicitly tracked for follow-up

launchd rollout / restart safety

• If a LaunchAgent plist changed, reload the job definition with launchctl bootout ...plist + launchctl bootstrap ...plist instead of assuming kickstart -k will apply the new arguments
• After gateway plist changes, launchctl print gui/$(id -u)/ai.hermes.gateway shows live arguments that exactly match the edited plist (gateway run without stale --replace)
• After runs-proxy plist or script changes, launchctl print gui/$(id -u)/ai.hermes.openwebui-runs-proxy shows the expected program arguments and a fresh PID
• Gateway health recovery is given enough wait time after reload (real run observed: healthy on try 13 with a 1s poll loop)
• OpenWebUI runs proxy health recovery is given enough wait time after reload (real run observed: healthy on try 2 with a 1s poll loop)
• 8642, 8653, and 8646 are all re-checked after reload before declaring success

OpenWebUI / gateway / email

• OpenWebUI resolves the live caller, not a cached operator default
• Telegram and LINE use the bound principal, not chat labels alone
• Email handoff re-checks the live recipient principal
• Shared surfaces fail closed when caller identity is ambiguous

Regression tests to run

• tests/agent/test_system_prompt_principal_precedence.py
• tests/agent/test_system_prompt_shared_knowledge.py
• tests/gateway/test_principal_profile_routing_phase1.py
• tests/gateway/test_principal_profile_runtime_phase2.py
• tests/gateway/test_api_server_session_ownership.py
• tests/hermes_cli/test_web_server_session_access.py
• tests/gateway/test_verified_email_handoff_guard_helpers.py
• tests/gateway/test_api_server.py -k 'ResponsesEndpoint or ResponsesStreaming or Truncation or ConversationParameter'
• tests/gateway/test_multiplex_api_server_routing.py
• tests/hermes_cli/test_managed_downloads.py

Done means

• The docs in this folder describe the current behavior accurately
• implementation-status.md reflects the latest landed isolation phases
• Code comments point back here and to technical-design.md
• No code path can silently merge private data into the shared system profile