AI/Tasks/OrchestratorUpgrades/Orchestrator_Phase05_WeatherRoutingHardening_Strict.txt
|
STRICT MODE: Harden weather and web-specific routing logic.
Mission: - Complete this phase with bounded steps and deterministic fallback behavior. - Apply changes ONLY within weather/web routing and directly related provenance validation. Concrete scope: - AI/Tasks/OrchestratorUpgrades/Overview.txt - AI/Tasks/OrchestratorUpgrades/Orchestrator_Phase05_WeatherRoutingHardening_Strict.txt - NOAA ZIP discovery, search-result interpretation, candidate provenance rules, and execute-before-clarify controls for weather-specific tool flows. Requirements: - Strengthen NOAA and ZIP candidate extraction with provenance validation. - Score multiple ZIP candidates against the requested city/state or geography before execution. - Reject weak or unrelated five-digit-only values from noisy search content. - Require explicit confidence or evidence before a forecast request is made. - Add a clear fallback path: ask for the ZIP or force clarification when confidence is insufficient. - Define the candidate data structure, including ZIP, city, state, source URL or source identifier, evidence span, extraction method, confidence, and timestamp. - Define a deterministic scoring formula or ordered rules for candidate selection rather than leaving scoring open-ended. - Require minimum confidence and margin-over-second-place thresholds before a weather request is executed. - Preserve general tool safety, authorization, and validation boundaries. - Keep the model deterministic, centralized, and mechanically testable. - Add characterization or phase-specific tests before or alongside each major change. - Do not expand into broad memory-privacy or telemetry redesign beyond what is required for weather-routing hardening. - Enforce sensitive-data redaction and resource-limit checks in any diagnostics or logging introduced here. - Do not weaken authorization, confirmation, sandboxing, or execution safety requirements. - Define compatibility and rollback behavior when weather-routing confidence thresholds or candidate-scoring logic change. Step budget (hard limits): 1) Discovery: <= 6 read/search tool calls. 2) Edits: <= 8 file edits total. 3) Validation: 1 build + 1 targeted test run; allow exactly 1 repair cycle if failures are phase-related. 4) Stop once acceptance criteria pass. Loop guards: - If weather-routing changes start to bypass general tool policy or validation, stop and re-establish the owning abstraction. - If the refactor expands beyond ZIP provenance and NOAA route safety, stop and keep the phase boundary explicit. Fallback behavior: - Fallback A: add candidate scoring and provenance checks in the routing layer before changing execution logic. - Fallback B: keep the current domain flow and wrap it with stricter candidate selection and clarification gating. - Fallback C: after one failed repair cycle, emit FAILED_VALIDATION with the exact failing assertions. Acceptance criteria: - ZIP and location discovery require provenance and confidence checks. - Untrusted or noisy five-digit values are rejected unless evidence supports them. - The route asks for clarification or blocks execution when confidence is low. - Characterization or phase-specific tests cover ZIP candidate scoring and route safety. - Logging and diagnostics redact sensitive data and respect resource limits. - The orchestration layer does not weaken authorization, confirmation, sandboxing, or execution safety requirements. Validation commands: - dotnet build src/TechToolbox.Agent/TechToolbox.Agent.csproj -c Release - dotnet test src/TechToolbox.Agent/Tests/TechToolbox.Agent.Tests.csproj -c Release --filter "Orchestrator|Weather|NOAA|ZIP|Search|Routing|Strict" Output results in markdown using this structure: 1) Summary 2) Files changed 3) Weather-routing contract 4) Tests added/updated 5) Validation results 6) Step-budget report (planned vs actual counts) |