AI/Tasks/OrchestratorUpgrades/Orchestrator_Phase04_ValidationRepairLoopProgress_Strict.txt
|
STRICT MODE: Harden decision validation, repair budgets, and loop-progress tracking.
Mission: - Complete this phase with bounded steps and deterministic fallback behavior. - Apply changes ONLY within decision validation, repair logic, loop guards, and directly related orchestrator diagnostics. Concrete scope: - AI/Tasks/OrchestratorUpgrades/Overview.txt - AI/Tasks/OrchestratorUpgrades/Orchestrator_Phase04_ValidationRepairLoopProgress_Strict.txt - Decision schema validation, repair flow, guard history tracking, no-progress budgets, and structured diagnostic output. Requirements: - Replace permissive parsing with a versioned decision schema and structured validation errors. - Add a global decision-repair budget and preserve invalid original decisions for diagnostics. - Add bounded streaming validation for long final answers and tool decisions. - Replace naive repeated-call detection with recent-history tracking and progress signatures. - Distinguish same-call, same-target, and same-evidence detection policy as separate guard behaviors. - Publish the decision schema, including required fields, allowed operations, mutual exclusions, numeric ranges, maximum lengths, and unknown-field behavior. - Define whether schema validation occurs before semantic validation and policy validation, and what action is taken for each validation failure: block, clarify, fail, or repair. - Preserve current behavior where the task is only to make validation and loop guards deterministic and bounded. - Keep the model deterministic, centralized, and mechanically testable. - Add characterization or phase-specific tests before or alongside each major change. - Do not expand into weather-specific routing, memory privacy, or telemetry redaction beyond what is required for validation/loop guard hardening. - Enforce sensitive-data redaction and resource-limit checks in any diagnostics or logging introduced here. - Do not weaken authorization, confirmation, sandboxing, or execution safety requirements. - Define compatibility and rollback behavior for decision-schema or guard-policy changes that affect validation outcomes. Step budget (hard limits): 1) Discovery: <= 6 read/search tool calls. 2) Edits: <= 8 file edits total. 3) Validation: 1 build + 1 targeted test run; allow exactly 1 repair cycle if failures are phase-related. 4) Stop once acceptance criteria pass. Loop guards: - If validation ownership starts to overlap with tool policy or execution semantics, stop and re-establish the owning abstraction. - If repeated guard refactoring becomes a broader rewrite than validation and loop tracking, stop and keep the phase boundary explicit. Fallback behavior: - Fallback A: introduce structured validation errors and a repair budget before changing guard behavior. - Fallback B: preserve current validation behavior and add explicit guard wrappers around it. - Fallback C: after one failed repair cycle, emit FAILED_VALIDATION with the exact failing assertions. Acceptance criteria: - Decision validation is versioned, structured, and bounded. - A global repair budget is enforced. - Loop-progress detection uses bounded history and explicit progress signatures. - Guard behaviors are distinct for same-call, same-target, and same-evidence conditions. - Characterization or phase-specific tests cover malformed decisions, repair exhaustion, and loop detection. - Logging and diagnostics redact sensitive data and respect resource limits. - The orchestration layer does not weaken authorization, confirmation, sandboxing, or execution safety requirements. Validation commands: - dotnet build src/TechToolbox.Agent/TechToolbox.Agent.csproj -c Release - dotnet test src/TechToolbox.Agent/Tests/TechToolbox.Agent.Tests.csproj -c Release --filter "Orchestrator|Decision|Validation|Repair|Loop|Guard|Strict" Output results in markdown using this structure: 1) Summary 2) Files changed 3) Validation and loop-progress contract 4) Tests added/updated 5) Validation results 6) Step-budget report (planned vs actual counts) |