Initial server source import

2026-09-30 20:30:56 +03:00
commit 170dd941b9
498 changed files with 261563 additions and 0 deletions
@@ -0,0 +1,105 @@
+# Extended Live Validation
+
+Date: 2026-09-26
+Run: `e6ac8aec-ec20-4ba4-a924-abe5ee95d82c`
+Conclusion: `pass_with_documented_deviations`
+
+## Scope
+
+The production validation ran one native Windows worker slot and one WSL/Docker
+worker slot through discovery, assignment, execution, progress, diagnostics,
+bundle ingestion, projection, capacity release, and scheduled keycheck. The run
+started at `2026-09-26T00:29:30.500768Z`; terminal server evidence was captured
+at `2026-09-26T03:54:04.573328Z`.
+
+This report contains derived counts, classifications, and cryptographic
+identities only. Raw provider values and worker payloads remain in private
+evidence storage.
+
+## Outcome
+
+- 314 reservations reached terminal outcomes: 309 acknowledged and 5 refunded.
+- All 309 accepted bundles were ingested in one attempt, projected, settled, and
+  released; no unresolved reservation or duplicate receipt remained.
+- The accepted cohort covered DockerHub (105), GitLab (102), and Hugging Face
+  (107), split across Windows (167) and WSL (147).
+- 3,042 progress events covered all 309 accepted reservations with no duplicate
+  reservation sequence.
+- All 348 projection jobs completed and released in one attempt with no error.
+- The sealed terminal cut at controller revision 143 had no live assignment,
+  pre-commit bundle, capacity use, publication outbox item, quarantine row,
+  waiting lock, or blocker.
+
+The 309 scan outcomes were 174 clean, 75 degraded, 56 error, and 4 found. Five
+findings were recorded and none were verified. The run recorded 100 classified
+scan errors, led by 71 download failures; source distribution was GitLab 92,
+DockerHub 7, and Hugging Face 1.
+
+## Diagnostics And Keycheck
+
+The validation captured 61 worker diagnostics: 52 result errors, 4 stage
+timeouts, 4 runner protocol failures, and 1 client process failure. Fifty-one
+were retryable and ten were nonretryable. The records remain available in the
+private raw bundle for exact-body investigation.
+
+The captured keycheck cohort contained 39 candidates. All completed in one
+attempt, released capacity, linked to a result, and projected. Results were 35
+invalid or revoked and 4 no-balance; 37 came from API execution and 2 from
+cache. At the terminal cut the global keycheck queue had 184 completed and no
+pending, leased, or deferred candidate.
+
+Nine pending DockerHub discovery retry rows with zero attempts remained as
+expected durable discovery backlog, not as leaked worker-pipeline work.
+
+## Evidence Integrity
+
+- Server chain: 202 valid records, sequence `-1..199`, with all 575 referenced
+  objects (1,245,586,992 bytes) verified.
+- Server NDJSON SHA-256:
+  `a6beb1864c540fc5f22b2b647730a39e45dfddb10cf5ceff5e0181eb1a8f0bd8`.
+- Server run SHA-256:
+  `5ae58df5f67a8d2a8d4e84d73262a6e7009e03dcbcc9ea176537b384d4e693c3`.
+- Worker chain: 158 valid records, sequence `0..157`, with all 2,136 referenced
+  objects (7,733,582,955 bytes) verified.
+- Worker NDJSON SHA-256:
+  `03a04a0da5a2db6bd02f2aaed41c191554b135ec287fbc1e2d9998d77da59898`.
+- Worker final payload SHA-256:
+  `f652e5f98fb23f9aa2fb4e44e10168a931ea973be616fb09dbdb72001ce961ad`.
+
+The machine-readable verification manifest is at
+`build/extended-live-validation/runs/e6ac8aec-ec20-4ba4-a924-abe5ee95d82c/verification-manifest.json`.
+The complete server evidence root is retained in private production storage;
+the local run root contains compact server artifacts and complete worker
+evidence.
+
+## Observed Deviations
+
+The authenticated `recheck all` operation was accidentally used where only two
+pending candidates should have been rechecked. This caused 35 broad provider
+checks contrary to `KEYCHECK-001`. The command processed all 35, skipped none,
+returned success, produced no failed operation, and all resulting effects
+settled. This is an operator-scope deviation, not an approved workflow change.
+
+Six monitor samples encountered transient database statement timeouts. Every
+monitor error recovered, and the evidence chains remained valid. The runtime
+and edge each recorded zero restart during the validation.
+
+## Remaining Defects
+
+- Windows GitLab filename-too-long checkout recovery exists locally but is not
+  deployed.
+- Invalid API-key classification has a local fix that is not deployed.
+- A roughly 20-second WSL clock-domain monotonic failure remains open.
+- Generic `WorkerContractError` diagnostics can lose structured field detail.
+- Monitor aggregate queries can exceed their statement timeout under load.
+
+Focused regression coverage for Git checkout recovery, Hugging Face long paths,
+and direct remote credentials passed: 145 tests in 155.26 seconds.
+
+## Restore
+
+Production was restored with compare-and-swap transitions from revision 143 to
+146: cancel the validation drain, resume discovery, then resume
+dispatch. Final state was dispatch open, discovery open, drain normal, healthy
+core services and producers, and exactly one live assignment on each worker.
+No further validation probe is required.