Initial server source import

This commit is contained in:
sashatrask
2026-09-30 20:30:56 +03:00
commit 170dd941b9
498 changed files with 261563 additions and 0 deletions
@@ -0,0 +1,102 @@
## ADDED Requirements
### Requirement: Exact distributed core source set
The server core profile SHALL contain exactly `gitlab`, `dockerhub`, and `huggingface`, and SHALL NOT start GitHub or any other discovery source as part of that profile.
#### Scenario: Core profile starts
- **WHEN** Supervisor starts the distributed core profile
- **THEN** it starts one discovery producer for GitLab, DockerHub, and HuggingFace and no GitHub producer
### Requirement: Discovery-only producer isolation
Each core discovery producer SHALL search its provider, normalize targets, and admit them to PostgreSQL without claiming targets, acquiring scan slots, invoking a scanner, or staging result bundles locally.
#### Scenario: GitLab discovery finds targets
- **WHEN** the GitLab producer completes a provider search
- **THEN** it enqueues the normalized targets and records zero local scan requests
#### Scenario: DockerHub discovery resolves targets
- **WHEN** the DockerHub producer processes pages, retries, tags, or digests
- **THEN** it may persist discovery progress and immutable targets but never invokes TruffleHog or reserves those targets locally
#### Scenario: HuggingFace discovery finds Spaces
- **WHEN** the HuggingFace producer returns Space identifiers
- **THEN** it drops records explicitly marked private, protected, gated, or disabled and enqueues the remaining identifiers without entering a local scan path or making a second per-Space verification request
#### Scenario: Pending backlog exists
- **WHEN** a scheduled discovery interval arrives while pending targets already exist
- **THEN** the producer still performs the configured discovery cycle unless discovery is paused
### Requirement: Durable operations control state
The system SHALL persist discovery pause, dispatch pause, drain state, revision, actor, operation identity, and update timestamps in PostgreSQL so that control state survives process and host restarts.
#### Scenario: Runtime restarts while paused
- **WHEN** discovery and dispatch are paused and the runtime restarts
- **THEN** both effective gates remain paused after startup
#### Scenario: Stale control form is submitted
- **WHEN** a mutation supplies a revision older than the current control revision
- **THEN** the system rejects it without changing control state or writing a success audit event
#### Scenario: Explicit pause coexists with drain
- **WHEN** an operator explicitly pauses discovery, enters drain, and later cancels drain
- **THEN** the explicit discovery pause remains set
### Requirement: Transactional discovery admission gate
The system SHALL enforce the effective discovery gate in the same transaction that admits discovered targets or claims discovery-specific retry work.
#### Scenario: Pause races target admission
- **WHEN** discovery pause commits before a producer admission transaction commits
- **THEN** no newly discovered target is admitted by that transaction
#### Scenario: Discovery is paused before provider request
- **WHEN** a producer begins a cycle while discovery is effectively paused
- **THEN** it performs no provider request and records a paused cycle outcome
#### Scenario: Upload-derived work arrives during pause
- **WHEN** result ingestion creates projection or keycheck work while discovery is paused
- **THEN** that work remains admissible because it is not provider discovery
### Requirement: Transactional dispatch gate
The system SHALL enforce the effective dispatch gate within the reservation transaction so that no new remote assignment can be issued after dispatch pause or drain commits.
#### Scenario: Dispatch pause races claim
- **WHEN** dispatch pause commits before a worker claim transaction commits
- **THEN** the claim returns a paused or no-work response and creates no reservation
#### Scenario: Existing worker uploads while paused
- **WHEN** dispatch is paused and a worker with an existing assignment reports status or uploads its result
- **THEN** the server accepts the valid request under the existing assignment authority
#### Scenario: Assignment expires while paused
- **WHEN** an existing assignment expires during dispatch pause
- **THEN** the reaper processes it normally without issuing replacement work
### Requirement: Drain lifecycle
Entering drain SHALL effectively pause discovery and dispatch while preserving status, terminal report, upload, receipt replay, ingestion, projection, and maintenance paths needed to finish accepted work.
#### Scenario: Drain begins with active assignments
- **WHEN** drain is requested while remote assignments are active
- **THEN** the state becomes `draining`, no new targets or assignments are admitted, and existing workers retain their result path
#### Scenario: Drain reaches completion
- **WHEN** no live remote assignments remain and every accepted result bundle has reached database commit
- **THEN** the reconciler advances the state to `drained`
#### Scenario: Pending targets remain
- **WHEN** pending queue targets remain but all issued assignments and pre-commit bundles are resolved
- **THEN** drain may still become `drained`
#### Scenario: Projection work remains
- **WHEN** projection or keycheck work remains after its result bundle is database-committed
- **THEN** that work does not prevent the control state from becoming `drained`
### Requirement: Discovery process observability
Supervisor and the operations console SHALL expose each discovery producer's source, role, lifecycle state, last cycle result, last successful discovery time, next scheduled run, and bounded safe error category.
#### Scenario: Provider rejects credentials
- **WHEN** a discovery producer receives a provider authorization error
- **THEN** operations state reports the source and safe authorization category without exposing the credential or provider response body containing secrets
#### Scenario: Producer is stopped
- **WHEN** an operator stops a managed producer through a typed Supervisor action
- **THEN** structured state identifies it as stopped without changing the persisted discovery pause flag
@@ -0,0 +1,182 @@
## ADDED Requirements
### Requirement: Shared strict document validation
Configuration and secrets preview, startup, and privileged apply SHALL use the same side-effect-free validator for bounded UTF-8 YAML, duplicate keys, mapping roots, strict scalar types and bounds, known keys, exact core profile, auth-pool schemas, unique entry names, reference integrity, package capabilities, and deployment paths.
#### Scenario: Valid configuration is previewed
- **WHEN** an operator submits a candidate satisfying the complete schema
- **THEN** preview returns normalized validation success and a bounded diff without activating the candidate
#### Scenario: Duplicate or unknown key is submitted
- **WHEN** a candidate contains a duplicate mapping key or unsupported field
- **THEN** validation fails before staging or restart and identifies the field without echoing secret values
#### Scenario: Secret reference is invalid
- **WHEN** configuration selects an auth-pool entry that does not exist in the candidate secrets
- **THEN** combined validation fails and neither document becomes active
### Requirement: Separate immutable package authority
Worker package manifests SHALL resolve only beneath a separate root-owned, non-writable package authority, while runtime-generated initialization state, locks, and PostgreSQL credentials SHALL remain in the private writable data volume rather than the active config/secrets bind.
#### Scenario: Package manifest uses the active-document directory
- **WHEN** configuration references a package manifest beneath the config/secrets authority or outside the fixed package root
- **THEN** validation fails before startup or privileged apply
#### Scenario: Runtime initializes with active documents mounted read-only
- **WHEN** the runtime initializes or restarts with the active config/secrets directory mounted read-only
- **THEN** its initialization marker, singleton lock, and generated PostgreSQL password remain writable only through fixed private data paths
### Requirement: Candidate revisions and compare-and-swap
The system SHALL stage validated candidate revisions separately from active files and SHALL require expected active and candidate SHA-256 identities when saving or applying them.
#### Scenario: Candidate is saved
- **WHEN** an operator saves valid bytes against the current active hash
- **THEN** the candidate is durably staged with a new hash and the active file is unchanged
#### Scenario: Active file changed through SSH
- **WHEN** the active hash differs from the expected hash submitted by a stale page
- **THEN** save or apply fails without replacing either active file
#### Scenario: Candidate changed concurrently
- **WHEN** the supplied candidate hash is no longer current
- **THEN** apply is rejected before host lifecycle changes begin
### Requirement: Plaintext secrets editing without persistence leakage
The protected secrets page SHALL allow authorized operators to view and edit the complete plaintext YAML document while responses remain no-store and secret values remain absent from browser persistence, application logs, diffs outside that page, operation status, and audit records.
#### Scenario: Operator opens secrets page
- **WHEN** an authenticated operator requests the dedicated secrets editor
- **THEN** the current document is rendered in a server-side form over the protected no-store response
#### Scenario: Secrets candidate is validated
- **WHEN** an operator previews or saves changed secrets
- **THEN** validation results name safe field paths and hashes but do not repeat credential values
#### Scenario: Operator leaves the page
- **WHEN** the browser navigates to another admin page
- **THEN** the application has written no secret value to local storage, session storage, service workers, or client-side application state
### Requirement: Closed host-agent protocol
The privileged host operations agent SHALL accept only a canonical operation UUID, one fixed action enum, expected active hashes, and expected candidate hashes from the authorized runtime peer over a local Unix socket. Deployment profile is root-installed host policy and SHALL NOT be added to that request.
#### Scenario: Valid apply request arrives
- **WHEN** the authorized runtime UID submits an exact valid request
- **THEN** the agent verifies the persisted operation and hashes before acquiring the singleton apply lock
#### Scenario: Request includes path or command data
- **WHEN** a request includes a service name, path, shell text, environment, Docker argument, or unknown field
- **THEN** the agent rejects it before any privileged action
#### Scenario: Request attempts topology selection
- **WHEN** a request includes a profile, ingress port, upstream, Caddy path, unit, service, command, environment, Compose file, or Compose argument
- **THEN** the agent rejects it before reading candidate documents or stopping the deployment
#### Scenario: Unauthorized local peer connects
- **WHEN** a process with an unapproved peer identity uses the socket
- **THEN** the agent rejects the request regardless of its JSON body
### Requirement: Coordinated apply with health verification
For config, secrets, or combined apply, the host agent SHALL revalidate fixed candidate files, verify compare-and-swap hashes, create byte-identical backups, stop the fixed Truf deployment, atomically replace active files, recreate the exact root-selected runtime and edge profile, attest its fixed topology, and wait for required health before reporting success. In shared-host mode it SHALL NOT stop, restart, reload, reconfigure, or remove host Caddy, X-UI, or another unrelated service.
#### Scenario: Combined apply succeeds
- **WHEN** both candidates validate and the restarted deployment reaches Supervisor ACTIVE, PostgreSQL READY, Worker API, ingester, and projector health
- **THEN** the operation completes successfully with resulting hashes and retained rollback evidence
#### Scenario: Validation changes between preview and apply
- **WHEN** host-side revalidation or hash verification differs from the accepted candidate operation
- **THEN** the agent aborts before stopping the healthy runtime
#### Scenario: Apply is requested while another is active
- **WHEN** the singleton operation lock is held
- **THEN** the second request is rejected or remains queued without overlapping lifecycle mutations
#### Scenario: Shared-host profile is healthy
- **WHEN** shared-host apply recreates a host-network runtime with no published ports and an edge bound only to fixed loopback, and all runtime and edge health checks pass
- **THEN** the operation succeeds without a host-Caddy or X-UI lifecycle action
#### Scenario: Shared-host topology drifts
- **WHEN** runtime publishes a port, edge binds a non-loopback address, profile metadata changes, or Compose labels, mounts, network, capabilities, or Caddyfile differ from the fixed profile
- **THEN** the agent rejects the deployment before candidate replacement or reports failed health without claiming success
#### Scenario: Shared-host rollback succeeds
- **WHEN** candidate health fails and the previous Truf runtime and edge are restored
- **THEN** rollback completes exactly once without changing host Caddy, X-UI, or another unrelated service
### Requirement: Automatic rollback and failed hold
If the new deployment fails its bounded health check, the agent SHALL restore byte-identical backups and verify the previous deployment; if rollback also fails, it SHALL stop retrying and retain a failed-hold state and all evidence for SSH recovery.
#### Scenario: New configuration fails startup
- **WHEN** the restarted runtime cannot reach required health within the deadline
- **THEN** the agent restores the prior active files and restarts the previous deployment
#### Scenario: Rollback succeeds
- **WHEN** the restored deployment reaches required health
- **THEN** the operation records rolled-back status and safe failure category without claiming apply success
#### Scenario: Rollback fails
- **WHEN** neither the candidate nor restored deployment becomes healthy
- **THEN** the agent enters failed hold, performs no replacement loop or forced authority release, and preserves backups and diagnostics
### Requirement: Logical managed roots
The generic file page SHALL address only configured logical roots with explicit read, create/replace, and delete permissions, and SHALL never accept an absolute root from a client.
#### Scenario: Operator lists a managed runtime root
- **WHEN** a valid logical root ID for logs, keycheck projections, or result projections and a canonical relative directory are requested
- **THEN** the service returns a bounded read-only listing of permitted regular files and directories under that root
#### Scenario: Operator downloads rotated result projections
- **WHEN** the operator requests a permitted active or rotated result projection within its configured file-size bound
- **THEN** the service returns that regular single-link file without granting mutation access or exposing the backing runtime path
#### Scenario: Result projection names are allowlisted
- **WHEN** the result-projection root is listed or read
- **THEN** only active `scan_results.jsonl` and `found_secrets.jsonl` files and their exact six-digit generation names are visible, while locks, databases, ledgers, scan errors, temporary/quarantine directories, malformed generations, and recovery artifacts remain unavailable
#### Scenario: Large result projection is downloaded
- **WHEN** an allowed result projection is within the larger result-root byte bound
- **THEN** the service copies and hashes an unchanged source revision into an anonymous same-volume snapshot using bounded chunks, permits only one such snapshot at a time, streams the snapshot with bounded memory, and releases the snapshot and concurrency slot after response completion or failure
#### Scenario: Operator requests excluded storage
- **WHEN** a request targets application code, config/secrets through the generic page, PostgreSQL storage, sockets, host-agent metadata, raw result bundles, or an unknown root
- **THEN** the service rejects it without revealing host paths or existence details
### Requirement: Descriptor-safe path containment
Managed-file traversal and mutation SHALL use a retained root directory descriptor, component-wise no-follow operations, canonical relative components, and regular single-link file checks.
#### Scenario: Relative traversal is attempted
- **WHEN** a path contains an empty, dot, dot-dot, absolute, drive-qualified, backslash, NUL, over-depth, or over-length component
- **THEN** the request is rejected before filesystem access outside the retained root
#### Scenario: Symlink is swapped during access
- **WHEN** a path component becomes a symlink between validation and open
- **THEN** no-follow descriptor traversal fails without accessing the link target
#### Scenario: Hardlink or special file is targeted
- **WHEN** the final object is multi-linked or is not a regular file
- **THEN** view, download, replace, and delete are rejected
### Requirement: Durable bounded file mutation
An allowed managed-file create or replace SHALL use an exclusive same-directory temporary regular file, bounded bytes, fsync, atomic descriptor-relative replacement, directory fsync, and final ownership/type/mode verification.
#### Scenario: File replacement succeeds
- **WHEN** an authorized bounded replacement is submitted against the current file hash
- **THEN** readers observe either the complete old file or complete new file and audit records the safe before/after hashes
#### Scenario: File is concurrently changed
- **WHEN** the current file hash differs from the expected hash
- **THEN** replacement fails without overwriting the concurrent change
#### Scenario: Upload exceeds the root limit
- **WHEN** submitted bytes exceed the configured bounded file size
- **THEN** the service rejects and removes temporary data without changing the target
### Requirement: Content-free operational audit
Configuration, secrets, and managed-file operations SHALL record actor, typed action, logical target, timestamps, result, safe category, byte counts where applicable, and before/after hashes, but SHALL NOT record file contents or credentials.
#### Scenario: Managed file is deleted
- **WHEN** an allowed delete succeeds against the expected hash
- **THEN** audit records the logical root/path and previous hash without retaining deleted content
#### Scenario: Secrets apply fails
- **WHEN** a secrets operation fails validation, startup, or rollback
- **THEN** status and audit expose only the bounded failure category and document hashes
@@ -0,0 +1,120 @@
## ADDED Requirements
### Requirement: Protocol 2 source capabilities
Protocol-2 worker packages SHALL advertise explicit source, worker-platform, and planning-kind capabilities for GitLab, DockerHub, and HuggingFace, and the server SHALL issue work only when the selected package supports the complete assignment capability.
#### Scenario: Compatible package requests work
- **WHEN** a protocol-2 package advertising the required capability requests a supported target
- **THEN** the server may create an assignment using that capability
#### Scenario: Package lacks planning capability
- **WHEN** a package advertises the source but not the required planning kind
- **THEN** the server rejects the claim without reserving a target
#### Scenario: Package advertises unknown capability
- **WHEN** a package manifest contains an unknown source, platform, or planning kind
- **THEN** package validation fails closed
### Requirement: Canonical multisource execution plans
The server SHALL create and validate immutable execution snapshots using `exact_git_v1` for GitLab, `docker_direct_v1` for DockerHub, and `huggingface_space_v1` for HuggingFace.
#### Scenario: GitLab assignment is issued
- **WHEN** a GitLab target is claimed
- **THEN** the assignment binds the existing exact commit and Git scan plan under `exact_git_v1`
#### Scenario: DockerHub assignment is issued
- **WHEN** a public DockerHub target is claimed
- **THEN** the assignment binds an immutable digest reference under `docker_direct_v1` and does not depend on a mutable tag
#### Scenario: HuggingFace assignment is issued
- **WHEN** a public HuggingFace Space is claimed
- **THEN** the assignment binds its canonical Space identifier under `huggingface_space_v1`
#### Scenario: Snapshot shape does not match source
- **WHEN** an execution snapshot's source, worker platform, or planning kind combination is invalid
- **THEN** the server and worker reject it before scanner execution
### Requirement: Fenced assignment authority for every source
Every supported source assignment SHALL bind one user, device, target, fixed expiry, immutable execution snapshot, result reservation, and result bundle identity using the existing PostgreSQL authority model.
#### Scenario: Lost claim response is retried
- **WHEN** the server committed an assignment but the worker did not receive the response
- **THEN** retrying the same admission request returns the same assignment and immutable execution snapshot
#### Scenario: Stale worker uploads
- **WHEN** a worker uploads with an expired, replaced, or mismatched reservation token
- **THEN** the server rejects the upload without changing queue or bundle authority
#### Scenario: Valid result commits
- **WHEN** a valid assigned worker uploads and finalizes its bundle
- **THEN** ingestion commits the queue result and downstream projection work exactly once
### Requirement: Eligible source fallback
The assignment service SHALL try other eligible configured sources when one supported source has no claimable target, while still issuing at most one assignment for an admission request.
#### Scenario: Initially selected source is empty
- **WHEN** the first eligible source has no claimable target and another eligible source does
- **THEN** the same claim request may receive one assignment from the other source
#### Scenario: All eligible sources are empty
- **WHEN** no compatible source has a claimable target
- **THEN** the claim returns no work and creates no reservation
### Requirement: Legacy protocol-1 completion compatibility
After protocol-2 cutover, the server SHALL stop issuing new claims to protocol-1 packages but SHALL continue status, terminal report, upload, receipt replay, and immutable snapshot reconciliation for already-issued protocol-1 assignments until they resolve or expire.
#### Scenario: Protocol-1 package requests a new claim
- **WHEN** a legacy GitHub/GitLab-only package requests new work after cutover
- **THEN** the server returns an incompatibility response and creates no assignment
#### Scenario: Existing protocol-1 assignment uploads
- **WHEN** a legacy worker uploads a valid result for an assignment issued before cutover
- **THEN** the server accepts and ingests it under its original immutable authority
#### Scenario: Legacy snapshot is reconciled
- **WHEN** the server reconstructs a lost response for an existing protocol-1 assignment
- **THEN** it reads the original snapshot without rewriting it into protocol 2
### Requirement: DockerHub end-to-end canary
The rollout SHALL prove a bounded DockerHub `search -> enqueue -> claim -> scan -> upload -> ingestion -> projection` cycle using a public image resolved to an immutable digest before broader new-source enablement.
#### Scenario: DockerHub canary succeeds
- **WHEN** a canary producer discovers the configured public image and a compatible worker processes it
- **THEN** the target reaches database-committed ingestion and projection under one fenced assignment
#### Scenario: Mutable tag changes during canary
- **WHEN** the discovered tag changes after queue admission
- **THEN** the worker still scans the immutable digest bound in its assignment
#### Scenario: Registry credentials would be required
- **WHEN** the DockerHub canary target cannot be scanned without private registry credentials
- **THEN** the worker returns a bounded inaccessible-provider result, the server applies its declared retryability, and no discovery or registry credential is transferred in the assignment
### Requirement: HuggingFace remote processing
The system SHALL support the same fenced claim-to-ingestion lifecycle for public HuggingFace Spaces without invoking the scanner on the server.
#### Scenario: Public Space completes
- **WHEN** a compatible worker claims and scans a public HuggingFace Space
- **THEN** its result is uploaded, ingested, and projected under the bound assignment
#### Scenario: Space is inaccessible without worker credentials
- **WHEN** a tokenless worker cannot read a claimed HuggingFace Space because its repository is private, protected, removed, or otherwise unavailable
- **THEN** it returns a non-retryable inaccessible result, the server does not retry that target, and the server discovery token is never exposed
### Requirement: Worker-authoritative provider access
The server SHALL validate canonical target and assignment authority but SHALL treat worker execution as the final provider-access check. A source SHALL NOT require a per-target server access probe, durable public-access proof, proof-freshness state, broad child-environment credential scrubbing, credential sandbox, or post-hoc redaction pipeline unless the operator separately approves an explicit OpenSpec requirement and implementation task.
#### Scenario: Provider accessibility changes after discovery
- **WHEN** a canonical target becomes inaccessible before worker execution
- **THEN** the worker returns the source's bounded permanent or retryable provider-failure result and the server settles or retries it according to that result
#### Scenario: Another source adapter is proposed
- **WHEN** implementation would add preventive access proof or source-specific security infrastructure beyond the assignment's declared fields
- **THEN** implementation pauses until the operator approves a dedicated requirement and task
### Requirement: Credential and result secrecy
Provider credentials, worker device tokens, authorization headers, and result contents SHALL NOT appear in operation status, audit records, Supervisor snapshots, or routine assignment logs.
#### Scenario: Assignment logging occurs
- **WHEN** any supported source assignment is created, retried, rejected, or completed
- **THEN** logs identify bounded source and authority metadata without credential values or result payload bytes
@@ -0,0 +1,142 @@
## ADDED Requirements
### Requirement: Protected typed admin routes
The operations console SHALL expose explicit server-rendered routes and exact mutation forms behind the existing random admin path, Caddy Basic authentication, trusted edge marker, exact same-origin check, CSRF validation, no-store responses, and restrictive security headers.
#### Scenario: Authorized operator opens a page
- **WHEN** Caddy authenticates the request and injects the trusted marker and operator identity
- **THEN** the requested operations page renders escaped server-side HTML with no client-side secret persistence
#### Scenario: Direct backend request lacks marker
- **WHEN** a request reaches an admin route without the trusted edge marker
- **THEN** the backend rejects it regardless of supplied operator headers
#### Scenario: Mutation has stale or invalid CSRF
- **WHEN** a POST has a missing, duplicate, or invalid CSRF value or wrong Origin
- **THEN** the backend rejects the mutation without side effects
#### Scenario: Unknown route or form action is submitted
- **WHEN** a request contains an unsupported method, route shape, action, field, or duplicate field
- **THEN** it fails closed without invoking Supervisor, database mutations, or host operations
### Requirement: Trusted operator attribution
Caddy SHALL strip any inbound operator identity header and inject the authenticated Basic-auth username, and the backend SHALL trust that identity only with the private edge marker.
#### Scenario: Client spoofs operator header
- **WHEN** a public request supplies its own operator identity header
- **THEN** Caddy removes it and the audit actor is the authenticated Basic-auth user
#### Scenario: Mutation is accepted
- **WHEN** an authenticated operator performs a valid mutation
- **THEN** the control or operation record and its audit event identify that operator
### Requirement: Exact production ingress profiles
The production deployment SHALL use exactly one root-installed profile: `standalone-edge-v1` or `shared-host-edge-v1`. The selected profile SHALL NOT be supplied by an admin request, host-agent request, runtime document, or other unprivileged input.
#### Scenario: Standalone edge is selected
- **WHEN** `standalone-edge-v1` is installed
- **THEN** the managed Truf edge remains the sole Truf listener on host port 443 and retains the exact runtime-network-namespace contract
#### Scenario: Shared-host edge is selected
- **WHEN** `shared-host-edge-v1` is installed
- **THEN** the existing root-owned host Caddy remains the sole owner of ports 80/443 and proxies only fixed Truf routes to a managed edge bound at `127.0.0.1:18766`
#### Scenario: A request attempts to select topology
- **WHEN** a request supplies a deployment mode, upstream, port, Caddy path, unit, service, command, or Compose argument
- **THEN** it is rejected before lifecycle work
### Requirement: Shared-host route confinement
The shared-host profile SHALL install a fixed root-owned route-only host-Caddy snippet. It SHALL claim only `/api/v1/worker/*`, the exact random admin-prefix root, and that prefix's subtree. It SHALL strip inbound private and transit headers, inject an independent ingress marker and canonical client address, and preserve the managed edge's authentication, operator attribution, denylist, redacted logging, and security-header behavior without adding a listener, TLS policy, global error handler, trusted-proxy policy, catch-all, or unrelated route.
#### Scenario: An unrelated host route is requested
- **WHEN** a request does not match a Truf worker or admin path
- **THEN** the Truf snippet does not handle or alter the request
#### Scenario: The loopback Truf edge is unavailable
- **WHEN** a matching route cannot reach `127.0.0.1:18766`
- **THEN** host Caddy fails that Truf request without forwarding it to X-UI or another fallback upstream
#### Scenario: A client supplies transit headers
- **WHEN** a public request supplies an ingress marker, forwarded address, private edge marker, or operator identity
- **THEN** host Caddy strips those values and injects only its reviewed ingress marker and observed client address
### Requirement: Runtime overview
The overview page SHALL report bounded structured health for Supervisor, PostgreSQL, required pipeline workers, discovery producers, queue status, active remote assignments, result bundles, operation controls, and recent operation outcomes.
#### Scenario: Runtime is healthy
- **WHEN** all required components hold valid authority and health
- **THEN** the overview reports the runtime active and identifies each required component without exposing secrets
#### Scenario: Component is unavailable
- **WHEN** a health source times out or returns malformed state
- **THEN** the overview reports that component unavailable without blocking the rest of the page
### Requirement: Search controls
The search page SHALL expose each core producer's structured state and typed start, stop, restart, pause, resume, and interval controls while clearly separating process lifecycle from the persistent discovery gate.
#### Scenario: Operator pauses search
- **WHEN** an operator submits pause with the current control revision
- **THEN** the persistent discovery gate changes atomically and every producer stops admitting new discovered targets
#### Scenario: Operator restarts one producer
- **WHEN** an operator selects restart for an allowed producer ID
- **THEN** only that managed discovery process restarts and the persistent pause state is unchanged
### Requirement: Dispatch and drain controls
The workers/dispatch page SHALL expose persistent dispatch pause/resume, drain start/cancel, drain progress, compatible package state, worker users/devices, assignment counts, and upload availability.
#### Scenario: Operator pauses dispatch
- **WHEN** the current revision is submitted to the pause action
- **THEN** no new worker assignment can commit while valid existing uploads remain accepted
#### Scenario: Operator starts drain
- **WHEN** drain is started
- **THEN** the page reports draining progress from authoritative assignment and bundle counts until the state becomes drained
#### Scenario: Stale page attempts resume
- **WHEN** another operator has changed the control revision before resume is submitted
- **THEN** the console reports a revision conflict and does not overwrite the newer state
### Requirement: Typed Supervisor operations
The console SHALL use a closed Supervisor protocol for structured snapshot, allowlisted managed-source lifecycle actions, and bounded log tail, and SHALL NOT forward generic command strings.
#### Scenario: Operator requests source status
- **WHEN** the Supervisor page loads
- **THEN** it displays structured source IDs, roles, phases, process state, restart state, and safe errors without parsing a text dashboard
#### Scenario: Operator tails logs
- **WHEN** an allowed managed source and bounded line count are submitted
- **THEN** Supervisor returns only that source's bounded log tail
#### Scenario: Input resembles a shell command
- **WHEN** an operator submits command text, a path, or an unrecognized source ID
- **THEN** the request is rejected and no generic Supervisor command or operating-system shell is called
### Requirement: Durable asynchronous operation status
Long-running restart and apply actions SHALL create a PostgreSQL operation record before execution and SHALL remain queryable by operation ID across runtime/admin restarts.
#### Scenario: Apply restarts the admin process
- **WHEN** the process that accepted an apply request terminates during the coordinated restart
- **THEN** the operator can reopen the operation URL and observe reconciled success, rollback, or failure state
#### Scenario: Unknown operation is requested
- **WHEN** an operator requests an operation ID that does not exist or is not canonical
- **THEN** the console returns not found without searching filesystem paths or host-agent state by user input
### Requirement: Append-only audit view
The audit page SHALL show bounded append-only events for accepted and completed controls, Supervisor actions, configuration/secrets operations, and managed-file mutations, including actor, action, logical target, time, result, and safe before/after identity.
#### Scenario: Secret apply is audited
- **WHEN** a secrets candidate is accepted and later applied or rolled back
- **THEN** audit events record hashes and outcomes but no secret value, candidate bytes, authorization data, or CSRF value
#### Scenario: Audit pagination is requested
- **WHEN** an operator navigates audit history
- **THEN** the backend returns a bounded deterministic page without unbounded database or browser output
### Requirement: Existing worker API availability
Adding the operations console SHALL NOT weaken or couple public worker endpoints to admin page availability.
#### Scenario: Admin feature is disabled or unhealthy
- **WHEN** the admin console is disabled or a Supervisor/host-agent status dependency is unavailable
- **THEN** authenticated worker status, upload, terminal report, and receipt paths continue under their existing authority