Initial server source import
This commit is contained in:
sashatrask
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-09-05
|
||||
@@ -0,0 +1,75 @@
|
||||
## Context
|
||||
|
||||
The configured sources contain 543 query occurrences covering 133 normalized terms. Canonical PostgreSQL lineage links discovery scans to credentials through both candidate and result records, and the dashboard already defines the strict `usable_llm` tier. Applying that rule across every provider identified 38 non-operational terms with zero historical strict-usable linkage despite 117,492 scans and 1,890.6 cumulative scanner-hours. The same terms consumed 18,641 scans and 317.1 scanner-hours in the latest 30-day window.
|
||||
|
||||
The runtime configuration is code-authority protected. Query rotation state, target queues, scan history, and credential history are separate persisted authorities and must not be rewritten to deploy this change.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Remove only globally zero-yield terms with enough exposure to support a conservative decision.
|
||||
- Apply one decision consistently anywhere the exact normalized term is configured.
|
||||
- Preserve operational source sentinels and all terms with any demonstrated strict-usable linkage.
|
||||
- Remove stale per-query overrides and verify deterministic post-prune query sets.
|
||||
- Deploy through the coordinated authority lifecycle and verify normal source rotation.
|
||||
|
||||
**Non-Goals:**
|
||||
- Deleting, reprioritizing, or rewriting existing target backlog or historical records.
|
||||
- Optimizing for one provider, broad `alive`, raw findings, or candidate volume.
|
||||
- Changing detector routing, keycheck classification, source concurrency, scan limits, or query-state files.
|
||||
- Claiming that a retired term can never produce a useful credential in the future.
|
||||
|
||||
## Decisions
|
||||
|
||||
### Use all-provider strict-usable evidence
|
||||
|
||||
A term is eligible only when no credential linked to that term has ever reached the canonical dashboard `usable_llm` tier and no earliest-origin credential attributed to it has reached that tier. Candidate/result scan links are unioned before attribution so migrated and resolver-routed credentials are not lost.
|
||||
|
||||
The decision is global by case-normalized exact term. If a term produced one strict-usable credential for any provider or source, it remains configured everywhere. This is more conservative than pruning source-term pairs independently and avoids removing cross-provider terms such as `groq`, `llm`, `chat`, `rag`, or `langchain`.
|
||||
|
||||
Alternative: use broad `status_group=alive` or OpenAI-only yield. Rejected because broad alive contains unproven and historically misclassified statuses, while provider-only analysis can remove terms that work for another provider.
|
||||
|
||||
### Require meaningful exposure
|
||||
|
||||
A zero-yield term qualifies when either it has at least 30 linked credential observations, or it has at least 200 completed scan events and 20 cumulative scanner-hours. The credential branch tests precision; the cost branch catches terms that repeatedly consume work without reaching candidate intake. The threshold is applied to all retained history, with the latest 30-day cost recorded as corroborating evidence.
|
||||
|
||||
Alternative: remove every zero-yield term. Rejected because recent and low-sample terms have insufficient evidence. Those terms remain canaries.
|
||||
|
||||
### Exempt source-operational sentinels
|
||||
|
||||
`gharchive`, `gharchive-files`, and `gists` are sole query tokens used to operate dedicated sources rather than interchangeable discovery keywords. They remain even though they have no strict-usable attribution. Emptying those lists would disable or invalidate source operation rather than merely prune a search term.
|
||||
|
||||
### Retire the approved cohort consistently
|
||||
|
||||
Remove these 32 terms from GitHub, GitLab, DockerHub, npm, PyPI, and package-git: `autonomous`, `benchmarks`, `claw`, `code-assistant`, `codegen`, `dspy`, `embedding`, `embeddings`, `eval`, `evals`, `gateway`, `grok`, `haystack`, `inference`, `inference-api`, `knowledge`, `llamaindex`, `model`, `model-router`, `models`, `ollama`, `orchestration`, `prompts`, `replicate`, `rerank`, `reranker`, `retrieval`, `router`, `tokenizer`, `tool-use`, `vector`, and `vllm`.
|
||||
|
||||
Remove `chatgpt`, `gpt`, and `moonshot` from those six sources and Postman. Remove `openai` from GitHub, GitLab, DockerHub, and Postman. Remove `dashscope-intl.aliyuncs.com` and `generativelanguage.googleapis.com` from Postman.
|
||||
|
||||
This removes 219 occurrences. Resulting list sizes are GitHub 64, GitLab 46, DockerHub 46, npm 43, PyPI 43, package-git 43, and Postman 33.
|
||||
|
||||
### Keep deployment configuration-only
|
||||
|
||||
Delete the three `openai` query overrides together with the query entries. Existing rotation reads `query_index` modulo the current list length, so no persisted state edit is needed. Runtime is stopped before editing and restarted only after tests and strict OpenSpec validation.
|
||||
|
||||
Alternative: rewrite query indices or purge queued targets attributed to removed terms. Rejected because both mutate independent durable authority and are unnecessary for preventing future discovery.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [Historical zero yield may not predict future supply] -> Keep low-sample terms, retain all historical evidence, and make rollback a configuration-only restoration.
|
||||
- [Earliest-origin attribution can hide useful rediscovery] -> Require zero strict-usable linkage across every scan link in addition to zero origin yield.
|
||||
- [Large list reduction changes rotation cadence] -> Verify exact list sizes and allow normal modulo-based state handling; do not edit source state.
|
||||
- [Completed OpenAI rollout previously required the literal term] -> Record the requirement retirement explicitly and retain higher-signal bounded ecosystem queries.
|
||||
- [Authority drift during a live edit] -> Use coordinated stop, test, and canonical start rather than relying on fail-close shutdown.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. Add configuration contract tests for the exact retired set, retained sentinels, uniqueness, post-prune sizes, and absence of orphaned overrides.
|
||||
2. Stop the authenticated runtime coordinately.
|
||||
3. Remove the 219 query occurrences and three matching overrides from `app/config.yaml`; do not edit state or queue data.
|
||||
4. Run focused query tests, configuration/runtime safety tests as applicable, and strict OpenSpec validation.
|
||||
5. Restart through `start_runtime.ps1` and verify authenticated supervisor, PostgreSQL, pipeline readiness, source processes, and query-list loading.
|
||||
6. Roll back by restoring the configuration entries and overrides through the same coordinated lifecycle if source health regresses.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None.
|
||||
@@ -0,0 +1,23 @@
|
||||
## Why
|
||||
|
||||
Current discovery rotations spend substantial scanner time on query terms that have accumulated meaningful exposure without linking to a single strict-usable credential for any provider. Removing only this globally zero-yield cohort reduces avoidable discovery and scan work while preserving every query with demonstrated usable yield.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Remove 38 sufficiently exposed, globally zero-yield search terms from the configured GitHub, GitLab, DockerHub, npm, PyPI, package-git, and Postman rotations.
|
||||
- Remove query-scoped overrides whose corresponding query is retired.
|
||||
- Preserve source-operational sentinel queries and every term linked to at least one historical strict-usable credential for any provider.
|
||||
- Preserve historical queue rows, scan results, credential lineage, deduplication state, and all runtime concurrency limits.
|
||||
- Define a repeatable evidence rule for future pruning instead of using raw findings or provider-specific yield alone.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `discovery-keyword-pruning`: Evidence-based, all-provider retirement of sufficiently tested zero-yield discovery terms.
|
||||
|
||||
### Modified Capabilities
|
||||
- `openai-discovery-coverage`: Retire the literal `openai` core query after its bounded rollout produced no strict-usable credential for any provider.
|
||||
|
||||
## Impact
|
||||
|
||||
The change affects `app/config.yaml`, focused query-configuration tests, and authority-managed source rotation after a coordinated restart. It removes 219 configured query occurrences but introduces no schema migration, dependency, queue rewrite, credential recheck, detector change, or scan-concurrency change.
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: All-provider strict-yield pruning rule
|
||||
The system SHALL retire a discovery term only when canonical lineage shows zero historical strict-usable credential linkage for every provider and the term has meaningful measured exposure.
|
||||
|
||||
#### Scenario: Any strict-usable linkage preserves a term
|
||||
- **WHEN** any credential linked to a configured term has ever met the canonical `usable_llm` rule
|
||||
- **THEN** that term SHALL remain in every configured source rotation
|
||||
|
||||
#### Scenario: Credential exposure qualifies a zero-yield term
|
||||
- **WHEN** a term has zero strict-usable linkage and at least 30 linked credential observations
|
||||
- **THEN** the term SHALL qualify for retirement
|
||||
|
||||
#### Scenario: Scanner-cost exposure qualifies a zero-yield term
|
||||
- **WHEN** a term has zero strict-usable linkage, at least 200 scan events, and at least 20 cumulative scanner-hours
|
||||
- **THEN** the term SHALL qualify for retirement
|
||||
|
||||
#### Scenario: Low-exposure zero-yield term remains a canary
|
||||
- **WHEN** a zero-yield term satisfies neither exposure condition
|
||||
- **THEN** it SHALL remain configured until more evidence is available
|
||||
|
||||
### Requirement: Approved global retirement cohort
|
||||
The system SHALL omit the approved 38-term zero-yield cohort from every source rotation where each exact term was configured.
|
||||
|
||||
#### Scenario: Shared broad-source cohort is removed
|
||||
- **WHEN** GitHub, GitLab, DockerHub, npm, PyPI, or package-git loads its query rotation
|
||||
- **THEN** it SHALL omit `autonomous`, `benchmarks`, `claw`, `code-assistant`, `codegen`, `dspy`, `embedding`, `embeddings`, `eval`, `evals`, `gateway`, `grok`, `haystack`, `inference`, `inference-api`, `knowledge`, `llamaindex`, `model`, `model-router`, `models`, `ollama`, `orchestration`, `prompts`, `replicate`, `rerank`, `reranker`, `retrieval`, `router`, `tokenizer`, `tool-use`, `vector`, and `vllm`
|
||||
|
||||
#### Scenario: Cross-source zero-yield terms are removed
|
||||
- **WHEN** an affected rotation is loaded
|
||||
- **THEN** `chatgpt`, `gpt`, and `moonshot` SHALL be absent from GitHub, GitLab, DockerHub, npm, PyPI, package-git, and Postman, and `openai` SHALL be absent from GitHub, GitLab, DockerHub, and Postman
|
||||
|
||||
#### Scenario: Zero-yield Postman signatures are removed
|
||||
- **WHEN** Postman loads its query rotation
|
||||
- **THEN** `dashscope-intl.aliyuncs.com` and `generativelanguage.googleapis.com` SHALL be absent
|
||||
|
||||
#### Scenario: Post-prune list sizes are deterministic
|
||||
- **WHEN** canonical configuration is loaded
|
||||
- **THEN** query counts SHALL be GitHub 64, GitLab 46, DockerHub 46, npm 43, PyPI 43, package-git 43, and Postman 33
|
||||
|
||||
### Requirement: Operational and historical authority is preserved
|
||||
Keyword retirement SHALL stop future discovery for the retired terms without deleting or rewriting source state, target queues, scans, findings, credentials, or results.
|
||||
|
||||
#### Scenario: Dedicated source sentinels remain
|
||||
- **WHEN** archive and gist source rotations are loaded
|
||||
- **THEN** `gharchive`, `gharchive-files`, and `gists` SHALL remain as their sole configured query tokens
|
||||
|
||||
#### Scenario: Persisted rotation index remains valid
|
||||
- **WHEN** an existing query index exceeds a shortened query list
|
||||
- **THEN** normal modulo-based rotation SHALL select a valid configured query without a state-file edit
|
||||
|
||||
#### Scenario: Existing backlog remains intact
|
||||
- **WHEN** the pruned configuration is deployed
|
||||
- **THEN** previously admitted targets and all historical attribution records SHALL remain unchanged
|
||||
|
||||
### Requirement: Retired query overrides are removed
|
||||
The canonical configuration SHALL NOT retain a query override for a retired query.
|
||||
|
||||
#### Scenario: Literal OpenAI overrides are absent
|
||||
- **WHEN** GitHub, GitLab, and DockerHub configuration is loaded
|
||||
- **THEN** each source SHALL omit the `openai` query override while preserving overrides for retained bounded queries
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Query-scoped safety bounds
|
||||
The system SHALL support exact-query overrides for configured queries only, limited to `pages`, `per_page`, and `max_targets`, without changing source-wide defaults for other queries.
|
||||
|
||||
#### Scenario: Configured query receives bounded arguments
|
||||
- **WHEN** a source builds arguments for a configured query with an exact override
|
||||
- **THEN** it SHALL apply that query's configured page, page-size, and target bounds
|
||||
|
||||
#### Scenario: Retired query has no override
|
||||
- **WHEN** a query is removed from a source rotation
|
||||
- **THEN** the source SHALL NOT retain an override for that query
|
||||
|
||||
#### Scenario: Ordinary query retains source defaults
|
||||
- **WHEN** the same source builds arguments for any query without an override
|
||||
- **THEN** it SHALL retain the source-wide page, page-size, and target values
|
||||
|
||||
#### Scenario: Invalid override fails closed
|
||||
- **WHEN** a query override is not a mapping or contains a key outside the allowlist
|
||||
- **THEN** argument construction SHALL fail before discovery or queue mutation
|
||||
|
||||
## REMOVED Requirements
|
||||
|
||||
### Requirement: Exact OpenAI core discovery
|
||||
**Reason**: The completed bounded rollout produced 43 linked origin credentials and no strict-usable credential for any provider, meeting the approved global retirement rule.
|
||||
|
||||
**Migration**: Remove `openai` from GitHub, GitLab, and DockerHub rotations and allow normal modulo-based query rotation to continue without editing persisted source state.
|
||||
|
||||
### Requirement: Source-specific rollout limits
|
||||
**Reason**: The exact-query rollout is complete and its query is being retired, so source-specific `openai` execution bounds are no longer active policy.
|
||||
|
||||
**Migration**: Remove the three matching `openai` overrides while retaining the generic exact-query override mechanism and all overrides for configured ecosystem queries.
|
||||
|
||||
### Requirement: End-to-end canary evidence
|
||||
**Reason**: The exact-query canary reached terminal evidence and its measured all-provider strict yield is captured by the pruning decision.
|
||||
|
||||
**Migration**: Evaluate future keyword retirement under `discovery-keyword-pruning` using canonical all-provider lineage and measured exposure.
|
||||
@@ -0,0 +1,19 @@
|
||||
## 1. Configuration Contract
|
||||
|
||||
- [x] 1.1 Update focused query tests to assert the exact retired cohort, retained sentinels, retained productive terms, and deterministic list sizes.
|
||||
- [x] 1.2 Assert that retired queries have no orphaned query overrides while retained bounded ecosystem queries remain unchanged.
|
||||
|
||||
## 2. Runtime Configuration
|
||||
|
||||
- [x] 2.1 Stop the authenticated runtime through the coordinated lifecycle before changing authority-covered configuration.
|
||||
- [x] 2.2 Remove the approved 219 query occurrences and three `openai` overrides from `app/config.yaml` without modifying persisted state or backlog data.
|
||||
|
||||
## 3. Verification
|
||||
|
||||
- [x] 3.1 Run focused query/configuration tests and verify canonical configuration loads with the required query sets and counts.
|
||||
- [x] 3.2 Run strict OpenSpec validation and relevant supervisor/runtime safety tests.
|
||||
|
||||
## 4. Deployment
|
||||
|
||||
- [x] 4.1 Start the runtime through `start_runtime.ps1` and verify authenticated supervisor and managed PostgreSQL readiness.
|
||||
- [x] 4.2 Verify pipeline readiness, normal source processes, keychecks, and post-prune query loading without queue mutations.
|
||||
Reference in New Issue
Block a user