Learning V1
ENV
Schema and pool reach every Job
All 120 dynamic runner Jobs imported the runtime ConfigMap that carries LEARNING_QUEUE_SCHEMA and LEARNING_QUEUE_PRODUCER_POOL_SIZE.
Kubernetes scale proof
The proof runs every production phase.
A disposable Kind namespace runs Postgres, queue migration, two reconciler replicas, and one Kubernetes Job per attempt. The runner bundles the production plan, batch, reduce, and queue functions with controlled database, model, and blob adapters.
12Runs succeeded
120Jobs created
0Same-work overlaps
12Peak concurrency
Workload
Failure is part of the test
SHAPE
12 scoped Containers × 7 phase items
Each run plans five batches, runs them in parallel, then reduces. The first plan, first batch, and reduce attempt fail once per run under the production retry policy.
2 reconcilers8 workers each
ASSERTIONS
- 12 plans, 60 batches, and 12 reductions finish
- 36 retries and no extra failures
- No same-work overlap
- Batch fanout reaches five per Container
- All 120 Jobs import runtime config
Measured result
166.750 seconds to drain the workload
Production wiring
The chart carries the same queue contract
BUDGET
Count every active connection
The Helm budget includes each active runner’s two product-store connections plus the configured producer pool. The default Learning topology needs at least 513 connections.
What it caught
The proof changed the implementation
IMAGE
Startup dependency leak
The queue entry imported a Flue-only OpenAI dependency. Run-state code moved to a small production-safe module.
LOCK
Cross-pod group race
Queue grouping alone allowed a fetch race. A scoped session advisory lock now spans the full Job lifetime.
RETRY
Batch-coupled failure
One failed Job retried successful siblings. Single-job callbacks now isolate settlement while local workers keep parallelism.