# Stage 37: Generate Evaluation (aq37_generate_evaluation / DB-UNIT-17)

Synthesizes deterministic grading harnesses, golden fixtures, adversarial traps, and abstention holdouts for any task queue or capability.

- Atomic micro-queue: `aq37_generate_evaluation` (visibility: `public_commons`)
- Backing database unit: `DB-UNIT-17` (`Cascading Decision Models, Local Classifiers & Abstention Core`)
- Database cell: `lsh-db.teleon.dev` (`teleon_core.decision_rubric_schemas`, `teleon_core.cascade_classification_ledger`, `teleon_core.abstention_calibration_sets`)
- Claim row hydration: `single_self_contained_flat_task_row` (100% context inline, normalized writeback)
- Track 2 Sister queue (`SISTER-17`): `sq17-improve-decisions-queue.teleon.dev` (Distill frontier decisions into compact SetFit/ModernBERT/GLiNER2 CPU heads and tighten natural no-match abstention thresholds.)
- Lanes: Golden & Edge Cases, Executable Grading, Abstain Holdout

## Execution engines & AI Agent freedom contract

- **Deterministic (0 model calls)**: `rule_model2vec_setfit_gliner_cpu_cascade_v1` (Property-test, schema-validator & holdout-split compiler)
- **Hybrid (<=1 model call)**: `clef_jev_clairic_typed_decision_escalation_v1`
- **AI Agent & Bot Fleet (1..6 calls)**: `ambiguous_tail_reasoning_arbitrator_v1` (Adversarial benchmark & rubric designer)
  - Contract: **Instructions + Resources + Freedom to search for new resources and new ways not bound by instructions**

## Database invariants

- Static rubrics precede dynamic item state so input-only decision APIs capture cached-prefix discounts.
- Below-threshold confidence returns explicit ABSTAIN or escalates rather than pushing mismatched components.

## External & internal bot fleet actions

- Build 0-call assertions and property checks wherever correctness is objectively testable
- Include matched benign twins and abstention cases to measure false-positive rates
- Version evaluation suites in Store S8 so every solver and grid cell runs against identical gates

## Sister self-tuning areas

- Distill recurring multi-question rubrics into 8-to-32-shot SetFit and ONNX INT8 CPU classifiers (DIM-03, DIM-06)
- Calibrate per-domain abstention thresholds on 25%+ natural no-suitable-component holdouts (DIM-01, DIM-06)
- Benchmark choice-order invariance across Clef-flash, Jev, Clairic, and Decisions endpoints (DIM-02, DIM-08)