33. The rubric in full

This is the complete, authoritative rubric — eight dimensions, each scored 0–4 against anchored, observable indicators, with half-steps allowed. The companion chapter how to read the rubric explains why it is shaped this way and how to use it; this page carries every anchor, every caution, and the grep/AST cheat-sheet in full, so it can be scored from directly.

Three terms carry weight throughout and are defined earlier in the book: the IO line and loop-above vs loop-below (Part II ch. 16), and goal-fanout — a coordinator turning one robot-wide goal into per-subsystem states (Part I ch. 5). On effort: budget roughly an hour per repo for a careful confirmed score — the grep pass takes minutes; opening the files behind every positive hit is the bulk of the work.

Why dimensions, not a single ladder score

The maturity ladder is right about sequence within a dimension — nobody reaches unit tests without an IO layer, because the IO layer is what makes subsystems testable. But real teams adopt unevenly across dimensions: a team can have a clean IO layer and zero tests, or AdvantageKit logging bolted onto spaghetti coordination, or Choreo trajectories driven by dead-reckoned odometry. A single ladder score hides exactly the gaps that matter most — “you’ve built the IO layer, tests are one step away, and you’re not taking it.”

So: score each dimension independently, report the vector, and read the shape of the profile, not the sum. One revision to the ladder this implies — the teaching order bundles “IO layer + simulation + lightweight logging” as one leap and “FSM + tests + replay” as another; in the wild these decompose. That bundling is the right teaching order but the wrong measurement instrument, so the rubric splits them.

Scoring rules

• Unit — each team’s most recent real competition-season repo (2025 or 2026; not templates, training repos, or off-season toys). Record the repo name and season on the scoresheet.
• Adjacent repos count for one dimension only — separate team-library repos (SuperCORE, WarlordsLib, 3128-common, NOMADBase) and training repos count toward D8 (Sustainability) but not toward the code dimensions, which are scored from the season repo alone.
• Score what’s used, not what’s present. A Choreo vendordep JSON with no .traj files and no Choreo imports is not Choreo adoption. An empty src/test folder is not testing. Confirm every indicator by opening at least one file — see Part I ch. 3 — the IO seam for why the seam is the load-bearing structure to confirm.
• Half-steps are allowed (e.g., 2.5) when a team sits clearly between anchors. Don’t agonize. A legitimate 2.5 on D1 is IO interfaces on most mechanisms but selection still scattered across files — confirmed past the L2 anchor, short of L3. An illegitimate 2.5 is splitting the difference because you couldn’t tell whether the Superstructure is real — that’s an unconfirmed candidate, not a half-step; go open the file.
• Known blind spot — repos are shallow clones with .git stripped, so commit history, PRs, and contributor counts are unobservable. D8 is scored from artifacts only and should be treated as a floor, not a ceiling.
• Language note — anchors are written in Java/WPILib terms. Kotlin (6695) and Python/RobotPy teams hit the same anchors with different syntax: MagicBot’s framework FSM counts at D2 level 2, its component injection counts at D1 level 2.

Corpus prevalence (measured)

From the tree-sitter → DuckDB index of 55 season repos. Use these to calibrate: a marker present in 3 teams is a ceiling signal; one present in 45 is table stakes. Confirm by reading regardless.

Most-common subsystems (by subsystems/<dir>): vision · intake · drive · shooter · climber · elevator, then arm · LEDs · indexer · turret. Manipulator is rare; subsystem names are game-dependent — score the IO/coordination structure, not the mechanism roster.

What predicts competition results — and why you must read the code

Measured 2026 against Statbotics EPA, leave-one-team-out cross-validated over 232 team-years / 55 teams, cluster-bootstrap CIs. (The cross-validation pooled every available season repo per team — 232 team-years across the 55 teams — while rubric scoring uses each team’s most recent season.)

The two validation datasets. This section reports the cross-validated study over the full indexed corpus. The book’s other validation — the San Diego scoresheet — is a companion study: 24 local teams hand-scored on this rubric and paired with season-matched EPA. The two are different samples answering related questions; where this section calibrates the instrument (confirmed vs mechanical passes, per-dimension trust), ch. 34 asks whether the scores track winning in one region.

Three results should shape how you use this rubric:

1. Confirming use, not presence, roughly doubles the rubric’s predictive validity. On the same 55 teams, the agent-confirmed D1–D8 (a model that opened the files) predicts EPA at Spearman ρ ≈ 0.53; the mechanical candidate pass (grep/SQL hits, scored as presence) reaches only ρ ≈ 0.29. The paired difference is significant (95% CI [0.04, 0.44]). This is the empirical case for the golden rule — the cheap pass is a lead sheet, not a score.

2. Most of what predicts EPA from “code” is just size and program age — not engineering quality. A model of ~50 raw code features hits ρ ≈ 0.58, but codebase size + program maturity alone scores ≈ 0.60, while the sophistication features (the rubric’s structural patterns, size removed) score ≈ 0.38, and a within-team (fixed-effects) view collapses to ≈ 0.05. Bigger, older, better-resourced programs have both more code and better results. Do not read a high raw correlation as “good code wins” — and do not reward sheer volume.

3. The mechanical pass is trustworthy on some dimensions and not others. Agreement (quadratic-κ) between the mechanical candidate and the agent-confirmed level:

   Dimension	κ (mechanical vs confirmed)	Trust the grep?
   D4 Testing	0.86	yes — src/test + asserts is unambiguous
   D1 Architecture	0.82	yes — *IO interfaces are concrete
   D3 Simulation	0.81	yes
   D2 Coordination · D5 Logging	0.74	mostly — confirm the coordinator is real, logging covers all subsystems
   D6 Auto/Path · D7 Vision	0.60	no — confirm by reading (files present ≠ trajectories driven / vision fused)
   D8 Sustainability	0.57	no — read the README/CI/library; history is shallow

   Spend your reading budget where κ is low: D6, D7, D8 are where “present” most diverges from “used.”

One design decision deserves its own sentence: the eight dimensions are equally weighted, deliberately. An EPA-optimal re-weighting was tested and was not distinguishable from equal weight on this sample — so don’t over-engineer the sum.

D1 — Hardware decoupling (architecture)

The ladder’s spine — how far has the team pushed the boundary between subsystem logic and physical devices?

Grep: interface .*IO (the spine), plus its implementations — *IOSim* (the sim impl, 19 teams) and a hardware impl named by device: *IOTalonFX* / *IOKraken* / *IOSparkMax* / *IOSpark* / *IOPigeon2* / *IONavX* / *IOLimelight* / *IOPhoton*. Do not rely on *IOReal* — only ~5 teams use that name; the robust signal is an interface *IO with ≥2 implementations, one of them *IOSim. Also @AutoLog, YAGSL config dirs (src/main/deploy/swerve), TunerConstants (CTRE generator). At L4: a generic base (MotorIO/ServoMotorSubsystem) reused across mechanisms — the *IONull* null-object variant is real but near-absent in the corpus (~1 team), so don’t require it.

Distinguish at level 3: an IO directory of concrete hardware wrappers (the 2056 case) is not an IO layer — there must be an interface with swappable implementations. Check for an actual interface and at least two implementations.

D2 — Coordination & decision logic

How does the robot decide what to do and keep mechanisms from fighting?

Grep: Superstructure (the dominant coordinator — 22 teams; check it’s a real goal-fanout, not a holder), generic *StateMachine (12 teams). Rarer variants, each a strong signal when present but few teams: enum WantedState/SystemState (2910 style — 2 teams), RobotManager (581/3128 centralized FSM — 2 teams), handleStateTransitions, jgrapht (3), AStarSolver, BehaviorTree (1). Don’t weight WantedState/RobotManager as the default — they’re niche; the common path is a Superstructure (level 3) optionally backed by a *StateMachine. A newer L3 marker worth recognizing: a request-based API — a Goal/SuperstructureRequest enum or sealed type plus a requestGoal(...)/request(...) method (often returning a Command) and a handleStateTransitions-style guard — which makes intent-vs-execution explicit.

Caution: a class named Superstructure that just holds references is level 1 wearing a level 3 name. Look for an actual goal-request API and transition logic.

D3 — Simulation

Can the code run, and surprise you, without the robot?

Grep: simulationPeriodic, the WPILib physics classes ElevatorSim|SingleJointedArmSim|FlywheelSim|DCMotorSim, maple-sim’s SwerveDriveSimulation (WPILib ships no swerve-drive sim class), maple-sim / org.ironmaple, *IOReplay*, IdealSim.

D4 — Testing & verification

The IO layer’s deferred dividend. Almost no team collects it — the sharpest discriminator in the corpus.

Grep: src/test/java tree, @Test, assertEquals, .github/workflows/*.yml containing test, runToCompletion, CommandTestBase.

Note the asymmetry with D3: physics sim without tests (common) scores D3 high / D4 low. That gap is the single most actionable finding for a team — flag it in notes.

D5 — Logging, telemetry & diagnostics

“When the robot misbehaves on the field, how do we know why?”

Grep: doglog, Epilogue, @Logged, URCL, org.littletonrobotics.junction, Logger.processInputs, @AutoLog, FaultReporter, SystemCheck, committed .json AdvantageScope layouts.

Caution: AdvantageKit vendored but with processInputs on one subsystem out of ten is level 2, not 3. The level is about coverage, not presence — gauge coverage by the count of Logger.processInputs / recordOutput call sites relative to the subsystem roster (a handful = partial; dozens across every subsystem = real L3).

D6 — Autonomous & path planning

Are the three path concerns (authored path, optimal trajectory, reactive avoidance) present and pulled apart?

Grep: pathplanner dir under src/main/deploy, PathPlannerLib.json, choreo / .traj / .chor, Repulsor, RepulsorField, AutoBuilder, AutoBuilder.pathfind* (on-the-fly, L3).

Confirm use — this is the lowest-trust dimension (κ 0.60). Choreo .traj/.chor files count for nothing on their own: 15 teams in the corpus carry Choreo files they never reference in code, and PathPlanner .auto files frequently carry choreoAuto:false. Require an actual code reference — fromChoreoTrajectory(...), Choreo.loadTrajectory(...), a choreo.auto.AutoFactory — before crediting Choreo at L3. Likewise confirm AutoBuilder.configure(...) is wired with real constraints, not just imported.

D7 — Localization & vision

What does the robot believe about where it is, and how is that belief maintained?

Grep: photonlib, PhotonCamera, LimelightHelpers, addVisionMeasurement, SwerveDrivePoseEstimator, RobotState, TimeInterpolatableBuffer, MegaTag. For the L3 fused-and-filtered rung, the discriminating markers are setVisionMeasurementStdDevs (dynamic std-dev tuning) and a VisionIO interface with a sim variant (vision decoupled from Drive); for L4, a dedicated RobotState whose pose lives in a TimeInterpolatableBuffer/ConcurrentTimeInterpolatableBuffer.

Confirm use (κ 0.60). addVisionMeasurement is near-universal (50/55 teams) — it is the L2 floor, not evidence of L3. The level is set by the filtering (rejection logic, per-tag std-devs) and architecture (IO seam, central world-model), which you must read to see.

D8 — Sustainability & process (artifact-based)

Will this codebase survive its seniors graduating? Scored from artifacts only — treat as a floor.

Grep: README.md length and content, CONTRIBUTING, spotless, checkstyle, .github/workflows, separate lib/common repos in team folder, shared/ or variant packages.

Scoresheet template

One row per team. The sum is reported, but the profile is the finding.

Reading the profile: common shapes

• Balanced climber — all dimensions within ±1 of each other. The ladder is working; next step is whatever’s lowest.
• Architecture without verification (D1 ≥ 3, D3/D4 ≤ 1) — adopted the IO layer’s form without its payoff. Likely copied a template (check against 5712-style AdvantageKit templates). Highest-leverage intervention: one unit test.
• Tooling adopter (D5/D6 high, D1/D2 low) — uses AdvantageScope/PathPlanner/Choreo but the code underneath is baseline command-based. Tools were installable; architecture wasn’t. Intervention: an IO layer on one subsystem.
• Template inheritor — D1 = 3 exactly matching a known public template, everything else low. Distinguish from organic adoption by checking whether IO interfaces exist for their mechanisms or only the swerve they forked.
• Legacy program — competent older patterns (solid command-based, good autos) with no post-2022 tooling. Different conversation: modernization, not fundamentals.
• Verification ceiling (everything ≥ 3 except D4) — the regional-elite profile; even strong teams rarely test. D4 ≥ 2 is the rarest marker in the national corpus and the clearest signal of real software-engineering culture.

Suggested scoring procedure

1. Identify the latest real season repo; record season and language.
2. Run the grep cheat-sheet (one pass per dimension) to generate candidate levels.
3. Open the files behind every positive hit — confirm use, not presence. Adjust the level.
4. Check the team’s other repos only for D8 (libraries, templates, training).
5. Write 2–3 sentences of profile notes: the shape, the likely explanation, the one highest-leverage next step for the team.

A worked example of what step 2 produces — the raw mechanical-candidate output for one Reefscape 2025 repo, with per-dimension AST hits and the files to open — lives in the repository at knowledge/examples/sample-score-output-reefscape2025.md. It is a lead sheet in exactly the sense above: a heuristic Σ floor plus a list of files to confirm, not a score.