experiment_run.nudge_id), the short steering line that tells enrollees how to phrase their questions. Compare the runs' scorecards afterward.mono.experiment record) is set up in the Design-mode Screen. It holds the parts that stay constant across every run: the assistant's hidden instructions (system_prompt), the enrollee-visible framing (scenario), the opener (opening_message), and whether enrollees see the model competition (blinded, on by default). It also holds the grading set-up: the chosen scoring rubric (score_rubric_version), picked from a drop-down of the rubrics the middle tier bundles rather than typed by hand; the one trusted judge model (score_judge_provider · score_judge_model); and the weights that fill score_weights, seeded from the install default. Its launch defaults (run_defaults) supply a starting model set-up, a starting nudge, and whether to keep candidate answers.
model_config and its member models) exists, authored in the Model-configurations Screen.cohort with cohort_member rows), exists, built in the Cohorts Screen.experiment.run_defaults, model_config, cohort_memberexperiment_run) is created, meaning one live execution of the study. It starts at state = running and points at this run's nudge, model set-up, and cohort through foreign keys (nudge_id, model_config_id, cohort_id). Those definitions, and the parent experiment, are held immutable while the run references them, so later edits cannot change what this run used — with no copy to store.
chat_round, with its chat_round_candidate answers). The study's one trusted judge fills the rubric's factor scores (chat_round.factor_scores) and the headline chat_round.score_composite.state = done, a one-row scorecard (run_result) is built. It records the composite mean and median, a five-number summary per factor, and pass-count, speed, and token statistics, all computed with the enrollee as the unit.experiment_run, run_enrollment, chat_round, chat_round_candidate, run_result| If… | Then… |
|---|---|
An enrollee in the chosen group is already in another run that is running or paused | The launch is blocked and the Launch-run Screen shows an overlap warning. End or wait for the other run, or pick a non-overlapping group — which is why a series on one group runs one run at a time. |
| Fewer than two runs have finished | There is nothing to compare yet. |
| They try to clear a run that is still live | Refused; a run is cleaned up or purged only once it is done or aborted. |
experiment_run.nudge_id (same group and model set-up) with no overlap, Then N runs reach state = done, N run_result scorecards exist, and the Results Screen shows a by-nudge comparison of their composite_mean.experiment_run.state) — and what happens on each change.experimenter-monitor.html). Pause, resume, and abort are the buttons there. Reaching done builds the scorecard; only a done or aborted run can be cleared.run_result when the run ends, and the cross-run comparison is query-time SQL that groups those run_result rows by nudge. The blocked-overlap branch is in the journey above.An experimenter is trying to learn one thing: does the way you coach people to phrase their questions change how good the assistant's answers are? That coaching line is what ChatMaestro calls the nudge. Examples are “keep it short and direct” and “ask for step-by-step reasoning.”
To test the nudge fairly, everything else has to stay the same. So the experimenter sets the study up once: the assistant's hidden instructions, what enrollees see, which AI model set-up answers, and how answers are graded. They also build one group of enrollees to run it on. All of that is fixed.
Then they run the study several times in a row, on the same group, changing only the nudge each time. ChatMaestro pre-fills the launch form from the study's defaults, so the only thing they touch is the nudge. One rule keeps things honest: two runs cannot be live on the same people at once, because their answers would get tangled, so the runs go one after another.
While a run is live, a single judge AI scores every question-and-answer the same way. It grades four qualities: whether the answer is grounded in the source material, relevant to the question, internally coherent, and does what was asked, which is called instruction-following. Those four qualities roll into one headline score. When a run ends, ChatMaestro rolls the scores up into a one-line scorecard for that run.
Finally, the experimenter opens the Results Screen and puts the scorecards side by side. Because only the nudge changed between runs, any difference in the scores points at the nudge. To see why, they can open any exchange and read all the candidate answers the models produced, not just the one the enrollee saw.
run_result rows on the changing nudge. Every step above maps to existing tables, fields, and status values.