Skip to main content

StudyConfig

from llenergymeasure import StudyConfig

Concept

StudyConfig is the thin resolved container that the study layer operates on. It holds a flat list of fully-validated ExperimentConfig objects (the sweep is already expanded - no axes, no templates) plus two operational blocks: output for where to write results, and study_execution for how to sequence experiments across cycles.

The key design point is that sweep resolution happens at YAML parse time in the loader, before the StudyConfig is constructed. By the time a StudyConfig exists, every item in experiments is a concrete, validated configuration ready to run. The study runner and harness never perform expansion - they iterate the list.

StudyConfig is rarely constructed directly in userland. The normal path is to pass a YAML path to run_study and let the loader produce it. Direct construction is useful for programmatic test generation or pipeline integration.


Construction

From YAML (via run_study - standard path)

from llenergymeasure import run_study

result = run_study("study.yaml")

From YAML (via loader, for inspection)

from llenergymeasure.config.loader import load_study_config
from pathlib import Path

study = load_study_config(path=Path("study.yaml"))
print(f"Expanded to {len(study.experiments)} experiments")

Programmatic construction

from llenergymeasure import StudyConfig, ExperimentConfig
from llenergymeasure.config.models import TaskConfig

study = StudyConfig(
experiments=[
ExperimentConfig(task=TaskConfig(model="gpt2"), engine="transformers"),
ExperimentConfig(task=TaskConfig(model="gpt2-medium"), engine="transformers"),
],
study_name="gpt2-family",
)

Fields

Core fields

FieldTypeDefaultDescription
experimentslist[ExperimentConfig](required)Resolved list of experiments to run. Minimum length 1.
study_namestr | NoneNoneStudy name used in output directory naming (e.g. gpt2-sweep_2026-05-07T14-00-00).
study_design_hashstr | NoneNone16-char SHA-256 hex of the resolved experiment list (execution block excluded). Set by the loader after expansion; None when constructed programmatically.

output block (OutputConfig)

Controls where results are written:

FieldTypeDefaultDescription
results_dirstr | NoneNoneBase directory for study results. None defers to user config or the built-in default (./results). A timestamped subdirectory is created inside this path.
save_timeseriesboolTruePersist GPU power/thermal/memory samples as a Parquet sidecar per experiment. Set to False to save disk space on long sweeps.

study_execution block (ExecutionConfig)

Controls sequencing, cycles, and failure handling:

FieldTypeDefaultDescription
n_cyclesint1Number of times to repeat the full experiment list.
experiment_order"sequential" | "interleave" | "shuffle" | "reverse" | "latin_square""sequential"Ordering across cycles. "interleave" runs A,B,A,B; "shuffle" randomises per-cycle.
experiment_gap_secondsfloat | NoneNoneSeconds to wait between individual experiments. None uses machine default from user config.
cycle_gap_secondsfloat | NoneNoneSeconds to wait between full cycles. None uses machine default.
shuffle_seedint | NoneNoneExplicit seed for shuffle ordering. None derives seed from study_design_hash (same study always shuffles identically).
skip_preflightboolFalseSkip Docker pre-flight checks in the execution block (the --skip-preflight CLI flag always overrides).
max_consecutive_failuresint10Circuit breaker: abort after N consecutive failures. 0 = disabled.
circuit_breaker_cooldown_secondsfloat60.0Cooldown before a half-open probe experiment after the circuit breaker trips.
wall_clock_timeout_hoursfloat | NoneNoneStudy-level wall-clock timeout. None = no limit.
experiment_timeout_secondsfloat600.0Per-experiment timeout. Applies to both local and Docker paths.
stdout_silence_timeout_secondsfloat300.0Docker watchdog: kill the container if it emits no stdout for this many seconds. Raise to 600-900s for TRT-LLM builds with infrequent compile progress output.
deduplicate_equivalentboolTrueWhen True, configs that are library-equivalent (same resolved config hash after engine-invariants dormant resolution) are deduplicated. --no-dedup CLI flag is the override.

Runner and image overrides

FieldTypeDefaultDescription
runnersdict[str, str] | NoneNonePer-engine runner: {"transformers": "local", "vllm": "docker"}. None = use user config or auto-detection.
imagesdict[str, str] | NoneNonePer-engine Docker image overrides: {"vllm": "ghcr.io/org/img:tag"}. None = smart default (local build then registry fallback).

Loader-populated metadata fields

These are written by the loader and study runner; you rarely set them manually:

FieldTypeDefaultDescription
skipped_configslist[dict][]Grid points that failed Pydantic validation at sweep-expansion time.
dedup_mode"resolved" | "off""resolved"Current deduplication mode. Set via ExecutionConfig.deduplicate_equivalent.
pre_run_equivalence_groupslist[dict][]Equivalence groups computed at expansion time and written to equivalence_groups.json in the results bundle.
declared_resolved_config_hasheslist[str][]Resolved config hashes parallel to the pre-expansion sweep input. Used by the harness for sidecar tagging.

Validation

StudyConfig enforces extra="forbid". Unknown top-level keys raise ValidationError.

experiments must be non-empty (Pydantic min_length=1). A StudyConfig with zero experiments cannot be constructed.

Each item in experiments is individually validated as an ExperimentConfig (see ExperimentConfig for per-experiment validation rules, including engine-section matching and engine-invariants checking).


Common patterns

Inspect expanded experiments before running

from llenergymeasure.config.loader import load_study_config
from pathlib import Path

study = load_study_config(path=Path("sweep.yaml"))
print(f"{len(study.experiments)} experiments after expansion")
for i, exp in enumerate(study.experiments, 1):
print(f" {i:>3}. {exp.task.model} / {exp.engine}")

Override output directory programmatically

study = load_study_config(path=Path("study.yaml"))
study = study.model_copy(
update={"output": study.output.model_copy(update={"results_dir": "/data/results"})}
)
result = run_study(study)

Pass a pre-built StudyConfig to run_study

from llenergymeasure import run_study, StudyConfig, ExperimentConfig
from llenergymeasure.config.models import TaskConfig, ExecutionConfig

study = StudyConfig(
experiments=[ExperimentConfig(task=TaskConfig(model="gpt2"), engine="transformers")],
study_name="quick-check",
study_execution=ExecutionConfig(n_cycles=3, experiment_order="interleave"),
)
result = run_study(study)

See also