Skip to main content

Parameter Curation

Note: This document covers engine-API-parameter introspection and Pydantic-model curation (what fields each engine accepts, type information, drift detection). For the runtime validation of parameter values (how invalid combinations are caught before engine initialisation), see parameter-discovery.md.

llem exposes engine parameters to users through hand-authored Pydantic models. This document explains how those models stay in sync with the underlying engines.

Overview​

  • Programmatic discovery introspects engine APIs and writes engines/*/schema.discovered.json (the ground truth for "what parameters does this engine accept").
  • Pydantic curation is the hand-authored set of sub-config models that expose typed, documented fields to users.
  • Drift checker flags Pydantic fields with no corresponding discovered entry.
  • LLEM_NATIVE_FIELDS is the "yes, this divergence is intentional" allowlist - it suppresses known-good exceptions so the drift checker only reports unexpected divergence.

Programmatic discovery​

scripts/engine_introspectors/ introspects each engine's public Python API (e.g. inspect.signature(vllm.LLM.__init__), inspect.signature(AutoModelForCausalLM.from_pretrained)) and writes the result to src/llenergymeasure/engines/{engine}/schema.discovered.json.

These JSON files are the ground truth for "what parameters does this engine version accept". They are stored in the repo and regenerated via the parameter-discovery pipeline when an engine version bumps (see schema-refresh.md).

Pydantic curation​

src/llenergymeasure/config/engine_configs.py contains hand-authored Pydantic models that llem exposes to users. Curation decisions:

  • Field names match native engine names. A field called quant_config maps directly to the engine kwarg quant_config. No translation layer, no llem aliases.
  • Sub-configs group related parameters. e.g. TensorRTKvCacheConfig groups all kv-cache knobs under tensorrt.kv_cache_config.*. The sub-config name matches the native engine kwarg name.
  • Types may be narrowed. A field typed str in discovery might become Literal["bfloat16", "float16", "float32"] in curation - this is intentional and allowed by the drift checker.
  • Descriptions are added. Pydantic Field(description=...) docs are user-facing; discovery has none.

Drift checker​

scripts/check_pydantic_matches_discovered.py compares the set of leaf field names in the Pydantic models against the discovered schemas and reports two kinds of drift:

KindMeaning
pydantic_onlyPydantic has a field that discovery doesn't - likely a stale field that was removed from the engine, or a kwargs-passed field invisible to signature inspection
type_mismatchBoth sides have the field but with different types (beyond intentional narrowing)

Run it locally:

python scripts/check_pydantic_matches_discovered.py

CI runs it automatically on every PR.

LLEM_NATIVE_FIELDS​

Some Pydantic fields legitimately have no discovered counterpart. Common reasons:

ReasonExample
Passed via **kwargs, invisible to inspect.signaturetransformers.dtype - from_pretrained accepts it as a kwarg alias
llem surfaces a sub-config field that the engine accepts as a flat kwarg at a different nesting leveltensorrt.quant_algo inside TensorRTQuantConfig
Beam-search or speculative-decoding params from a separate params classvllm.beam_width (from BeamSearchParams, not LLM.__init__)

These are listed in LLEM_NATIVE_FIELDS in the drift checker. Each entry suppresses one pydantic_only warning for a named (engine, field_name) pair.

When to add an entry: when the drift checker flags a pydantic_only field and you have confirmed it is intentionally in the Pydantic model but unreachable by signature-based discovery. Add a comment explaining why.

When to remove an entry: when the corresponding Pydantic field is deleted. Stale entries are harmless but misleading - remove them during the same PR that removes the field.

Never add an entry to paper over a naming divergence. If a Pydantic field is named differently from the engine kwarg, rename the field instead.

See also​