Invalid Parameter Combinations

Auto-generated. Do not edit by hand: run python scripts/generate_invalid_combos_doc.py (or make docs-all).

This document lists parameter combinations that fail validation or run differently than declared. The error rules are enforced at config load time with a clear error message; the dormant rules are accepted but silently normalised by the engine. Both are derived from the live rule corpus (src/llenergymeasure/engines/<engine>/rules.yaml) plus the cross-engine ExperimentConfig validators, so this page cannot drift from what actually fires at runtime.

Config Validation Errors

These combinations are rejected at config load time with a clear error message. Rows citing a rule id come from that engine's shipped rule corpus; the rest are ExperimentConfig pydantic validators.

Engine	Invalid Combination	Reason	Resolution
all	engine section mismatch	The engine section must match the engine field (validate_engine_section_match).	Ensure the transformers:/vllm:/tensorrt: section matches the engine: field.
all	mode section mismatch	The mode namespace must match the serving_mode field: a server: section is legal only under serving_mode=server, and serving_mode=server requires a server: section with a traffic spec (validate_mode_section_match).	Ensure the server: section matches serving_mode; add a server: section (with traffic.rate) for serving_mode=server, or set serving_mode: offline.
transformers	engine=transformers and serving_mode=server	transformers server mode is a fast-follow: at the pinned version transformers serve is upstream-scoped to moderate load and exposes no first-class health endpoint, so it does not clear the server-mode stability gate (validate_transformers_server_unsupported).	Use engine=vllm or engine=tensorrt for server mode, or set serving_mode=offline for transformers.
all	serving_mode=server and server.traffic.window_requests set	Count-bound measured windows are not supported at v0.7: the server-mode measurement path (measured-span timing and the per-level stability gate) is duration-grounded (validate_server_window_supported).	Use server.traffic.window_seconds instead (it defaults to the minimum-window-duration floor when omitted).
all	passthrough_kwargs key collision	passthrough_kwargs keys must not collide with ExperimentConfig fields (validate_passthrough_kwargs_no_collision).	Set the named field directly instead of via passthrough_kwargs.
all	unknown field on the engine section wrapper	A key placed directly on the engine section (not under engine_params/sampling_params) is never forwarded to the engine (validate_engine_section_extras).	Move the key under .engine_params or .sampling_params.
transformers	attn_implementation in [flash_attention_2, flash_attention_3] and dtype=float32	attn_implementation='flash_attention_2'/'flash_attention_3' requires dtype='float16' or dtype='bfloat16'; FlashAttention does not support float32 computation (validate_transformers_flash_attn_dtype).	Set transformers.engine_params.dtype to float16 or bfloat16.
tensorrt	engine_path is set and backend != trt	engine_path loads a prebuilt compiled-TensorRT engine directory, which only the trt constructor can read; the pytorch backend (the default) would misinterpret it as a checkpoint (validate_tensorrt_engine_path_backend).	Set tensorrt.engine_params.backend to trt, or drop engine_path to build from the model checkpoint.
tensorrt	batching_type not in [STATIC, INFLIGHT]	batching_type must be one of STATIC or INFLIGHT	Adjust the field(s) so the condition no longer holds; see rule tensorrt_analyst_error_batching_type_76e16449.
tensorrt	load_format not in [auto, dummy]	load_format must be one of auto, dummy	Adjust the field(s) so the condition no longer holds; see rule tensorrt_analyst_error_load_format_835500f5.
tensorrt	orchestrator_type not in [rpc, ray]	orchestrator_type must be one of rpc or ray	Adjust the field(s) so the condition no longer holds; see rule tensorrt_analyst_error_orchestrator_type_a5b183fd.
tensorrt	tokenizer_mode not in [auto, slow]	tokenizer_mode must be one of auto, slow	Adjust the field(s) so the condition no longer holds; see rule tensorrt_analyst_error_tokenizer_mode_c342cdfb.
tensorrt	truncate_prompt_tokens < 1	truncate_prompt_tokens must be greater than or equal to 1. Otherwise, an error is raised.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_analyst_error_truncate_prompt_tokens_5e1c2d17.
tensorrt	backend != trt and fast_build == True	fast_build=True requires the trt backend; the pytorch backend (TorchLlmArgs) has no fast_build field. Set backend=trt or remove fast_build.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_backend_fast_build_requires_trt.
tensorrt	backend != trt and quant_config is set	quant_config requires the trt backend; the pytorch backend (TorchLlmArgs) has no quant_config field. Set backend=trt or remove quant_config.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_backend_quant_config_requires_trt.
tensorrt	acceptance_length_threshold < 0	speculative_config.acceptance_length_threshold must be >= 0, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_decodingconfig_raises_acceptance_length_threshold_lt_0.
tensorrt	acceptance_window < 0	speculative_config.acceptance_window must be >= 0 (0 disables), got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_decodingconfig_raises_acceptance_window_lt_0.
tensorrt	max_input_len < 1	max_input_len must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_engineparams_raises_max_input_len_lt_1.
tensorrt	max_num_tokens < 1	max_num_tokens must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_engineparams_raises_max_num_tokens_lt_1.
tensorrt	max_seq_len < 1	max_seq_len must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_engineparams_raises_max_seq_len_lt_1.
tensorrt	pipeline_parallel_size < 1	pipeline_parallel_size must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_engineparams_raises_pipeline_parallel_size_lt_1.
tensorrt	tensor_parallel_size < 1	tensor_parallel_size must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_engineparams_raises_tensor_parallel_size_lt_1.
tensorrt	free_gpu_memory_fraction > 1	kv_cache_config.free_gpu_memory_fraction must be a float between 0 and 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_kvcacheconfig_raises_free_gpu_memory_fraction_gt_1.
tensorrt	free_gpu_memory_fraction < 0	kv_cache_config.free_gpu_memory_fraction must be a float between 0 and 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_kvcacheconfig_raises_free_gpu_memory_fraction_lt_0.
tensorrt	max_gpu_total_bytes < 0	kv_cache_config.max_gpu_total_bytes must be non-negative, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_kvcacheconfig_raises_max_gpu_total_bytes_lt_0.
tensorrt	max_ngram_size <= 0	speculative_config.max_ngram_size must be positive, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_lookaheadconfig_raises_max_ngram_size_le_0.
tensorrt	max_verification_set_size <= 0	speculative_config.max_verification_set_size must be positive, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_lookaheadconfig_raises_max_verification_set_size_le_0.
tensorrt	max_window_size <= 0	speculative_config.max_window_size must be positive, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_lookaheadconfig_raises_max_window_size_le_0.
tensorrt	max_batch_size < 0	engine_params.max_batch_size must be non-negative, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_raises_max_batch_size_lt_0.
tensorrt	min_p > 1.0	min_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_samplingparams_raises_min_p_gt_1p0.
tensorrt	min_p < 0.0	min_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_samplingparams_raises_min_p_lt_0p0.
tensorrt	n < 1	n must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_samplingparams_raises_n_lt_1.
tensorrt	repetition_penalty <= 0.0	repetition_penalty must be >= 0.0, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_samplingparams_raises_repetition_penalty_le_0p0.
tensorrt	temperature < 0.0	temperature must be >= 0.0, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_samplingparams_raises_temperature_lt_0p0.
tensorrt	top_k < 0	top_k must be >= 0, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_samplingparams_raises_top_k_lt_0.
tensorrt	top_p > 1.0	top_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_samplingparams_raises_top_p_gt_1p0.
tensorrt	top_p < 0.0	top_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule tensorrt_samplingparams_raises_top_p_lt_0p0.
transformers	pad_token_id < 0	pad_token_id should be positive.	Adjust the field(s) so the condition no longer holds; see rule transformers_analyst_error_pad_token_id_90669ba0.
transformers	load_in_4bit=True and load_in_8bit=True	load_in_4bit and load_in_8bit are both True, but only one can be used at the same time	Adjust the field(s) so the condition no longer holds; see rule transformers_bnb_load_in_4bit_xor_load_in_8bit.
transformers	cache_implementation is set and cache_implementation not in [static, offloaded_static, sliding_window, hybrid, hybrid_chunked, offloaded_hybrid, offloaded_hybrid_chunked, dynamic, dynamic_full, offloaded, quantized, paged]	Invalid cache_implementation (nonsense). Choose one of: ('static', 'offloaded_static', 'sliding_window', 'hybrid', 'hybrid_chunked', 'offloaded_hybrid', 'offloaded_hybrid_chunked', 'dynamic', 'dynamic_full', 'offloaded', 'quantized', 'paged')	Adjust the field(s) so the condition no longer holds; see rule transformers_cache_choice_cache_implementation_not_in_allowlist.
transformers	compile_config is set and type(compile_config) is not CompileConfig	You provided compile_config as an instance of <class 'str'>, but it must be an instance of CompileConfig.	Adjust the field(s) so the condition no longer holds; see rule transformers_compile_config_type_compile_config_type_not_in_CompileConfig.
transformers	early_stopping is set and early_stopping not in [False, True, never]	early_stopping must be a boolean or 'never', but is sometimes.	Adjust the field(s) so the condition no longer holds; see rule transformers_early_stopping_type_early_stopping_not_in_allowlist.
transformers	prompt_lookup_num_tokens < 1	prompt_lookup_num_tokens must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_engineparams_raises_prompt_lookup_num_tokens_lt_1.
transformers	type(llm_int8_threshold) is not float	llm_int8_threshold must be a float	Adjust the field(s) so the condition no longer holds; see rule transformers_extractor_error_llm_int8_threshold_faf7fb57.
transformers	num_return_sequences is set and num_return_sequences > 1 and do_sample is unset and num_beams is unset	Greedy methods (do_sample != True) without beam search do not support num_return_sequences different than 1 (got {num_return_sequences}).	Adjust the field(s) so the condition no longer holds; see rule transformers_extractor_error_num_return_sequences_do_sample_75f96548.
transformers	num_beams=1 and num_return_sequences > 1 and do_sample=False	Greedy methods (do_sample != True) without beam search do not support num_return_sequences different than 1 (got 2).	Adjust the field(s) so the condition no longer holds; see rule transformers_num_return_vs_beams_do_sample_eq_false_and_num_beams_eq_1.
transformers	num_beams < @transformers.sampling_params.num_return_sequences	num_return_sequences (4) has to be smaller or equal to num_beams (2).	Adjust the field(s) so the condition no longer holds; see rule transformers_num_return_vs_beams_num_beams_lt_num_return_sequences.
transformers	num_return_sequences > @transformers.engine_params.num_beams	num_return_sequences has to be smaller or equal to num_beams.	Adjust the field(s) so the condition no longer holds; see rule transformers_num_return_vs_beams_num_return_sequences_gt_num_beams.
transformers	max_new_tokens <= 0	max_new_tokens must be greater than 0, but is -1.	Adjust the field(s) so the condition no longer holds; see rule transformers_output_token_ids_max_new_tokens_le_zero.
transformers	type(bnb_4bit_quant_type) is not str	bnb_4bit_quant_type must be a string	Adjust the field(s) so the condition no longer holds; see rule transformers_raises_bnb_4bit_quant_type_not_type_str.
transformers	type(bnb_4bit_use_double_quant) is not bool	bnb_4bit_use_double_quant must be a boolean	Adjust the field(s) so the condition no longer holds; see rule transformers_raises_bnb_4bit_use_double_quant_not_type_bool.
transformers	early_stopping not in [None, True, False, never]	early_stopping must be a boolean or 'never', but is {early_stopping}.	Adjust the field(s) so the condition no longer holds; see rule transformers_raises_early_stopping_not_in_set.
transformers	type(load_in_4bit) is not bool	load_in_4bit must be a boolean	Adjust the field(s) so the condition no longer holds; see rule transformers_raises_load_in_4bit_not_type_bool.
transformers	type(load_in_8bit) is not bool	load_in_8bit must be a boolean	Adjust the field(s) so the condition no longer holds; see rule transformers_raises_load_in_8bit_not_type_bool.
transformers	epsilon_cutoff >= 1.0	epsilon_cutoff has to be a float > 0 and < 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_epsilon_cutoff_ge_1p0.
transformers	epsilon_cutoff <= 0.0	epsilon_cutoff has to be a float > 0 and < 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_epsilon_cutoff_le_0p0.
transformers	eta_cutoff >= 1.0	eta_cutoff has to be a float > 0 and < 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_eta_cutoff_ge_1p0.
transformers	eta_cutoff <= 0.0	eta_cutoff has to be a float > 0 and < 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_eta_cutoff_le_0p0.
transformers	max_matching_ngram_size < 1	max_matching_ngram_size must be > 0, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_max_matching_ngram_size_lt_1.
transformers	min_length < 0	min_length has to be a non-negative integer, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_min_length_lt_0.
transformers	min_p > 1.0	min_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_min_p_gt_1p0.
transformers	min_p < 0.0	min_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_min_p_lt_0p0.
transformers	repetition_penalty <= 0.0	repetition_penalty must be >= 0.0, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_repetition_penalty_le_0p0.
transformers	temperature < 0.0	temperature must be >= 0.0, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_temperature_lt_0p0.
transformers	top_h > 1.0	top_h must be in the range (0, 1], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_top_h_gt_1p0.
transformers	top_h <= 0.0	top_h must be in the range (0, 1], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_top_h_le_0p0.
transformers	top_k < 0	top_k must be >= 0, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_top_k_lt_0.
transformers	top_p > 1.0	top_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_top_p_gt_1p0.
transformers	top_p < 0.0	top_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_top_p_lt_0p0.
transformers	typical_p >= 1.0	typical_p has to be a float > 0 and < 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_typical_p_ge_1p0.
transformers	typical_p <= 0.0	typical_p has to be a float > 0 and < 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_typical_p_le_0p0.
transformers	watermarking_config is set and type(watermarking_config) is not WatermarkingConfig, SynthIDTextWatermarkingConfig	watermarking_config must be a WatermarkingConfig or SynthIDTextWatermarkingConfig instance.	Adjust the field(s) so the condition no longer holds; see rule transformers_samplingparams_raises_watermarking_config_type_invalid.
vllm	cudagraph_mm_encoder=True and encoder_cudagraph_max_images_per_batch < 0	encoder_cudagraph_max_images_per_batch must be non-negative (0 = auto-infer), got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_compilationconfig_raises_encoder_cudagraph_max_images_per_batch_lt_0.
vllm	kv_cache_memory_bytes < 1	kv_cache_memory_bytes must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_engineparams_raises_kv_cache_memory_bytes_lt_1.
vllm	pipeline_parallel_size < 1	pipeline_parallel_size must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_engineparams_raises_pipeline_parallel_size_lt_1.
vllm	tensor_parallel_size < 1	tensor_parallel_size must be >= 1, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_engineparams_raises_tensor_parallel_size_lt_1.
vllm	data_parallel_size <= 1 and data_parallel_external_lb is set	data_parallel_external_lb can only be set when data_parallel_size > 1	Adjust the field(s) so the condition no longer holds; see rule vllm_parallelconfig_raises_data_parallel_external_lb_set_true.
vllm	data_parallel_size_local > @data_parallel_size	data_parallel_size_local ({data_parallel_size_local}) must be <= data_parallel_size ({data_parallel_size})	Adjust the field(s) so the condition no longer holds; see rule vllm_parallelconfig_raises_data_parallel_size_local_gt_ref_data_parallel_size.
vllm	frequency_penalty > 2.0	frequency_penalty must be in [-2.0..2.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_frequency_penalty_gt_2p0.
vllm	frequency_penalty < -2.0	frequency_penalty must be in [-2.0..2.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_frequency_penalty_lt_neg2p0.
vllm	min_p > 1.0	min_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_min_p_gt_1p0.
vllm	min_p < 0.0	min_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_min_p_lt_0p0.
vllm	max_tokens is set and min_tokens > @max_tokens	min_tokens must be less than or equal to max_tokens={max_tokens}, got {min_tokens}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_min_tokens_gt_ref_max_tokens.
vllm	min_tokens < 0	min_tokens must be greater than or equal to 0, got {min_tokens}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_min_tokens_lt_0.
vllm	n < 1	n must be at least 1, got {n}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_n_lt_1.
vllm	type(n) is not int	n must be an int, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_n_not_type_int.
vllm	presence_penalty > 2.0	presence_penalty must be in [-2.0..2.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_presence_penalty_gt_2p0.
vllm	presence_penalty < -2.0	presence_penalty must be in [-2.0..2.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_presence_penalty_lt_neg2p0.
vllm	repetition_penalty <= 0.0	repetition_penalty must be greater than zero, got {repetition_penalty}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_repetition_penalty_le_0p0.
vllm	temperature < 0.0	temperature must be non-negative, got {temperature}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_temperature_lt_0p0.
vllm	top_k < -1	top_k must be 0 (disable), or at least 1, got {top_k}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_top_k_lt_neg1.
vllm	type(top_k) is not int	top_k must be an integer, got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_top_k_not_type_int.
vllm	top_p > 1.0	top_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_top_p_gt_1p0.
vllm	top_p <= 0.0	top_p must be in [0.0..1.0], got {declared_value}.	Adjust the field(s) so the condition no longer holds; see rule vllm_samplingparams_raises_top_p_le_0p0.
vllm	max_num_partial_prefills > 1 and long_prefill_token_threshold > @max_model_len	long_prefill_token_threshold ({long_prefill_token_threshold}) cannot be greater than the max_model_len ({max_model_len}).	Adjust the field(s) so the condition no longer holds; see rule vllm_schedulerconfig_raises_long_prefill_token_threshold_gt_ref_max_model_len.
vllm	max_long_partial_prefills > @max_num_partial_prefills	self.max_long_partial_prefills={max_long_partial_prefills} must be less than or equal to self.max_num_partial_prefills={max_num_partial_prefills}.	Adjust the field(s) so the condition no longer holds; see rule vllm_schedulerconfig_raises_max_long_partial_prefills_gt_ref_max_num_partial_prefills.
vllm	max_num_batched_tokens < @max_model_len and enable_chunked_prefill is unset	max_num_batched_tokens ({max_num_batched_tokens}) is smaller than max_model_len ({max_model_len}). This effectively limits the maximum sequence length to max_num_batched_tokens and makes vLLM reject longer sequences. Please increase max_num_batched_tokens or decrease max_model_len.	Adjust the field(s) so the condition no longer holds; see rule vllm_schedulerconfig_raises_max_num_batched_tokens_lt_ref_max_model_len.
vllm	max_num_batched_tokens < @max_num_seqs	max_num_batched_tokens ({max_num_batched_tokens}) must be greater than or equal to max_num_seqs ({max_num_seqs}).	Adjust the field(s) so the condition no longer holds; see rule vllm_schedulerconfig_raises_max_num_batched_tokens_lt_ref_max_num_seqs.

Dormant Parameters

These combinations pass validation, but the engine silently normalises or ignores the declared field: the declared value is not the effective value. The study planner deduplicates configs that differ only in a dormant field, so the GPU runs such a cell once. Normalised fields names the paths the engine drives back to their default.

Engine	Combination	Effect	Normalised fields
transformers	use_cache=False	You have not set use_cache to True, but cache_implementation is set to static.cache_implementation will have no effect.	-
transformers	do_sample=False and epsilon_cutoff is set	do_sample is not set to True. However, epsilon_cutoff is set to {declared_value} -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset epsilon_cutoff.	-
transformers	do_sample=False and eta_cutoff is set	do_sample is not set to True. However, eta_cutoff is set to {declared_value} -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset eta_cutoff.	-
transformers	do_sample=False and min_p is set	do_sample is not set to True. However, min_p is set to {declared_value} -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset min_p.	-
transformers	do_sample=False and temperature is set	do_sample is not set to True. However, temperature is set to {declared_value} -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset temperature.	-
transformers	do_sample=False and top_h is set	do_sample is not set to True. However, top_h is set to {declared_value} -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset top_h.	-
transformers	do_sample=False and top_k is set	do_sample is not set to True. However, top_k is set to {declared_value} -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset top_k.	-
transformers	do_sample=False and top_p is set	do_sample is not set to True. However, top_p is set to {declared_value} -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset top_p.	-
transformers	do_sample=False and typical_p is set	do_sample is not set to True. However, typical_p is set to {declared_value} -- this flag is only used in sample-based generation modes. You should set do_sample=True or unset typical_p.	-
transformers	return_dict_in_generate=False and output_attentions is set	return_dict_in_generate is NOT set to True, but output_attentions is. When return_dict_in_generate is not True, output_attentions is ignored.	-
transformers	return_dict_in_generate=False and output_hidden_states is set	return_dict_in_generate is NOT set to True, but output_hidden_states is. When return_dict_in_generate is not True, output_hidden_states is ignored.	-
transformers	return_dict_in_generate=False and output_logits is set	return_dict_in_generate is NOT set to True, but output_logits is. When return_dict_in_generate is not True, output_logits is ignored.	-
transformers	return_dict_in_generate=False and output_scores is set	return_dict_in_generate is NOT set to True, but output_scores is. When return_dict_in_generate is not True, output_scores is ignored.	-
transformers	num_beams=1 and early_stopping is set	num_beams is set to 1. However, early_stopping is set to {declared_value} -- this flag is only used in beam-based generation modes. You should set num_beams>1 or unset early_stopping.	-
transformers	num_beams=1 and length_penalty is set	num_beams is set to 1. However, length_penalty is set to {declared_value} -- this flag is only used in beam-based generation modes. You should set num_beams>1 or unset length_penalty.	-
vllm	all2all_backend in [pplx, naive]	Enforced by rule vllm_parallelconfig_dormant_all2all_backend_in.	all2all_backend
vllm	distributed_executor_backend=external_launcher and data_parallel_rank is set	Enforced by rule vllm_parallelconfig_dormant_data_parallel_rank_set_true.	data_parallel_rank
vllm	seed=-1	Enforced by rule vllm_samplingparams_dormant_seed_eq_neg1.	seed

Engine Capability Matrix

Feature	Transformers	vLLM	TensorRT
Tensor Parallel	Yes	Yes	Yes
BitsAndBytes (4-bit)	Yes	No	No
BitsAndBytes (8-bit)	Yes	No	No
Prefix Caching	No	Yes	No
torch.compile	Yes	No	No
Speculative Decoding	Yes	Yes	No
Static KV Cache	Yes	No	No

Notes:

• vLLM supports 4-bit via AWQ/GPTQ quantized models, not bitsandbytes

Recommended Configurations by Use Case

Memory-Constrained (Consumer GPU)

engine: transformers
transformers:
  engine_params:
    load_in_4bit: true
    bnb_4bit_quant_type: nf4

High Throughput (Production)

engine: vllm
vllm:
  engine_params:
    gpu_memory_utilization: 0.9
    enable_prefix_caching: true

Maximum Performance (Ampere+)

engine: tensorrt
tensorrt:
  engine_params:
    dtype: float16
    quant_config:
      quant_algo: FP8  # Hopper only