Design of Experiments

mixle.doe covers the design and analysis loop around expensive black-box functions. It is useful when you cannot evaluate every input, labels are expensive, simulation is slow, or you need to quantify how inputs drive model outputs.

The package is organized around four phases:

1. generate an initial design;

2. fit or query a surrogate;

3. choose follow-up points by acquisition, information gain, or active learning;

4. analyze sensitivity, uncertainty propagation, or calibration.

Design Generators

Design generators return NumPy arrays scaled to supplied bounds.

from mixle.doe import latin_hypercube, sobol_design

bounds = [(0.0, 1.0), (-2.0, 2.0)]

x_lhs = latin_hypercube(bounds, n=16, seed=0)
x_sobol = sobol_design(bounds, n=32)

Available generators include:

• random designs;

• Latin hypercube and maximin Latin hypercube;

• Sobol and Halton sequences;

• MaxPro designs;

• full factorial, fractional factorial, and Plackett-Burman designs;

• central composite and Box-Behnken response-surface designs;

• simplex lattice and simplex centroid mixture designs;

• optimal designs under D, A, I, G, E, and c criteria.

Design Diagnostics

Use diagnostics before spending an expensive run budget:

from mixle.doe import design_diagnostics
from mixle.doe.optimal import polynomial_features

report = design_diagnostics(x_lhs, polynomial_features(degree=1))
print(report)

Diagnostics help compare coverage, spacing, and projection behavior across candidate designs.

Run diagnostics before evaluating the expensive function. Once budget has been spent, a poor design cannot be repaired without changing the evidence trail.

Bayesian Optimization

The Bayesian optimization layer provides standard single-point acquisitions and batch/high-dimensional variants.

from mixle.doe import minimize

def objective(x):
    return expensive_simulator(x)

result = minimize(
    objective,
    bounds=[(0.0, 1.0), (-2.0, 2.0)],
    n_init=8,
    n_iter=24,
    seed=0,
)

print(result.x_best, result.y_best)

Acquisition functions include expected improvement, log expected improvement, probability of improvement, upper confidence bound, Thompson sampling, and knowledge gradient. Advanced routes include Monte-Carlo q-EI, local penalization, max-value entropy search, trust-region BO, constrained BO, multi-objective BO, and multi-fidelity BO.

Record the acquisition, surrogate, seed, bounds, and evaluated points. Bayesian optimization evidence is a path, not only the final best value.

Active Learning

Active learning chooses points that most improve a surrogate or parameter estimate.

from mixle.doe import active_learning_design

design = active_learning_design(
    initial_x,
    initial_y,
    bounds=[(0.0, 1.0), (-2.0, 2.0)],
    n_iter=10,
)

Lower-level scoring functions include alm_scores, alc_scores, expected_information_gain_linear, and expected_information_gain_nmc.

Active-learning batches should be evaluated on held-out or downstream task metrics. A high acquisition score means the point is informative under the surrogate, not that the resulting model improved.

Distillation and Cross-Modal Training Designs

mixle.doe.distillation treats teacher calls, human labels, and paired cross-modal records as expensive experiments. The selectors work from a fixed candidate pool and return the indices worth spending budget on.

Candidate-Pool Contract

A distillation design is only meaningful relative to the pool it selected from. Keep the following fields with the design artifact:

• a stable pool identifier or data fingerprint;

• the row order used by the selector;

• exclusion rules applied before selection;

• task labels, modality labels, costs, preferences, and uncertainty vectors;

• random seed and scoring weights; and

• the selected indices plus candidate and selected scores.

Do not treat selected indices as portable row identifiers unless the pool fingerprint and filtering rules travel with them. If a later run changes the pool, re-run the selector rather than reusing indices from a different candidate universe.

Use distillation_design for one pool:

import numpy as np

from mixle.doe import distillation_design

embeddings = np.asarray(pool_embeddings)
design = distillation_design(
    embeddings,
    n=32,
    task_labels=task_names,
    modalities=modality_tags,
    uncertainty=student_uncertainty,
    cost=teacher_cost,
    task_coverage_weight=2.0,
    modality_coverage_weight=1.5,
    seed=0,
)

selected_examples = [pool[i] for i in design.indices]

The score balances uncertainty, diversity in feature space, task coverage, modality coverage, preference, and cost. task_labels and modalities may be one tag per row or several tags per row.

Keep candidate-pool identity and exclusion rules with the design. A selector can only choose from the pool it was given, so missing or filtered candidates are part of the evidence.

Use multitask_distillation_design when the call site should read as a multi-task selector:

from mixle.doe import multitask_distillation_design

batch = multitask_distillation_design(
    embeddings,
    24,
    task_labels=[["caption"], ["retrieval", "caption"], ["qa"]],
    uncertainty=per_task_uncertainty,
    task_weights={"caption": 2.0, "retrieval": 1.0, "qa": 1.0},
    seed=1,
)

The returned DistillationDesign includes task_counts, candidate scores, selected scores, and metadata describing the target coverage.

Use cross_modal_distillation_design when rows may have several modality views and the training batch should contain paired or aligned examples:

from mixle.doe import cross_modal_distillation_design

paired = cross_modal_distillation_design(
    {
        "text": text_embeddings,
        "image": image_embeddings,
        "signal": signal_features,
    },
    n=20,
    task_labels=task_names,
    required_modalities=["text", "image"],
    min_modalities=2,
    alignment_weight=2.0,
    seed=2,
)

Rows with non-finite coordinates in one modality are treated as missing for that modality. Eligibility then follows min_modalities and required_modalities. Non-finite uncertainty, preference, cost, or coverage weights are rejected because those vectors define the ranking objective.

Record missing-modality counts and rejected ranking vectors. Cross-modal design quality depends on which rows were eligible, not only on which rows were selected.

Task and Modality Coverage

Task and modality coverage terms are batch-shaping constraints, not guarantees that a teacher is correct or a student will improve. Use them to make expensive labels more representative, then measure the downstream student on held-out task metrics.

For multi-task distillation:

• define task tags before looking at the selector output;

• keep task weights in the artifact when some tasks are deliberately more important;

• inspect task_counts for rare-task starvation; and

• separate teacher uncertainty from task coverage in the run report.

For cross-modal training:

• record which modalities were required and which were optional;

• distinguish missing modality views from non-finite ranking vectors;

• validate pair alignment before selection; and

• report held-out performance by modality availability pattern.

Interpreting Distillation Designs

All distillation selectors return a DistillationDesign. Treat it as part of the experiment record, not just as a list of row numbers:

indices

The candidate-pool indices selected for labeling, teacher calls, or paired training. These indices only make sense with the exact pool and filtering rules used for the run.

scores

The final sequential merit values for the selected rows. Use them for audit and debugging, not as calibrated probabilities.

candidate_scores

The base uncertainty/preference contribution before diversity, coverage, and cost are applied. This helps separate “the student was uncertain” from “the batch needed a task or modality.”

task_counts and modality_counts

The realized coverage of the selected batch.

metadata

Target coverage, eligible row count, and the scoring weights used by the selector.

For ordinary distillation_design, non-finite feature coordinates are filled by a column mean only inside the standardized design geometry so that a partly missing embedding dimension does not crash the selector. The original candidate pool is not modified. For cross_modal_distillation_design, non-finite coordinates mark that modality as missing for that row; eligibility then follows the required-modality and minimum-modality settings. Ranking vectors such as uncertainty, preference, and cost must be finite because they directly define the optimization objective.

Choosing a Selector

Use the selector that matches the budget question:

• use distillation_design when every row is one candidate and task or modality tags are optional coverage metadata;

• use multitask_distillation_design when the call site should make the multi-task intent explicit, especially with per-task uncertainty or task_weights;

• use cross_modal_distillation_design when each row can contain several aligned views and the batch must satisfy modality-presence constraints.

If a pool includes both high-risk and low-risk tasks, keep the task tags and weights in the run artifact. Without those fields, a later reader cannot tell whether the design under-sampled a rare task or whether that task was never in scope.

Sensitivity Analysis

Sensitivity tools quantify which inputs matter.

from mixle.doe import morris_screening, sobol_indices

morris = morris_screening(model_fn, bounds, n_trajectories=20, seed=0)
sobol = sobol_indices(model_fn, bounds, n=1024, seed=0)

Use Morris screening for a cheaper qualitative pass and Sobol indices when you need variance-based main and interaction effects.

Sensitivity analysis should name the input distribution and parameter bounds being analyzed. Indices are not transferable when the operating region changes.

Uncertainty Propagation and Calibration

propagate and unscented_transform push input uncertainty through a model. calibrate implements Kennedy-O’Hagan style calibration to field data.

from mixle.doe import propagate, unscented_transform

propagated = propagate(model_fn, input_distribution, n=1000, seed=0)
approx = unscented_transform(model_fn, mean, cov)

How DOE Connects to Task Distillation

mixle.task uses the same design philosophy for label acquisition. Active distillation treats teacher calls as an expensive experiment and spends the label budget on informative examples. Use Task Distillation for the task-facing workflow. mixle.task owns the teacher/student artifact lifecycle. mixle.doe owns pool selection. A common loop is:

1. embed or featurize an unlabeled pool;

2. use distillation_design or cross_modal_distillation_design to choose the next teacher batch;

3. call the teacher only for the selected rows;

4. train, calibrate, and profile the student with Task Distillation;

5. feed escalations, disagreement cases, failed capability probes, or under-covered tasks back as reference_features or higher uncertainty in the next design.

This keeps distillation budgets auditable. The design result records which coverage targets were requested, how many tasks and modalities were selected, and what score drove each picked row.

Release Evidence

For DOE workflows, preserve:

• bounds, constraints, random seed, and initial design;

• design diagnostics before expensive evaluations;

• surrogate and acquisition settings for sequential design;

• evaluated points, failed evaluations, and timeout policy;

• candidate-pool identity, missing-modality counts, and rejected rows for distillation selectors;

• sensitivity or propagation assumptions; and

• the downstream metric that consumed the design.

API Map

Area	Key imports
Space-filling designs	latin_hypercube, sobol_design, halton_design, maxpro_design
Classical designs	full_factorial, fractional_factorial, plackett_burman
Response-surface designs	central_composite, box_behnken, response_surface
Mixture designs	simplex_lattice, simplex_centroid, to_pseudocomponents
Optimal design	optimal_design, available_criteria, d_criterion
Bayesian optimization	minimize, propose_next, propose_batch, BayesianOptimizer
Advanced BO	turbo_minimize, constrained_minimize, multi_minimize
Active learning	active_learning_design, propose_active_learning
Analysis	sobol_indices, morris_screening, propagate, calibrate

Detailed API Inventory

Area	Imports
Bounds and random designs	Bounds, random_design, maximin_latin_hypercube
Factorial analysis	factorial_effects, FactorialEffects, polynomial_features
Response surfaces	ResponseSurface, OptimizationResult
Acquisition functions	expected_improvement, knowledge_gradient, propose_knowledge_gradient, log_expected_improvement, probability_of_improvement, upper_confidence_bound, thompson_sampling
Acquisition registry	register_acquisition, available_acquisitions
Optimal-design criteria	a_criterion, i_criterion, g_criterion, e_criterion, c_criterion, register_criterion
Constrained BO	ConstrainedBayesOptResult, probability_of_feasibility, propose_next_constrained
Multi-objective and batch BO	MultiObjectiveResult, pareto_mask, monte_carlo_qei, propose_qei_batch, propose_local_penalization
Entropy and trust-region BO	BayesOptResult, max_value_entropy_search, sample_max_values, propose_mes, TrustRegion
Multi-fidelity and sensitivity	multi_fidelity_minimize, fast_indices, dgsm
Propagation and calibration	register_propagator, KOCalibration