modelctl Best Practices

Prescriptive guidance for getting a model from a workstation onto an air-gapped edge box reliably. For the step-by-step commands, see Preparing Models with modelctl — this page is the why and the do/don’t on top of that workflow.

Pick the smallest install profile

modelctl keeps the heavy ML frameworks optional. The core install runs validate and package with no PyTorch or TensorFlow — so an operator can verify a bundle on a lean machine. Add only the extra the task needs.

Validate / package

pip install modelctl            # onnx + onnxruntime + numpy only

Convert

pip install 'modelctl[pytorch]'      # or [tensorflow] / [all]

Quantize

pip install 'modelctl[quantize]'     # pulls ml_dtypes for INT8

quantize and tensorflow are mutually exclusive

The [quantize] extra needs a newer ml_dtypes than the [tensorflow]/[all] extra allows (TensorFlow caps ml-dtypes<0.4). Install one profile or the other — convert with TensorFlow in one environment, quantize in another.

Hand over a self-contained source

Give the convert box a model it can load without your training code:

• PyTorch → TorchScript .pt (torch.jit.save). A bare state_dict is rejected (no architecture); a pickled nn.Module is a best-effort fallback only.
• TensorFlow → SavedModel directory (the well-supported path; .keras/.h5 also work).

Why: the workstation that runs convert then needs no model class definition — the export is reproducible from the artifact alone.

Always pin --target-ort to the device

Opset support is checked against the edge ONNX Runtime, not your workstation’s. Pin the device’s version so a too-new export fails on your desk, not on the floor:

modelctl validate --model model.onnx --target-ort 1.18

A model exported at an opset the device’s runtime can’t execute is the single most common “works here, fails there” trap. Pinning closes it before the bundle ships.

Treat validate as a release gate

Validation is fail-soft — it runs every check and reports each, rather than stopping at the first problem. Block shipping on a non-zero exit. The full set:

Check	What it asserts
ONNX schema valid	onnx.checker accepts the graph
Opset supported	opset ≤ the edge ORT’s max
Single input tensor	engine feeds exactly one input, positionally
Input dtype float32	engine feeds float32
Input rank	matches the declared shape (when given)
ONNX Runtime load	runtime instantiates the session
Test inference	a zero-filled forward pass succeeds

Map exit codes in your tooling:

Exit	Meaning
0	all checks passed
1	a validation check failed
3	a converter framework (torch/TF) is not installed
5	post-conversion validation failed
7	quantization failed (e.g. ml_dtypes missing)

Quantize deliberately, then re-validate

Dynamic INT8 (modelctl quantize, or --quantize on convert/package) trades accuracy for size and speed. Two rules:

1. Re-validate the quantized model and confirm prediction quality on representative data before shipping — quantization can shift outputs.
2. Don’t assume it shrinks. On very small models the INT8 scale/zero-point nodes can outweigh the savings; the win shows up on real-sized models.

Quantize needs the extra

quantize / package --quantize require pip install 'modelctl[quantize]'. Without it you get a clear exit 7 naming the missing module — install it and re-run.

Ship reproducible, verifiable bundles

Pin the timestamp so rebuilds are byte-identical — essential for audit and change review onto an air-gapped box:

modelctl package --model model.onnx --name pump-anomaly-v1 --timestamp 1700000000
# or: export SOURCE_DATE_EPOCH=1700000000

Then verify integrity on the device before activating:

cd pump-anomaly-v1 && sha256sum -c checksums.sha256

The checksums.sha256 covers model.onnx, metadata.json, and input_schema.json — a tamper or a partial copy is caught at the edge, not at inference time.

Respect the edge input contract

The runtime feeds one float32 tensor, by position. A multi-input or non-float32 model loads but dies at inference on the box. validate catches this on the workstation — design the exported model to a single float32 input, and keep its window size and feature count matched to the datasource you pair it with (see Deploying Models).

Wire validate into CI

Run validate on every model build and gate the artifact on exit 0. The check needs no torch/TF, so the job stays lean:

pip install modelctl
modelctl validate --model build/model.onnx --target-ort 1.18 || exit 1

Catch a bad export in the pipeline instead of discovering it on a device you can’t reach.

Next steps

Preparing Models (how-to) The convert → validate → quantize → package commands.

Deploy the model Upload the ONNX, pair it with a datasource, activate.

Operating Best Practices Run the box healthily in production.