# Jetson Deployment

> Build, bundle, and install X-Edge AI on NVIDIA Jetson devices — Orin, Xavier, and the legacy TX2 fleet.

X-Edge AI runs on NVIDIA Jetson modules (arm64 / L4T) with the same three-service
stack as x86 boxes — frontend, backend, and the Python inference service. What
changes on Jetson is the **inference image**: it must be built against the
JetPack-provided CUDA/TensorRT libraries, with an ONNX Runtime wheel that matches
your JetPack version.

Jetson deployment ships from the dedicated **`Jetson-device`** release line.
Updates flow one-way from the main product line into it, so Jetson builds always
lag the newest x86 release by design.

## Supported profiles

| Profile | Device / JetPack | Base image | ONNX Runtime | Execution provider |
|---|---|---|---|---|
| `jetson` | Orin / Xavier / Nano, JetPack 5+ (L4T r35+) | `l4t-base:r35.4.1` | JetPack wheel (per device) | TensorRT → CUDA → CPU |
| `jetson-gpu` | TX2, JetPack 4.5 (L4T r32.5) | `l4t-base:r32.5.0` | `onnxruntime-gpu 1.10.0` (cp36 wheel) | CUDA (TensorRT EP not available on JetPack 4.5) |
| `jetson-cpu` | Any arm64 Jetson | `python:3.11-slim` (arm64) | `onnxruntime` (CPU) | CPU |

Two implications of the TX2 GPU profile worth knowing before you start:

- **ONNX opset ceiling.** ONNX Runtime 1.10 supports up to **opset 15**. Export
  TX2-bound models with `--opset 15` (the bundle seeds an opset-15 model for this
  reason). Newer Jetsons running current JetPack wheels accept opset 17.
- **Frozen Python 3.6 dependency set.** JetPack 4.5 is locked to Python 3.6, so
  the `jetson-gpu` image uses a pinned legacy dependency fork. Prefer `jetson`
  (Orin/JetPack 5) for new fleets.

## Prerequisites (on the Jetson)

Images are built **natively on the device** — cross-building L4T images under
QEMU is unreliable and unsupported.

```bash
docker version              # >= 20.10
docker compose version      # Compose v2 plugin required
sudo apt-get install -y zstd openssl
df -h ~                     # keep a few GB free (TX2 eMMC is small)
docker info | grep -i nvidia   # GPU profiles: NVIDIA container runtime must be wired in
```

For GPU profiles, pick the ONNX Runtime wheel matching your JetPack version from
the [Jetson Zoo](https://elinux.org/Jetson_Zoo#ONNX_Runtime) — the x86
`onnxruntime-gpu` pip wheel does **not** work on Jetson (it links x86 CUDA libs;
"missing library" at startup is the classic symptom).

## Build → bundle → install

```mermaid
flowchart LR
  A["build-release-images.sh\n(build + tag 3 images)"] --> B["build-release-bundle.sh\n(docker save → dist/<profile>-<version>/)"]
  B --> C["install.sh on the target\n(load images → compose up → seed admin)"]
  A -. "build host: Jetson, internet once" .-> B
  C -. "target host: air-gapped OK" .-> C
```

1. **Build the images** (on the build Jetson, internet required once):

   ```bash
   # GPU profile — pass the JetPack-matched ONNX Runtime wheel
   ORT_WHEEL_URL="https://<jetson-zoo-wheel-for-your-jetpack>.whl" \
     ./scripts/build-release-images.sh <version> --profile jetson-gpu

   # CPU profile — no wheel needed
   ./scripts/build-release-images.sh <version> --profile jetson-cpu
   ```

   Before the first GPU build, de-risk the wheel and GPU access with the probe
   script: `./scripts/spike-jetson-gpu.sh` (GO/NO-GO check).

2. **Package the offline bundle:**

   ```bash
   ./scripts/build-release-bundle.sh --profile jetson-gpu --version v<version>
   ```

   The output `dist/jetson-gpu-v<version>/` contains the compressed image
   archive with an integrity manifest, a Tegra-correct compose file, the seed
   model, and the `install.sh` / `update.sh` / `uninstall.sh` runbook scripts —
   the same layout as the x86 bundle described in
   [Offline Bundle Install](/install-deploy/offline-bundle-install/).

3. **Install on the target device** (copy the bundle directory via USB/scp):

   ```bash
   cd dist/jetson-gpu-v<version>
   sudo ./install.sh                    # auto-detects arm64 and accepts jetson-* bundles
   # sudo ./install.sh --with-systemd   # optional: start on boot
   ```

   The installer verifies the archive checksum, loads images, generates a unique
   JWT secret and admin password, starts the stack, waits for health, and prints
   the dashboard URL and admin password **once** — store it immediately.

4. **Verify GPU inference is real:**

   ```bash
   docker compose -f compose/docker-compose.release.yml ps                   # 3 services healthy
   docker inspect aiboard-inference-real --format '{{.HostConfig.Runtime}}'  # expect: nvidia
   docker logs aiboard-inference-real 2>&1 | grep -iE "CUDA EP|provider"     # CUDA/TensorRT EP active
   curl -s http://localhost:5000/api/system/metrics | grep -oE '"gpu"[^}]*}' # real GPU metrics
   ```

## Tegra-specific behavior

The Jetson bundle compose differs from the x86 one on purpose:

- **`runtime: nvidia`** and root inside the inference container — required for
  GPU device access on Tegra.
- **`/sys` mount** — GPU load and temperatures come from Tegra sysfs (there is
  no `nvidia-smi` on Jetson), feeding the dashboard's GPU panel.
- **`OPENBLAS_CORETYPE`** pin — avoids an OpenBLAS illegal-instruction crash on
  Tegra cores.
- **Per-field N/A metrics** — dashboard health fields a Jetson cannot report
  show as **N/A** instead of fake zeros.

Execution-provider selection works the same as on x86:
`EXECUTION_MODE=auto` tries TensorRT → CUDA → CPU and logs a warning if it
degrades. For a hard guarantee that the box never silently falls back to CPU,
set `EXECUTION_MODE=cuda` (or `tensorrt`) plus `STRICT_EP=1` — the service then
refuses to start instead of degrading. See
[Execution provider fell back to CPU](/troubleshooting/execution-provider-fallback/).

The `jetson-gpu` / `jetson-cpu` bundles are build- and lab-verified; final
end-to-end validation of the air-gapped install on a clean TX2 is still in
progress. Keep a serial/SSH console available during your first on-device
install.

## Versioning on the Jetson line

Jetson releases are versioned by the `VERSION` file only — the Jetson line never
creates product `v*` tags. When you see version drift between the dashboard's
About panel and your bundle, compare against the bundle manifest, not Git tags.
See [Versions & Updates](/operate/versioning/).

## If something goes wrong

| Symptom | Fix |
|---|---|
| `unknown shorthand flag: 'f' in -f` | Compose v2 plugin missing — `sudo apt-get install -y docker-compose-plugin` |
| `Cannot autolaunch D-Bus` during build | Headless credential-helper issue — the build script isolates it automatically; for ad-hoc `docker` commands use an empty `DOCKER_CONFIG` dir |
| GPU container: `no CUDA-capable device` | Host is missing the NVIDIA container runtime, or the compose lacks `runtime: nvidia` (the bundle compose already sets it) |
| Inference exits with a missing-library error | Wrong ONNX Runtime wheel for your JetPack — pick the matching wheel from the Jetson Zoo and rebuild |
| `install.sh`: "profile does not match bundle" | Pass `--profile jetson-gpu` (or `jetson-cpu`) explicitly — both are valid on arm64 |
| Model rejected at upload (TX2) | Model exported with opset > 15 — re-export with `--opset 15` for ONNX Runtime 1.10 |

## Next steps

  - [Offline Bundle Install](/install-deploy/offline-bundle-install/) — The full air-gapped install runbook shared by all profiles.
  - [Hardware Setup](/install-deploy/hardware-setup/) — Execution-provider modes and network prerequisites.
  - [Preparing Models with modelctl](/configure/prepare-models/) — Export ONNX models with the right opset for your device.
  - [GPU / CUDA errors](/troubleshooting/gpu-cuda-error-500/) — Troubleshoot GPU visibility and CUDA failures.