NVIDIA Snapshot reuses GPU state to shorten inference restarts NVIDIA Snapshot reuses GPU state to shorten inference restarts

A saved worker instead of another setup

An idle GPU is not immediately useful serving capacity. The model’s weights must reach its memory, and the inference engine must prepare its execution paths. Snapshot preserves both CPU and GPU process state after that preparation. A restored worker resumes the saved setup instead of repeating model loading, compilation and graph capture. NVIDIA’s November 10 technical report describes tests using B200 GPUs and three engines: vLLM, SGLang and TensorRT LLM. Its measured restores ranged from 3.6 to 42.7 seconds. Those clocks start after infrastructure provisioning, and exclude pulling the container image and setting up its runtime.

Storage changes the result

Crusoe’s separate November 10 report describes a joint test with NVIDIA using GLM-5.3 on HGX B300 hardware. Its restored worker reached readiness 31 seconds after pod creation, compared with roughly eight minutes from a cold start. The saved GPU state occupied 594 GB. Moving that artifact from network storage to local NVMe reduced readiness time from 50 seconds to 31. The GPU-state transfer itself took 15 seconds; that is one stage inside the larger readiness measurement. Crusoe tested the local storage without a warm file cache. Its result makes the storage path part of the deployment decision, alongside the model and GPU.

A response clock measures something different

Saturn Cloud’s benchmark, first published November 2 and updated November 10, measures the first successful response after worker creation. It reports six to 34 seconds for restored workers on an already-running B300 node. The comparison used downloaded weights and cached container images. Snapshot creation, machine provisioning and model downloads sit outside those startup figures. Each Qwen and GPT-OSS configuration had one paired run; GLM-5.3 had two. These are individual observations rather than averages across a large fleet. The eight-GPU GLM checkpoint occupied 564 GiB and took 41 seconds to create, a separate expense from bringing it back.

The surrounding platform still owns the schedule

The project’s documentation assigns Snapshot the capture and restore operations. Other systems decide which workers to save, when to stop them and when to request a replacement. A Kubernetes operator coordinates the work, with a privileged agent on GPU nodes and shared storage for checkpoint artifacts. That means adopting the component also involves permissions, storage management and integration with an existing serving platform. The prerequisites include Linux kernel 5.13 or newer and GPU Operator 26.3 or newer, with GPU partitioning through MIG disabled. Existing scheduling rules still decide whether a compatible machine is available when demand returns. The open repository is licensed under Apache 2.0.