Introduction

Nix deployments can be very bandwidth-intensive, and in certain deployments such as spacecraft or other very remote systems this can become a major hurdle.

This is the problem DeltaNAR aims to solve.

By computing the delta between the desired deployment state & what already exists in the Nix store on the host we can drastically reduce the bandwidth required to push update closures.

Installation

For closure size reasons DeltaNAR is distributed as 2 separate Nix packages:

• The packing program

This has a relatively larger set of dependencies & is not optimised for closure size.

• The unpacking program

Optimised for closure size & has as small of a dependency set as possible.

Flakes

{
  description = "DeltaNAR usage";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    deltanar.url = "github:nixos/adisbladis/deltanar";
    deltanar.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs =
    {
      self,
      nixpkgs,
    }:
    {
      devShells = forAllSystems (system: let
        pkgs = nixpkgs.legacyPackages.${system};
      in {
        default = pkgs.mkShell {
          packages = [
            deltanar.packages.${system}.pack
            deltanar.packages.${system}.unpack
          ];
        };
      });
    };
}

Classic Nix

You can just as easily use deltanar without using Flakes:

let
  pkgs = import <nixpkgs> { };
  inherit (pkgs) lib;

  deltanar = pkgs.callPackage (builtins.fetchGit {
    url = "https://github.com/adisbladis/deltanar.git";
  }) { };
in
  deltanar.pack

Getting started

This tutorial shows how to:

• Set up prerequisites
• Create a file containing the delta between what’s on the target host & deployment closure.
• Unpack file into a binary cache
• Populate the local Nix store

These steps apply to a host called spacecraft.

Creating gcroots

To calculate a diff DeltaNAR needs to know what is already in the store of the system being deployed to.

This is achieved by using a gcroots[1] mechanism mimicking that of Nix, with an additional level of structure imposed: There is one gcroots child directory per host.

Tip

It’s a good idea to symlink the DeltaNAR directory into /nix/var/nix/gcroots/ so the deployment host doesn’t garbage collect closures it requires for delta computation.

Steps

First, create a gcroot directory for host spacecraft:

• mkdir -p gcroots/spacecraft

Symlink an already deployed NixOS generation into the gcroots directory:

• ln -s /nix/store/5vg80fas99lkn1a5i2bnwgwd3ia3i82m-nixos-system-nixos-26.05pre-git gcroots/spacecraft

Note

DeltaNAR doesn’t contain a mechanism for managing gcroots. This needs to be done either manually or through custom scripting.

Packing

• dnar-pack --gcroots ./gcroots --host spacecraft --path /nix/store/7mdg60drrnh0wq1j8hmmbhll47czm107-nixos-system-nixos-26.05pre-git

This will create delta.dnar in the current working directory.

Unpacking

• dnar-unpack binary-cache --cache my-cache

This will unpack delta.dnar from the current working directory into a local binary cache directory at my-cache with the same layout as nix copy, which can then be imported using nix copy:

nix copy --from file://$(readlink -f my-cache) --all --no-check-sigs

Compression

DeltaNAR files are uncompressed, and compression is left up to the user. To pipe the DeltaNAR output use the special input/output argument -:

• dnar-pack ... --out - | xz > delta.dnar.xz
• xzcat delta.dnar.xz | dnar-unpack ... --input -

References

1. Nix pills - Garbage collector
2. nix.dev - Garbage collector roots

Deduplication

DeltaNAR tries to achieve maximum deduplication by doing multiple levels of analysis of what’s being deployed.

CDC

Individual files in the Nix store are chunked using a content defined chunker.

Files are transferred by transferring a list of content addressed chunks. If a sub-file chunk already exists in the target Nix store (even in another store path), it will be taken from the existing chunk, completely avoiding re-sending the data.

File

To avoid packing a long list of chunk entries for files which are fully identical, a hash per file is also computed. If a file hash matches exactly, its contents will be reused in full.

Directory

To avoid sending a long list of files for directories which are fully identical, a recursive directory hash is also computed.

If a directory hash matches exactly, a reference to it will be packed in the DNAR and the directory contents will be reused.

DNAR format

The DNAR format is specified using Protobuf.

{{#include ../../../dnar/dnar.proto}}

Acknowledgements

DeltaNAR is sponsored by OroraTech🚀