Introduction

Go to the contents

AegisGrid backs up files and block devices (images, LVM volumes, partitions, whole disks) and restores them. Every backup goes into a deduplicating store as a run: the agent cuts the source into pieces (chunks) whose boundaries follow the content, and stores only the chunks the store does not have yet. Every run is complete on its own and can be restored by itself. The data is compressed with zstd or lz4, encrypted on the client if wanted and sent to a backup server (bkp_target) over TLS.

This manual is written for administrators: they set up and run backups, the backup server and restores.

Readers who know version 1 with FULL, DIFF and INC backups find a comparison and a checklist for the update in Chapter 27 (“Switching from FULL/DIFF/INC”).

All examples work as printed; the output shown comes from real test runs. Where an example only works on a real LVM system, under Windows or with a hypervisor, it says so.

1.1 What AegisGrid can do#

• Deduplicated snapshots. Every backup is a complete point in time, but only what the store does not have yet is stored. There are no chains, no DIFF and no merging. Deduplication covers everything in the store, so also machines that share a store.
• Content-defined chunks (FastCDC). If something is inserted near the start of a source, the boundaries after it do not move; the following chunks stay the same and are recognized.
• A familiar view: the first run of a backup name is called Full, all later ones Incremental. This is only for orientation: every run can be restored on its own.
• Free space is skipped. On ext2/3/4 and NTFS the agent reads only the used space. Zero areas are recorded as holes, not stored.
• Consistent backups of running systems: LVM snapshots (Linux), VSS shadow copies (Windows) and freely configurable hooks.
• Files and folders: instead of whole disks, single paths can be backed up too, with permissions, ACLs and exclusions; single files, whole folders, earlier versions or a ZIP archive come back, and a run can be viewed as a folder. A ransomware warning protects the older states (Chapter 9).
• Network backup: the agent streams straight to bkp_target, fully TLS-encrypted (the server certificate is pinned), with sign-in by API keys that are bound to directories. Only what the store on the server is missing is sent.
• Encryption on the client (AES-256-GCM or ChaCha20-Poly1305, per store): the server sees neither content nor keys.
• Several destinations in one run (sinks): up to 8 stores, local and on servers; the source is read only once.
• Parallel and fast: reader, chunker and several workers (hash and compression) work as a pipeline; SHA-256 uses SHA-NI/AVX2 where available; with --hash blake3 the CPU load of hashing drops markedly.
• Protecting the data: retention per run, scrub (checking all data), repair from a second copy, partial restore of damaged backups, mirroring whole stores, also into S3 object storage, export of single runs as an archive (tape, USB) and WORM locks.
• Signed updates: bkp_updater installs updates atomically, tests them first and rolls back by itself if something goes wrong.

1.2 The programs#

Every program has --help.

1.3 Basic terms#

1.4 Overview of the flows#

Data flow

  Source (volume/file)                                     Destination
  ┌──────────────┐  snapshot   ┌───────────┐  local  ─────────►  directory (--out)
  │ /dev/vg0/data├────────────►│ bkp_agent │
  └──────────────┘  (LVM/VSS)  └─────┬─────┘  TLS    ┌────────────┐   ┌─────────────┐
                                     └───────────────►│ bkp_target ├──►│ /srv/backups│
                                                      └─────▲──────┘   └──────┬──────┘
                                                            │ TLS             │ local
                                                     ┌──────┴──────┐   ┌──────▼──────┐
  target volume  ◄─────────────────────────────────  │ bkp_restore │◄──┤ store (.cbs)│
                                                     └─────────────┘   └─────────────┘