Manual · Chapter 26

Troubleshooting

Go to the contents

The programs report errors as an ERROR line (for call and store errors as an error: line). The exact cause is in the message itself; the last line only summarizes, for example backup failed (see the message above) or restore failed (see the message above). If there was no message of its own before, the last line names the kind of error, for example restore failed: not found:

TextTypical cause
I/O errorReading the source or writing the store/target failed; disk full (also /tmp or --index-dir), server not reachable, connection broken.
invalid argument or requestWrong option or combination; the server refused the session (API key wrong, directory not in the ACL); --tls-ca unusable; a different --hash for an existing store.
not foundSource, store (store.json missing), backup name or run not found; --at lies before the first run.
not supported--snapshot on for a source that is not an LVM volume; function not available on this platform.
corrupt or inconsistent dataAn object does not match its ID or checksum. Run scrub, repair from a copy (Chapter 10).
protocol errorTLS check failed (--tls-ca does not match the server's certificate, e.g. after tls-init --force) or unsuitable program versions.
decryption failed (wrong key or corrupted data)Wrong or missing encryption key; or the data has been tampered with.

26.1 Backup problems#

SymptomCause and remedy
Snapshot becomes invalid / backup aborts--snapshot-size was too small for the changes during the backup. Increase the value (--snapshot-size 20% or 10G).
“A second agent aborts”Only one agent runs per volume. If cron jobs overlap, spread out the schedule.
Left-over snapshots <volume>_bkpsnapIntended after a crash so that the next run continues from the same point in time. Otherwise the next run cleans up snapshots with the tag bkp_agent; --discard-interrupted removes them at once. A volume without this tag is never touched.
Guest stays frozenThe post hook always runs, also on error and SIGTERM. If a guest still stays frozen, run virsh domfsthaw <guest> or fsfreeze -u by hand and check the hook command. --hook-timeout limits the hook run time.
Backup very slowFind the bottleneck with bkp_monitor (Chapter 13): reading, CPU or writing. Increase --threads, --hash blake3 for new stores, lower the compression (--level 1 or --compression lz4), index on SSD (--index-dir).
Backup over a slow line (internet, branch office)The end line of the run names, per target, the amount transferred, the rate and the share of the time the agent waited for the target; with a busy line this share is high, which is not an error then. Sending less helps most: use --no-fs-bitmap only when a bit-identical copy is needed (otherwise free areas are transferred too), and stay with zstd rather than --compression lz4 (lz4 saves CPU but sends more over the line); a higher --max-level lets the adaptive compression rise further on a slow line.
Every backup stores much that is new although little changesThe source changes in a scattered way (e.g. defragmentation, database reorganization, encrypted volumes) or the chunk size does not fit. store-info shows the chunk measurement (Chapter 25).
--hash or --chunk-size is refusedThe store has another method or another size. Without a value the run takes over that of the store; changing only works with --type full.
Exit 3A sink with required=no has failed; the others have their run. The next run fills up the missing sink.
Exit 75 with bkp_agent or --resume-pendingNot an error: the same job is already running. The run ends at once, without waiting or disturbing the running backup.
Exit 77, in the scheduler “Not started (license)”The license allows no new backups, mostly because the trial period is over or the license was not renewed. bkp_license status names reason and next step (section “Trial version and license”). Restoring goes on.
The space does not become free after deletingdelete/prune only free data no other run needs, and no packs younger than --gc-min-age (24 h). With WORM only after the lock has expired. store-info --exclusive shows what a name takes up alone.

26.2 Restore problems#

SymptomCause and remedy
Exit 3, messages about damaged areasSingle chunks were missing or damaged and were written with zeros (Chapter 7). Run scrub and repair on the store, then restore again or read from another sink.
... is not a usable block store: .../store.json: not found--store does not point to a store directory. What is meant is <out>/<store>.cbs, not the --out directory itself.
decryption failed--decrypt-key is missing or wrong. It is the key of the sink that was backed up into.
cmp with the original differsExpected if free areas were skipped (Chapter 3). Check with fsck and a file comparison or back up with --no-fs-bitmap.
Target smaller than sourceA device must be at least as large as the source (SOURCE in --list, source_size in --show-run).
Old backups (.infb, .csfb ...) are not foundThe format of version 1 is only read by bkp_restore_legacy (Chapter 27).

26.3 Update problems#

SymptomCause and remedy
no update url configured: nothing to dourl is missing in updater.conf. Without url updates are off.
Manifest refused (expired)The manifest on the update server has expired. Ask AegisGrid support; until then the installed programs keep running unchanged.
Manifest refused (number too low / replay)An older manifest was delivered. The server should always deliver the newest; --status shows the number seen last.
Signature invalidpublic_key in updater.conf does not match the publisher's key. Compare the key with the publisher.
... activated installations only (HTTP 403)The channel requires an activated installation. Activate with bkp_license activate and the activation code, without a code request the activation with bkp_license offline-request (chapter “Automatic updates”).
... refused this installation's certificate (HTTP 403)The installation certificate has expired or been revoked. bkp_license status shows the state; activate anew with bkp_license activate or make a new request.
Update refused: missing librariesdependencies = bundled or auto with a package that contains lib/; or allow_package_install = yes. The message names the missing libraries and packages.
Exit 75Not an error: another updater is running right now, or the job scheduler could not be asked about its jobs. The next run tries again.
No update although a new release existsThe manifest contains no matching package for this system (--status shows the detected system) or rollout_percent still excludes this machine.
Update fails after switchingThe updater switches back automatically. The version is not tried again until a newer one appears; --force forces a new attempt.