What you will learn

Automate a weekly restore rehearsal that proves a backup archive is actually usable — extracting it into a scratch location, running an integrity check, and comparing a sample against a manifest — because an unverified backup is only a hope.

Before you begin

Run this against a copy, never the production data path. Budget real disk space: a restore test needs room for the extracted data alongside the archive itself.

1. Pick the most recent backup and extract it in isolation

#!/usr/bin/env bash
set -Eeuo pipefail

VAULT_DIR="/backups/myapp/daily"
SCRATCH_DIR="$(mktemp -d /var/tmp/restore-test.XXXXXX)"
trap 'rm -rf -- "$SCRATCH_DIR"' EXIT

latest="$(find "$VAULT_DIR" -name '*.tar.gz' -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2-)"
[[ -n "$latest" ]] || { echo "no backup found in ${VAULT_DIR}" >&2; exit 1; }

echo "testing restore of: $latest"
sha256sum -c "${latest}.sha256"
tar -xzf "$latest" -C "$SCRATCH_DIR"

find -printf '%T@ %p\n' | sort -rn picks the newest file by actual modification time, which is more reliable than trusting filename lexical order once naming conventions ever change.

2. Confirm structure and row-level sanity, not just "tar exited 0"

EXTRACTED_ROOT="${SCRATCH_DIR}/$(basename "$latest" .tar.gz | sed 's/-[0-9]\{8\}-[0-9]\{6\}$//')"

required_files=("config.yaml" "data/schema.sql" "data/records.csv")
for f in "${required_files[@]}"; do
    [[ -f "${EXTRACTED_ROOT}/${f}" ]] || { echo "MISSING: ${f}" >&2; exit 1; }
done

row_count="$(wc -l < "${EXTRACTED_ROOT}/data/records.csv")"
if (( row_count < 100 )); then
    echo "SUSPICIOUS: only ${row_count} rows in records.csv (expected hundreds+)" >&2
    exit 1
fi

echo "structure ok, ${row_count} rows present"

A tar archive can extract cleanly and still be a useless backup — e.g. captured mid-write with a truncated file. Checking for expected files and a plausible row count catches that class of failure that exit codes alone miss.

3. Record the result so drift over time is visible

RESULT_LOG="/var/log/myapp/restore-tests.log"
echo "$(date -u +%FT%TZ) archive=${latest} rows=${row_count} status=ok" >> "$RESULT_LOG"

Example output

testing restore of: /backups/myapp/daily/myapp-20260919-030000.tar.gz
/backups/myapp/daily/myapp-20260919-030000.tar.gz: OK
structure ok, 48213 rows present

Verify the result

Deliberately corrupt a test archive (truncate it with truncate -s -1024 backup.tar.gz) and confirm the script fails loudly at the checksum step, not silently at extraction.

Troubleshooting

If sed in the basename step doesn't match, your archive naming changed — keep the extraction-root derivation in sync with whatever auto-backup-retention-vault actually names its files.

Next steps

Schedule this weekly under flock-cron-concurrency-guard, and alert via webhook-alert-dispatcher on any non-ok status so a broken backup is caught before it's needed for a real recovery.