Skip to content

Backups

Backups are managed via SQLBak, a backup service installed locally on the web and database servers. It connects to the SQLBak cloud dashboard where automated backup jobs can be created and monitored.

The backup flow: SQLBak creates a backup locally in a temp folder → uploads it to AWS S3 (immutable, primary backup storage) and the Hetzner Storage Box → the Synology NAS pulls a copy each morning as an on-premise redundancy layer.

Encryption

All backups are encrypted. The encryption password is stored in Dashlane.


Schedule

Backups run daily at 05:30.


Storage & retention

Storage Retention Purpose
AWS S3 (daily) 30 days Immutable daily backups
AWS S3 (weekly) 90 days Immutable weekly backups
AWS S3 (monthly) 180 days Immutable monthly backups
Hetzner Storage Box 1 month Secondary offsite copy
Synology NAS 30 days On-premise, fast restore

What is backed up

Type Scope
Database Full dump of all tables
Files All files inside the deployments folder of the Bedrock projects

AWS S3

S3 is the primary backup storage. Backups are stored in the eu-north-1 (Stockholm) region, ensuring data stays within the EU.

Object Lock (WORM)

The S3 bucket uses Object Lock in Compliance mode — a write-once-read-many model that prevents backups from being deleted or overwritten, even by an administrator. This protects against ransomware and accidental deletion.

Default retention is set at the bucket level and enforced on every uploaded object.

Bucket structure

SQLBak uploads all backups to the daily/ folder. An AWS Lambda function (backup-retention-manager) runs automatically on each upload and copies files to weekly/ and monthly/ folders with extended retention where applicable.

db/daily/<brand>/<filename>     → 30 days retention
db/weekly/<brand>/<filename>    → 90 days retention  (copied every Sunday)
db/monthly/<brand>/<filename>   → 180 days retention (copied first Sunday of month)

files/daily/<brand>/<filename>  → 30 days retention
files/weekly/<brand>/<filename> → 90 days retention  (copied every Sunday)
files/monthly/<brand>/<filename>→ 180 days retention (copied first Sunday of month)

Lifecycle Policy

A Lifecycle Policy automatically deletes objects after their retention period has expired. Object Lock prevents early deletion — the Lifecycle Policy only deletes objects once the lock has been lifted.

Folder Deleted after
daily/ 31 days
weekly/ 91 days
monthly/ 181 days

Lambda function

The backup-retention-manager Lambda function triggers on every S3 PUT event and handles the weekly/monthly copy logic. It runs in eu-north-1 and uses a restricted IAM role with only the permissions it needs (s3:GetObject, s3:PutObject, s3:PutObjectRetention).

# Logic summary
if Sunday:
    copy daily  weekly/ with 90 days retention
if first Sunday of month:
    copy daily  monthly/ with 180 days retention

SQLBak configuration

Setting Value
--single-transaction Enabled
--lock-tables Disabled

--single-transaction ensures the database dump is consistent without locking tables, preventing site downtime during backup.


Synology

The Synology NAS is an on-premise backup redundancy layer, independent of Hetzner. Each morning at 06:00 it connects to the Hetzner Storage Box and pulls the latest backup.

The fetch script is at /var/services/homes/pt/scripts/fetch-backups.sh and runs via the Synology task scheduler.

The Synology uses a read-only sub-account on the Storage Box. This means a Synology breach cannot touch the production environment — it only has read access to the backup storage, not to the web or DB servers.

Administration

Access method Details
QuickConnect quickconnect.to — QuickConnect ID: barberklingen
Direct (LAN) http://192.168.1.125:5000

A user account must be created by an administrator before you can log in.


Monitoring

Backup monitoring runs at two levels.

SQLBak sends email notifications on every backup job — both on success and on failure. This covers the daily backup execution itself and is the first line of alerting.

CloudWatch monitors the AWS Lambda function that handles the weekly/monthly copy logic. Notifications are sent via SNS to the team email.

Alarm Trigger
backup-retention-manager-errors Lambda function throws an error
backup-retention-manager-no-invocations Lambda function does not run on a given day

How to restore

Warning

Always double-check the restore destination before confirming. Restoring to the wrong location can overwrite production data.

Restore files

Overwrites everything

This replaces all files in the project with the backup contents.

  1. Go to sqlbak.com and sign in.
  2. Select the server Web - Production.
  3. Open the Jobs tab → select Production - Files.
  4. In the right panel, pick the backup you want to restore and click the restore icon.
  5. Follow the restore wizard, double-checking the restore location.
  6. Wait for the process to complete and verify the result.

Use this for selective file recovery

  1. Connect to the Hetzner Storage Box via SSH or FTP.
  2. Navigate to /sqlbak/production/files.
  3. Identify and download the backup for the site (usually the most recent).
  4. Decompress the backup locally.
  5. Connect to the production server with an FTP client.
  6. Upload only the files you want to restore to their correct destination.
  7. Wait for the upload to complete and verify the result.

Use this when restoring from weekly or monthly backups

  1. Go to the AWS S3 console.
  2. Navigate to the backup bucket.
  3. Browse to the relevant folder (daily/, weekly/, or monthly/).
  4. Select the file and click Download.
  5. Restore manually via FTP or SQLBak.

Restore database

Data loss

This performs a complete database restore. Any data added after the backup point will be lost.

  1. Go to sqlbak.com and sign in.
  2. Select the server DB - Production.
  3. Open the Jobs tab → select backup - production.
  4. In the right panel, pick the backup you want and click the restore icon.
  5. Follow the restore wizard. Make sure to select the correct target database.
  6. Wait for the process to complete and verify the result.

For further reference: SQL Server Restore Database from Backup — SqlBak Blog


Changelog

June 26, 2026

  • Added AWS S3 as primary immutable backup storage (Object Lock, Compliance mode, eu-north-1)
  • Implemented retention policy: 30 days daily, 90 days weekly, 180 days monthly
  • Deployed Lambda function backup-retention-manager for automated weekly/monthly copy logic
  • Configured S3 Lifecycle Policies for automatic cleanup after retention expiry
  • Enabled --single-transaction and disabled --lock-tables in SQLBak to eliminate backup-related downtime
  • Added CloudWatch alarms and SNS notifications for backup monitoring

November 22, 2023

  • Introduced SQLBak backup procedures
  • Extended backup storage period from 14 → 30 days
  • Removed the legacy cron backup scripts (/etc/cron.daily)
  • Refactored fetch-backups.sh on the Synology

September 6, 2023

  • Replaced push-based backups (web server → Synology) with a pull-based mechanism (Synology → Storage Box)
  • Synology now uses a read-only sub-account on the Hetzner Storage Box
  • Synology retention extended from 7 → 14 days