aegis -- an archive tool

Docs | Source | Issues | Patches

Aegis is a CLI tool for archiving important files in your life. I use it to archive photos, important documents, music, and other data that I never want to lose.

Archiving vs Backup

So you already have backups for your important data, right? Why is that not enough?

The problem aegis tries to solve is that backups are too passive for data you want to save for your entire lifetime. Any unintentional change is quietly backed up and eventually the original will be lost.

Any of the following possibilities could happen to your data:

1. You fat finger an rm and don't notice it
2. A bug in an image viewer overwrites a file after you close it
3. You copy your data to a new medium, you don't have ECC RAM and a bit flips
4. You are not storing your data on a bit-rot-proof FS and a bit flips

All of the above can be solved with proper backups only if you notice it. Aegis differs in that it requires you to explicitly review and snapshot changes you want to commit to the store. Think of it like git, but optimized for archiving files - lots of files of any size - to various backup locations.

Concepts

To get started, you just need to create an aegis.toml file in the directory you wish to archive:

name = "my-photos"
default_backend = "my-local-backup"

[backends.my-local-backup]
type = "fs"
dir = "/path/to/backup/location"

Aegis stores all of its data in a Backend, it is up to you how you want to configure these backends.

Some backends are designed to just store data, for example the "fs" backend will store snapshots and blobs in the provided directory.

Some backends are designed to compose other backends, for example the "tee" backend will save data to all downstream backends.

The "encrypt" backend will encrypt all data that passes through it.

Below is a sample config that sends data to a local "fs" backend, another "fs" backend that points to a mounted network share, a "b2" backend that uploads to a Backblaze B2 bucket, and an "encrypt" backend that wraps the B2 backend, encrypting all blobs passing through it:

name = "my-photos"
default_backend = "all"

[backends.all]
type = "tee"
sub_backends = ["my-local-backup", "network-share", "my-encrypted-b2"]

[backends.my-local-backup]
type = "fs"
dir = "/path/to/backup/location"

[backends.network-share]
type = "fs"
dir = "/mnt/some-network-mount"

[backends.my-encrypted-b2]
type = "encrypt"
key = "<REDACTED>" # generate with `aegis gen-key`
sub_backend = "my-b2-bucket"

[backends.my-b2-bucket]
type = "b2"
b2_key_id = "<REDACTED>"
b2_key = "<REDACTED>"
bucket_name = "my-b2-bucket"
prefix = "some/path/to/prefix/repo/with" # optional

Prior art:

• boar
• git-annex

I have used both of these solutions for many years before writing aegis. They both work great, but neither perfectly match my needs.