/ Backups

My Backup Setup

My backup setup is massively overengineered. But hey, overkill is underrated right? Right??

Ok fine I'm just a nerd.

Anyways, as I mentioned earlier, I use ZFS. ZFS's efficient and high performance incremental send and receive functionality is a key part of my system. But it all starts with the hardware.

The Hardware

I built my own NAS. Actually, two of them. All this really means is I built two computers and set up the software on it to share the storage space over the network as a network drive.

My NAS (at my apartment)

  • MSI Z77MA-G45 Motherboard
  • Intel Core i7 3770 CPU
  • 4x 8GB G. Skill DDR3 1600 RAM
  • Antec Kuhler 920 CPU Cooler
  • Seasonic SS-400FL Fanless Power Supply
  • Some random 2 port SATA controller
  • 1x 4TB Toshiba N300 hard drive + 5x shucked WD 4TB My Book external hard drives
  • ADATA SU800 128GB SSD
  • Silverstone Temjin TJ08-E Micro ATX Case

System two (at my parent's house) (also refered to as my parents' NAS)

  • Asrock H370M-ITX/ac Motherboard
  • Intel Core i3 8100 CPU
  • 2x 8GB Patriot DDR4 200 RAM
  • Noctua NH-L9x65 CPU Cooler
  • Corsair CX-500M Power Supply
  • 5x 3TB Seagate NAS + 1x 3TB Toshiba P300 hard drive (it originally had 6 of the Seagate drives but one died)
  • Adata XPG SX7000 128GB NVMe SSD
  • Fractal Node 304 Mini ITX Case

I built my NAS five years ago and my parents' NAS recently. Originally the 6x Seagate drives were in my NAS. When I built my parents' NAS, I copied the files from my NAS to my parents' NAS, and then swapped the drives so that I would have the greater capacity drives in my NAS.

If I could do it all over again, I wouldn't have bought drives all of the same model for my NAS. As I stated earlier, if you buy all your drives of the same model, the probabilities of failure of each drive are all correlated with each other. Now granted, I have ZFS configured in the equivalent of RAID 6, so it can withstand two simultaneous drive failures, meaning the probability of losing my array is low. However, it's still not good design to buy drives all of the same model.

Because overkill is underrated, I also bought an Odroid XU4 (a small board computer like a Raspberry Pi) and decided to use that to store yet another copy of my data. I use a 60 GB Mushkin SSD for the operating system (Arch Linux ARM) and a 640 GB 2.5" hard drive for storage.

Why two systems?

Offsite backup. Obviously relying on cloud storage is cheaper. The only problem is I have terabytes of data, and uploading that at 10 Mbps, along with the 1 TB of total upload and download bandwidth cap that my ISP restricts me to, makes cloud storage less effective. Could I still make it work eventually? Sure. But I enjoy building and using these systems. And I am always in complete control of my data.
Furthermore, my parents' NAS actually serves as a backup destination for my parent's data. My dad was about to buy more storage for this purpose, but I saved him from doing that. Also, this computer will serve as a media center PC as it is connected to their TV.

Software

I will admit that right now, I do not have automated backups to my parents' NAS. I am currently doing that manually, and plan on either writing the scripts to auomate this or use a program (such as zrepl) that someone else wrote.

A lot of data flows through my backup system:

  • Backup documents and photos from my desktop to my NAS via rsync. This is stored as its own ZFS dataset on my NAS
  • All of my school work files are stored in git repositories (git is a version control system). Any time I make significant changes to my work, the changes are committed to git and uploaded to my school's Github server
  • All of my Linux installations (on my desktop, NAS, parents' NAS, laptop, Odroid, and computer) are installed onto ZFS. I backup the entire root of the operating system for all these systems to my NAS, my parents' NAS, and my Odroid. This is done through ZFS send/receive
  • My NAS backs up documents, photos, TV, and movies to my parents' NAS via ZFS/send receive
    • If I backed up all of my TV and movies to my parents' NAS, my parents' NAS would actually run out of space. Hence, I organize my ZFS datasets into
      • tank/local/Movies
      • tank/replicate/Movies
      • tank/local/TV
      • tank/replicate/TV
        and only send tank/replicate to my parents' NAS
  • My parents' NAS backs up my parents' data to my NAS via ZFS send/receive
  • I'm contemplating doing cloud backup of my documents again. I used to use Arq Backup for this, but I had to run it in a virutal machine because Windows isn't supported, and I got tired of the virtual machine eating up system resources. If I decide to do this, I will most likely use restic to perform encrypted compressed backups to the unlimited Google Drive account I have from my alma mater
  • Every so often, I use rsync to synchronize my documents and photos between my desktop and laptop

So to recap, there are six copies of my most important documents (my school work): my desktop, laptop, NAS, parent's NAS, Odroid, and my school's Github server all have copies of my work.

If you're paying attention, it would seem I'm not in compliance with the 3-2-1 rule (namely, the 3 copies part) for my TV and movies. However, all of the TV and movies that I own are rips of DVDs and Blu-rays that I purchased, and hence, I have another copy.

My Backup Setup
Share this

Subscribe to Seonwoo's Musings