My Restic Backup config

I definalty need to move the code below to files or github, but I’m too lazy to do it now.

Why?

With the introduction of Time Machine, Apple has made it very easy to backup your Mac to an external hard drive. But it’s not easy to backup to NAS or cloud(either you need to pay for iCloud or you need to use Time Machine to backup to NAS, which is not very reliable).

I’m using the Time Machine every week because I need to manually connect my external hard drive to my mac. However, recently when I was investigating my potential backup solution, I found the reliability of Time Machine is debatable, especially through wifi connection. Please refer to this hackernews disccusion.

The motivation besides the inconvience manual plugin, is that my iCloud is nearly full of 200GB data since I backed up most of my stuff in mac to iCloud. So I need a simple backup solution for my Mac to NAS/other cloud services, which also follows the

Backup golden rule:3-2-1 backup rule: that is, 3 copies of your data, 2 of which are local but on different devices, and at least 1 copy offsite.

I also want to send my data to NAS/Cloud in encrypted format, so that I don’t need to worry about the security of my data.

Additionally, I need recieve iOS notification when my backup is done, so that I can be sure that my backup is working, or I can take action when my backup is not working.

Hardwares/Services

Before I move on, I think the hardware I got also needed to be mentioned here. I used a x86 mini pc(smaller and cheaper than a mac mini) as my NAS(also a lot of other services), which is running Ubuntu 20.04. I also got a 2TB external HDD which mounted as /mnt/backup in my NAS.

I got a didn’t get a Synology NAS, since I move a lot and am waiting for the releasing of full SSD NAS for better performance.

I realised that this external hard drive is not reliable since they are not built for 24/7 usage. That’s why I also backup this disk to other cloud service as well.

The cloud service I’m using is Backblaze B2. It’s the cheapest cloud storage service in general(include storing/downloading fees), which is only $0.005/GB/month. The B2 service also provides S3 compatible API, which means you can use any S3 compatible tools to access your data.

Actually their unlimited personal backup services is also a good choice, but considering my data size, I think it’s better to use B2 in long term.

Strategy

Simple solution: mac -> NAS -> B2 in my case but you can also just use mac to backup to both NAS and B2. In this blog I will only talk about the Mac -> NAS.

Restic

Restic is a fast, secure, multi-platform command line backup program. It’s also very easy to use and configure. It is encrypted by default, and is verifiable by design, meaning even your cloud provider can’t read your data.

In Restic, a repository is a directory where all the data is stored. It can be a local directory, a remote directory via SFTP/ S3/ other protocols. Last time I checked even though Restic support B2 protocal but offical document recommend use B2’s S3 API instead

Installation is just one line:

brew install restic

After that, setup a repository in your NAS, please refer to this if you are setting up with SFTP for your NAS, for example.

If you don’t want to specify your repo everytime you run restic, you can set it in your environment variable:

export RESTIC_REPOSITORY="sftp:username@hostname:/path/to/repo"
export RESTIC_PASSWORD="yourpassword" # you can use password
export RESTIC_KEY_FILE="/path/to/key/file" #you can also use a key file instead of password

You can specify the backup directory in your mac using a plain file(Within file, each line is a path), e.g. restic_backup_list.txt

I put all config related files in a folder called restic_backup in my home directory.

Here my shell command to backup my mac to NAS only and you’ll get the idea to backup mac to other services; please replace ‘your wifi name here’ and ‘your_User_Name’ with your own:

CMD1="/opt/homebrew/bin/restic backup --files-from /Users/your_User_Name/restic_backup/restic_backup_list.txt --password-file /Users/your_User_Name/restic_backup/restic_password"

# Check if AC power is connected
power_status=$(system_profiler SPPowerDataType  | grep "Connected:" | awk '{print $2}')

# Check if home wifi is connected
wifi_status=$(/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -I | awk '/ SSID:/ {print $2}')

if [ "$power_status" != "Yes" ]; then
    python3 ~/restic_backup/restic_noti_sender.py "power" "Power is not connected"
fi

if [ "$wifi_status" != "GL-5G" ]; then
    python3 ~/restic_backup/restic_noti_sender.py "wifi" "Wifi is not connected"
fi

# If both power and wifi are connected, run the script
if [ "$power_status" = "Yes" ] && [ "$wifi_status" = "your wifi name here" ]; then
    OUTPUT1=$($CMD1)
    STATUS1=$?
    if [ $STATUS1 -eq 0 ]; then
        BODY1="Restic Mac to NAS was successful"
        ###python3 ~/restic_backup/restic_noti_sender.py  "0" "$BODY1"
    else
        BODY1="Restic Mac to NAS failed with status $STATUS1 "
        ###python3 ~/restic_backup/restic_noti_sender.py  "$STATUS1" "$BODY1"
    fi
fi

Need to Send Nofitication?

The script will check if your mac is connected to power and your home wifi, if both are connected, it will run the restic backup command. In the python3 script, I’m using Bark to send iOS notification to my phone.

You can hosted your own Bark server or use the public one. I’m hosting my own server for privacy reason.

REMEBER TO UNCOMMENT THE LAST TWO LINES THAT STARTS WITH python3 IF YOU WANT TO SEND NOTIFICATION.

The python script is reviewed and modified by GPT4,

import sys
import requests
import json

device_key = "your key here"
URL = "https://api.day.app:443/push" # public bark server, you can confirm it from bark iOS app


def handle_power_failure(status, body):
    if 'power' in status:
        return 'power failed!', 'restic Mac to NAS', 'active', status
    else:
        return None


def handle_wifi_failure(status, body):
    if 'wifi' in status:
        return 'wifi failed!', 'restic Mac to NAS', 'active', status
    else:
        return None


def handle_success(status, body):
    if '0' == status:
        return 'restic succeed!', 'restic Mac to NAS', 'passive', body
    else:
        return None


def handle_error(status, body):
    explain = {
        '1': 'fatal error (no snapshot created)',
        '3': 'some source files could not be read (incomplete snapshot with remaining files created)'
    }
    if status in explain:
        body = f"exit code:{status}!{explain[status]} {body}"
    else:
        body = f"exit code:{status}!Check official doc. {body}"
    return 'restic failed!', 'restic Mac to NAS', 'active', body


status_handlers = [
    handle_power_failure,
    handle_wifi_failure,
    handle_success,
    handle_error,  # this should always be last, as it's the default case
]


def build_post_data(title, group, level, body):
    return {
        "body": body,
        "device_key": device_key,
        "title": title,
        "sound": "minuet.caf",
        "badge": 1,
        "icon": "https://avatars.githubusercontent.com/u/24937341?s=200&v=4",
        "group": group,
        "level": level
    }


def send_request(status, body):
    try:
        for handler in status_handlers:
            result = handler(status, body)
            if result is not None:
                title, group, level, body = result
                break
        else:
            print(f"Unhandled status: {status}")
            return

        data = build_post_data(title, group, level, body)

        response = requests.post(
            url=URL,
            headers={
                "Content-Type": "application/json; charset=utf-8",
            },
            data=json.dumps(data)
        )

        print('Response HTTP Status Code: {status_code}'.format(
            status_code=response.status_code))
        print('Response HTTP Response Body: {content}'.format(
            content=response.content))
    except requests.exceptions.RequestException:
        print('HTTP Request failed')


if __name__ == "__main__":
    status = sys.argv[1]
    body = sys.argv[2]
    send_request(status, body)

Schedule your backup

Try to run the shell now and see if it works.

Since mac can’t use cron, I’m using the official launchd to schedule my backup.

You can use the following for your launchd plist file, which is Apple’s official ‘cron-like’ solution, for automatic execution of commands.

Please replace ‘/Your/path/to/restic_backup/restic_backup.sh’ to your shell script and the time you want to run the script.(Currently 0:33 am everyday)

Please note ‘com.restic.backup’ within the label has to be compulsory as this plist file’s name, otherwise plist doesn’t work.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.restic.backup</string>

  <key>ProgramArguments</key>
  <array>
    <string>/Users/your_user_name/restic_backup/restic_backup.sh</string>
  </array>

  <key>StartCalendarInterval</key>
  <dict>
    <key>Hour</key>
    <integer>0</integer>
    <key>Minute</key>
    <integer>33</integer> 
  </dict>

  <key>RunAtLoad</key>
  <true/>
  <key>StandardOutPath</key>
  <string>/Users/your_user_name/restic_backup/restic_logs.log</string>

  <key>StandardErrorPath</key>
  <string>/Users/your_user_name/restic_backup/restic_logs.error.log</string>
</dict>
</plist>

Conclusion

The end. Enjoy your backup!