Is there a way to speed this up?
sudo rsync -aogW --partial --progress --exclude-from=/etc/restic.excludes -e 'ssh -p 22' / [email protected]:/Volumes/TD/rsync/beast_2023_11_07_1106
On the Debian side, rsync 3.2.7
On the iMac Pro side, rsync 3.2.7
Wired ethernet, 10GbE, send speed (in Mission Control) varies around 250MB/s to 500MB/s.
There are about 6.7M files to transfer. After ~42 hours, 1.1M have transferred.
This is what the output of the rsync command looks like:
tank/data/sleep_08-30/tank/data/sleep/files/md5/73/0905320c84d53c108436297fbc5f48
50,429 100% 53.76kB/s 0:00:00 (xfr#82163, to-chk=4655176/6705279)
tank/data/sleep_08-30/tank/data/sleep/files/md5/73/090a02e70e1eff0b4bd613026fe67c
51,262 100% 54.35kB/s 0:00:00 (xfr#82164, to-chk=4655175/6705279)
tank/data/sleep_08-30/tank/data/sleep/files/md5/73/093b2096cb530739c6f62354f1ce37
Restic, on the other hand, is wicked fast backing up to the same external drive hanging off the iMac (even while rsync is simultaneously running). I'm using rsync because I would like to have a file-by-file backup, as this drive is being backed up to Backblaze, and without a file-by-file backup, I would be faced with restoring the whole restic backup (8TB) if I lost my external drive and needed to recover files.
Thanks for any suggestions!
zfssend might be an option too, but setting up zfs on macOS could be a challenge, and I’m not sure if backblaze will back up a drive with that file system. Also don’t know how reliable zfs on macOS is.