Over the years I have accumulated a sizable music library (mostly flacs, adding up to a bit less than 1TB) that I now want to reorganize (ie. gradually process with Musicbrainz Picard).
Since the music lives in my NAS, flacs are relatively big and my network speed is 1GB, I insalled on my computer a hdd I had laying around and replicated the whole library there; the idea being to work on local files and the sync them to the NAS.
I setup Syncthing for replication and… everything works, in theory.
In practice, Syncthing loves to rescan the whole library (given how long it takes, it must be reading all the data and computing checksums rather than just scanning the filesystem metadata - why on earth?) and that means my under-powered NAS (Celeron N3150) does nothing but rescanning the same files over and over.
Syncthing by default rescans directories every hour (again, why on earth?), but it still seem to rescan a whole lot even after I have set rescanIntervalS to 90 days (maybe it rescans once regardless when restarted?).
Anyway, I am looking into alternatives.
Are there any you would recommend? (FOSS please)
Notes:
- I know I could just schedule a periodic rsync from my PC to the NAS, but I would prefer a bidirectional solution if possible (rsync is gonna be the last resort)
- I read about unison, but I also read that it’s not great with big jobs and that it too scans a lot
- The disks on my NAS go to sleep after 10 minutes idle time and if possible I would prefer not waking them up all the time (which would most probably happen if I scheduled a periodic rsync job - the NAS has RAM to spare, but there’s no guarantee it’ll keep in cache all the data rsync needs)
I have a very similar setup to yours, a relatively large music library around 1.7TB of mostly flac files on my server. I’m able to organize these files locally from my laptop, which at various times has run either OSX, various GNU/Linuxes, or Windows. However I do not bother pushing the files themselves back and forth over the network.
Even if I did, I wouldn’t automate the syncing, I’d only run it manually after I’d done my organizing with Picard for that day. After all, it the organization with Picard isn’t automated, why should the syncing be? I’d probably use rsync for this.
In actual practice I do this: Connect to my server from my laptop using ssh, forwarding X. Run Picard on the actual server through this remote connection. Picard runs just fine over ssh. Opening a browser from a Picard tag for occasional Musicbrainz.org stuff is a little slower but works. I would then use a tmux or screen session to run the rsync command when I’m done with Picard for the day for syncing to a backup if necessary.
I don’t really bother keeping a whole copy of my music collection locally on my laptop or phone though, since It’s been bigger than is practical for a long time. Managing multiple libraries and keeping the two in sync turned into such a hassle that I was spending more time organizing than actually listening (or making mixtapes/playlists). To listen to my music locally I’ve used either Plex or Jellyfin, sometimes MPD (like when my server was directly connected to my stereo receiver), or just shared the folder via samba and NFS.
In the third paragraph you mentioned “tux” but I’m guessing that you meant “tmux”. Just a clarification for readers not familiar with it and want to look it up.
Yeah, that was a typo. Thanks, I’ll fix it.
The main difference is probably that I have a desktop PC rather than a laptop (plus, a few old hard disks lying around).
I think I’ll keep the local replica even when I’m finished reorganizing the library: the local copy doubles as a backup and I must say I am enjoying the faster access times.
Oh yeah, I totally support the local copy. That will save you in times up hardware failure or fuck ups. I could just never keep up with the maintenance and kind of gave up making automatic backups and syncing. But reorganizing often translates to integrating deletions into rsync or whatever syncing protocol you use, and that has caused me headaches and heartaches.