Linux

5159 readers

99 users here now

A community for everything relating to the linux operating system

Also check out [email protected]

Original icon base courtesy of [email protected] and The GIMP

founded 1 year ago

MODERATORS

Ategon

anzo

dwraf_of_ignorance

Best practice for creating a unified view of data across multiple Linux systems? (self.linux)

submitted 1 day ago by CoderSupreme to c/linux

3 comments fedilink hide all child comments

I'm looking to create a unified view of data across multiple Debian-based devices without fully replicating the data. My current setup includes:

Main computer with one HDD
Server with four drives
A couple of Raspberry Pis

I want a folder on each device that provides access to the contents from all drives, but without actually copying or syncing the data to each device. I'm aiming for a solution that allows browsing and accessing all files from any device while keeping the actual data in its original location.

I've been looking into using a combination of MergerFS and SSHFS. The idea is to use SSHFS to mount remote directories and then use MergerFS to combine these mounts with local directories into a single view. However, I'm not sure if I should merge the drives in the server and share the merged folder with all other systems or I should share each drive with each system and merge them in there. Is this the best approach or are there better alternatives?

I want to avoid solutions like Syncthing, Dropbox, or Google Drive that would clone the entire data set to each device. I'm trying to avoid data duplication and save storage space on devices with smaller capacities.

I'd love to hear your thoughts on the MergerFS + SSHFS approach, or if you have any other suggestions that might better fit my needs. Any insights, recommendations, or personal experiences would be greatly appreciated!

Thanks in advance for your help!

top 3 comments

sorted by: hot top controversial new old

[–] nous 11 points 1 day ago

Sounds like you just need to keep the data on your server and use samba or NFS and a network mount on the other devices.

[–] nullPointer 6 points 1 day ago

NFS

[–] [email protected] 1 points 1 day ago

You want Upspin. I want Upspin. But Upspin never went anywhere (it's at least 7 years old... ever heard of it?), and I personally believe that it was because it's a royal PITA to set up, and because the tutorial had instructions that expected you to be using GCS. If you wanted to do everything on your LAN, it was even harder.

It's got all the of the features you mention, and it's really the only system that does what it does; I really did try in the early days to get it running, and failed. It still has the caveat:

Upspin has rough edges, and is not yet suitable for non-technical users.

and, at 7 years old, if it hasn't gotten anywhere yet, I think it never will. Commits trickle in, but there's really no significant progress in usability.

Read the mission statement. It's glorious. And then wallow in despair that nothing else does this, and it's a zombie project.