Hey #linux#debian people: it’s occurring to me that #rclone might not actually be the best way to do what I’m doing. So: if you had two Debian servers on a vpn, both with 1 gig fiber links to the internet, in cities 100 km apart, how would you go about having shared filesystems between them? Right now I am using rclone mount with sftp. Is there a less janky way?
@fastfinge are you aware rclone has a cache mount type (I forget details)? You could stack that on top of the sftp mount and it might be a bit smoother.
@fastfinge BTW Pretty sure the /srv stuff isn't even necessary anymore if you share more than one folder. @yo@tripplehelix@fosstodon.org@quanin@modulux
@fastfinge Do they need to be shared, or are you looking for eventual consistency? Because my first answer is that I wouldn't; I'd locally cache the data for some period of time be it seconds, minutes, hours or days, and then batch-sync it on a schedule. Removes an entire layer of complexity from proceedings.
@jscholes Right, but we don't know what files we're going to need until we need them. And we need to know what files are available at all times. So now we're into keeping an updated list in sync between two machines, and doing caching and batch syncs, and this is all starting to sound like a filesystem.
@jscholes NFS is pretty much what I want. In short, I'm managing my dozens of Linux ISO's and hours of public domain video, but the box that actually downloads the ISO's is on a different network from the one that requests, serves, and indexes them.
@fastfinge I feel like you might be able to accomplish it with an NFS mount, but I need to know more about how that shared filesystem is handled. Is it hosted on a drive belonging to one of your servers? Is it in object storage?
@quanin One server has all the storage, and the other has all the cpu and ram. The first server is just packed with 20 tb spinning discs, and enough cpu and ram to serve files.
@modulux The thing that scares me about nfs is that it’s an ancient protocol, and authentication and encryption feel like a bolted on mess. Especially because I haven’t set up Pam or whatever to share accounts between the systems. So I’m confused about what happens when nfs tries to sync permissions.
@fastfinge It's ancient, but there's been relatively newer versions. That said I'm only talking from second-hand knowledge since I never needed to set this up. Another option you might consider is sshfs.
@modulux@fastfinge NFS (even v3) with a VPN would be fine. I use it and works fine. I proposed Syncthing in the top post of this thread too.
There's a distinct lack of WAN-friendly shared filesystems. So I would suggest either VPN + traditional shared filesystem, syncthing, or something SSH-based (like the rclone you are already using, or the already suggested sshfs).
