Problems mounting NFS shares on Debian

beagle

I've been using CentOS mostly over the years but since all the changes triggered by RedHat I've been dipping my toes on Debian. So apologies for the newbie questions.

I've been playing around with TrueNAS Core and created a Debian VM to test Shinobi. The installation went fine but I'm having problems mounting a NFS share.

On CentOS I used to include this line on /etc/fstab:

<ip>:/exports /mnt/nfs_share nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,_netdev 0 0

On Debian, the share doesn't mount on boot, but it mounts if I run mount -a.

When I looked at the log I saw the message mount.nfs: Network is unreachable. It seems network-online.target - the dependency that systemd uses to trigger the network mounts - is reached before the DHCP client can get an IP. Therefore the NFS mount fails.

I've google'd for a bit and I found the following possible solutions:

(1) Use noauto,x-systemd.automount,x-systemd.mount-timeout=30,_netdev on /etc/fstab and let the share be mount on first use.

• My concern is the mount be unmounted if the connection is idle.

(2) Disable networking.service and use systemd-networkd.service instead and enable systemd-networkd-wait-online.service. Make all the network changes to move from the default /etc/network/interfaces to set the network on systemd.

• I'm trying to avoid moving away from the default config as much as possible.

(3) Create a new .mount file inside /etc/systemd/system.

• That seems like a lot of work just to create a mount.

What are you all using for your NFS mounts? Particularly when you are assigning the IP using DHCP.

CamoYoshi

Confirm that the network-online.target is finished loading before the system attempts to start the NFS daemon with a Wants=network-online.target instead of the nfs' unit file. Before= & After= might also be worth looking at, putting the mount attempt later on during boot. networking.service by default only looks to see if the interface is up (i.e. do we have a layer 1 connection on the network interface); you want to focus on network-online.target instead; it actually checks for if your interface has a valid IP address or not.

https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Before=

https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

Hopefully this gets you started on the right path... I'll admit my knowledge here is limited but this is what I know, for certain. Good luck!

beagle

@CamoYoshi said: you want to focus on network-online.target instead; it actually checks for if your interface has a valid IP address or not.

It seems it doesn't actually. network-online.target is reached before the DHCP client receives a valid IP. Therefore I need a different .target that is only loaded after receiving a valid IP. I could then use this .target on x-systemd.after=....target

According to systemd.mount documentation automount maybe the answer in the end:

The NFS mount option bg for NFS background mounts as documented in nfs(5) is detected by systemd-fstab-generator
and the options are transformed so that systemd fulfills the job-control implications of that option. Specifically 
systemd-fstab-generator acts as though "x-systemd.mount-timeout=infinity,retry=10000" was prepended to the option
list, and "fg,nofail" was appended. Depending on specific requirements, it may be appropriate to provide some of
these options explicitly, or to make use of the  "x-systemd.automount" option described below instead of using "bg".

havoc

Recently dealt with lots of debian NFS stuff. See my notes dropped into a pastebin...forgive the its written for myself style

Password is LES

https://pastebin.com/4B7w9tr7

Permissions are 99% of the problems. Note the hacky group solution (ctrl-f havoc...that part)

I'd suggest hardcode the IP for testing purposes to get past dhcp at least for testing

beagle

Thank you for your notes @havoc.

In my case the permissions are working fine. If I run a mount -a everything works. The problem is mounting the share at boot time.

dahartigan

Here's a working example of my fstab which is running on Debian 10, with the NFS server being over a wireguard link (wg0)

10.20.10.6:/srv/nfs4 /mnt/polarbear nfs vers=4.2,_netdev,noauto,x-systemd.automount,[email protected]

I suppose the key difference here is that this is going to wait for the wireguard connection to be established before it will connect. This may help you if the root of your issue is the difference between waiting for a network connection (eg eth0) or waiting for the resulting network connection from a systemd service (such as in my case wg-quick@wg0)

Mind you, with the above, absolutely zero other configuration of the Debian system was done (no changing permissions etc as mentioned earlier) so this is a very non-invasive approach.

beagle

Thank you @dahartigan. Interesting example using Wireguard. It would probably work with OpenVPN as well.

It seems x-systemd.automount is the way to go.

havoc

@beagle said:
Thank you for your notes @havoc.

In my case the permissions are working fine. If I run a mount -a everything works. The problem is mounting the share at boot time.

Another ugly hack, but you could stick a cron job into the startup with a delay to get around the dhcp and do a late mount that way

seanho

I'd think the root issue is systemd-networkd-wait-online finishing too early.

VM networking using bridge mode? DHCP enabled in /etc/network/interfaces in the VM? If so, networking.service is the unit that reads that and brings up the interface; it should be sequenced to run before wait-online.