Try this on your server and remount on the client:

# To free pagecache
echo 1 > /proc/sys/vm/drop_caches

# To free dentries and inodes
echo 2 > /proc/sys/vm/drop_caches

# To free pagecache, dentries and inodes
echo 3 > /proc/sys/vm/drop_caches

Source: https://unix.stackexchange.com/questions/433051/mount-nfs-stale-file-handle-error-cannot-umount