linux-kernel - Re: ls input/output error ("NFS: readdir(/) returns -5") on krb5 NFSv4 client using SHA2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <16121fbd-17d1-41c5-ac8d-177533e8afea@TylerWRoss.com>
Date: Tue, 18 Nov 2025 04:32:24 +0000
From: "Tyler W. Ross" <TWR@...erwross.com>
To: Scott Mayhew <smayhew@...hat.com>
Cc: Trond Myklebust <trondmy@...nel.org>, Chuck Lever <chuck.lever@...cle.com>, Anna Schumaker <anna@...nel.org>, Salvatore Bonaccorso <carnil@...ian.org>, "1120598@...s.debian.org" <1120598@...s.debian.org>, Jeff Layton <jlayton@...nel.org>, NeilBrown <neil@...wn.name>, Steve Dickson <steved@...hat.com>, Olga Kornievskaia <okorniev@...hat.com>, Dai Ngo <Dai.Ngo@...cle.com>, Tom Talpey <tom@...pey.com>, linux-nfs@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: ls input/output error ("NFS: readdir(/) returns -5") on krb5 NFSv4 client using SHA2

On 11/17/25 4:05 PM, Scott Mayhew wrote:
> On Mon, 17 Nov 2025, Tyler W. Ross wrote:
> 
>> Weird behavior I just discovered:
>>
>> Explicitly setting allowed-enctypes in the gssd section of /etc/nfs.conf
>> to exclude aes256-cts-hmac-sha1-96 makes both SHA2 ciphers work as
>> expected (assuming each is allowed).
>>
>> If allowed-enctypes is unset (letting gssd interrogate the kernel for
>> supported enctypes) or includes aes256-cts-hmac-sha1-96, then the XDR
>> overflow occurs.
>>
>> Non-working configurations (first is the commented-out default in nfs.conf):
>> allowed-enctypes=aes256-cts-hmac-sha384-192,aes128-cts-hmac-sha256-128,camellia256-cts-cmac,camellia128-cts-cmac,aes256-cts-hmac-sha1-96,aes128-cts-hmac-sha1-96
>> allowed-enctypes=aes256-cts-hmac-sha384-192,aes256-cts-hmac-sha1-96
>> allowed-enctypes=aes128-cts-hmac-sha256-128,aes256-cts-hmac-sha1-96
>> allowed-enctypes=aes256-cts-hmac-sha384-192,aes128-cts-hmac-sha256-128,aes256-cts-hmac-sha1-96
>>
>> Working configurations (first is default sans aes256-cts-hmac-sha1-96):
>> allowed-enctypes=aes256-cts-hmac-sha384-192,aes128-cts-hmac-sha256-128,camellia256-cts-cmac,camellia128-cts-cmac,aes128-cts-hmac-sha1-96
>> allowed-enctypes=aes256-cts-hmac-sha384-192,aes128-cts-hmac-sha256-128
>> allowed-enctypes=aes256-cts-hmac-sha384-192,aes128-cts-hmac-sha1-96
>> allowed-enctypes=aes128-cts-hmac-sha256-128,aes128-cts-hmac-sha1-96
>>
> 
> That doesn't really make sense.  You should only need to use the
> allowed-enctypes setting if you're talking to an NFS server that doesn't
> have support for the new encryption types.
> 
> It basically works like the "permitted_enctypes" option in krb5.conf,
> except it only affects NFS rather than affecting your krb5 configuration
> as a whole.

Agreed. It really doesn't make sense. It may just be me being confounded 
by some ancillary behavior I don't understand.

I find it especially strange that
allowed-enctypes=aes256-cts-hmac-sha384-192 works, but unset
allowed-enctypes with a manually acquired aes256-cts-hmac-sha384-192 
ticket doesn't work.

allowed-enctypes=aes256-cts-hmac-sha384-192 works both with an 
automatically acquired service ticket (kinit then ls) and a manually 
acquired service ticket (via kvno -e).

> Can you go back and re-do the tracepoint capture, except this time
> umount your NFS filessytems before starting the capture (i.e. perform
> the mount command while trace-cmd is running).  I'm curious what values
> the rpcgss_update_slack tracepoint shows.

Here are the 2 rpcgss_update_slack occurrences, with a couple lines of 
context. Let me know if you'd like the full report: it's ~1300 lines.

mount.nfs4-1043  [005] .....   190.746932: rpc_task_run_action:  task:00000002@...00001 flags=DYNAMIC|NO_ROUND_ROBIN|SOFT|SENT|TIMEOUT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_status
mount.nfs4-1043  [005] .....   190.746932: rpc_task_run_action:  task:00000002@...00001 flags=DYNAMIC|NO_ROUND_ROBIN|SOFT|SENT|TIMEOUT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_decode
mount.nfs4-1043  [005] .....   190.746933: rpc_xdr_recvfrom:     task:00000002@...00001 head=[0xffff8a61a2848fd4,4392] page=0(0) tail=[(nil),0] len=312
mount.nfs4-1043  [005] .....   190.746938: rpcgss_update_slack:  task:00000002@...00001 xid=0xb28269cc auth=0xffff8a6189400798 rslack=19 ralign=11 verfsize=9
mount.nfs4-1043  [005] .....   190.746939: rpc_task_run_action:  task:00000002@...00001 flags=DYNAMIC|NO_ROUND_ROBIN|SOFT|SENT|TIMEOUT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=rpc_exit_task
mount.nfs4-1043  [005] .....   190.746939: rpc_task_end:         task:00000002@...00001 flags=DYNAMIC|NO_ROUND_ROBIN|SOFT|SENT|TIMEOUT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=rpc_exit_task
mount.nfs4-1043  [005] .....   190.746940: rpc_stats_latency:    task:00000002@...00001 xid=0xb28269cc nfsv4 EXCHANGE_ID backlog=12836 rtt=136 execute=12995 xprt_id=1
--
mount.nfs4-1043  [002] .....   190.755687: rpc_task_run_action:  task:00000001@...00002 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_status
mount.nfs4-1043  [002] .....   190.755687: rpc_task_run_action:  task:00000001@...00002 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_decode
mount.nfs4-1043  [002] .....   190.755688: rpc_xdr_recvfrom:     task:00000001@...00002 head=[0xffff8a6182b4e6ac,2920] page=0(0) tail=[(nil),0] len=192
mount.nfs4-1043  [002] .....   190.755691: rpcgss_update_slack:  task:00000001@...00002 xid=0xb68269cc auth=0xffff8a6187759498 rslack=9 ralign=9 verfsize=9
mount.nfs4-1043  [002] .....   190.755694: rpc_task_run_action:  task:00000001@...00002 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=rpc_exit_task
mount.nfs4-1043  [002] .....   190.755694: rpc_task_end:         task:00000001@...00002 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=rpc_exit_task
mount.nfs4-1043  [002] .....   190.755694: rpc_stats_latency:    task:00000001@...00002 xid=0xb68269cc nfsv4 LOOKUP_ROOT backlog=7101 rtt=91 execute=7218 xprt_id=1


And here's with allowed-enctypes=aes256-cts-hmac-sha384-192

mount.nfs4-1100  [005] .....   580.221598: rpc_task_run_action:  task:00000002@...00001 flags=DYNAMIC|NO_ROUND_ROBIN|SOFT|SENT|TIMEOUT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_status
mount.nfs4-1100  [005] .....   580.221598: rpc_task_run_action:  task:00000002@...00001 flags=DYNAMIC|NO_ROUND_ROBIN|SOFT|SENT|TIMEOUT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_decode
mount.nfs4-1100  [005] .....   580.221598: rpc_xdr_recvfrom:     task:00000002@...00001 head=[0xffff8b2b98850fd4,4392] page=0(0) tail=[(nil),0] len=336
mount.nfs4-1100  [005] .....   580.221604: rpcgss_update_slack:  task:00000002@...00001 xid=0x4c050148 auth=0xffff8b2b88864818 rslack=25 ralign=14 verfsize=12
mount.nfs4-1100  [005] .....   580.221605: rpc_task_run_action:  task:00000002@...00001 flags=DYNAMIC|NO_ROUND_ROBIN|SOFT|SENT|TIMEOUT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=rpc_exit_task
mount.nfs4-1100  [005] .....   580.221606: rpc_task_end:         task:00000002@...00001 flags=DYNAMIC|NO_ROUND_ROBIN|SOFT|SENT|TIMEOUT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=rpc_exit_task
mount.nfs4-1100  [005] .....   580.221607: rpc_stats_latency:    task:00000002@...00001 xid=0x4c050148 nfsv4 EXCHANGE_ID backlog=13249 rtt=164 execute=13435 xprt_id=1
--
mount.nfs4-1100  [000] .....   580.230841: rpc_task_run_action:  task:00000001@...00002 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_status
mount.nfs4-1100  [000] .....   580.230841: rpc_task_run_action:  task:00000001@...00002 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=call_decode
mount.nfs4-1100  [000] .....   580.230841: rpc_xdr_recvfrom:     task:00000001@...00002 head=[0xffff8b2ba07b66ac,2920] page=0(0) tail=[(nil),0] len=204
mount.nfs4-1100  [000] .....   580.230845: rpcgss_update_slack:  task:00000001@...00002 xid=0x50050148 auth=0xffff8b2b88864b18 rslack=12 ralign=12 verfsize=12
mount.nfs4-1100  [000] .....   580.230847: rpc_task_run_action:  task:00000001@...00002 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=rpc_exit_task
mount.nfs4-1100  [000] .....   580.230847: rpc_task_end:         task:00000001@...00002 flags=MOVEABLE|DYNAMIC|SENT|NORTO|CRED_NOREF runstate=RUNNING|0x4 status=0 action=rpc_exit_task
mount.nfs4-1100  [000] .....   580.230848: rpc_stats_latency:    task:00000001@...00002 xid=0x50050148 nfsv4 LOOKUP_ROOT backlog=7760 rtt=98 execute=7878 xprt_id=1



TWR