linux-kernel - Re: [PATCH net] selftests: netdevsim: Fix ethtool-features.sh fail

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <aQnd1HQ-b7wFI2WP@krikkit>
Date: Tue, 4 Nov 2025 12:04:52 +0100
From: Sabrina Dubroca <sd@...asysnail.net>
To: Jakub Kicinski <kuba@...nel.org>
Cc: Wang Liang <wangliang74@...wei.com>, andrew@...n.ch,
	davem@...emloft.net, edumazet@...gle.com, pabeni@...hat.com,
	shuah@...nel.org, horms@...nel.org, netdev@...r.kernel.org,
	linux-kselftest@...r.kernel.org, linux-kernel@...r.kernel.org,
	yuehaibing@...wei.com, zhangchangzhong@...wei.com
Subject: Re: [PATCH net] selftests: netdevsim: Fix ethtool-features.sh fail

2025-11-03, 16:01:33 -0800, Jakub Kicinski wrote:
> On Mon, 3 Nov 2025 11:13:08 +0100 Sabrina Dubroca wrote:
> > 2025-10-30, 17:02:17 -0700, Jakub Kicinski wrote:
> > > On Fri, 31 Oct 2025 00:13:59 +0100 Sabrina Dubroca wrote:  
> > > > I guess it's improving the situation, but I've got a system with an
> > > > ethtool that accepts the --json argument, but silently ignores it for
> > > >  -k (ie `ethtool --json -k $DEV` succeeds but doesn't produce a json
> > > > output), which will still cause the test to fail later.  
> > > 
> > > And --json was added to -k in Jan 2022, that's pretty long ago.
> > > I'm not sure we need this aspect of the patch at all..  
> > 
> > Ok.  Then maybe a silly idea: for the tests that currently have some
> > form of "$TOOL is too old" check, do we want to remove those after a
> > while? If so, how long after the feature was introduced in $TOOL?
> > 
> > Or should we leave them, but not accept new checks to exclude
> > really-old versions of tools?  Do we need to document the cut-off ("we
> > don't support tool versions older than 2 years for networking
> > selftests" [or similar]) somewhere in Documentation/ ?
> 
> FWIW my current thinking is to prioritize test development and kernel
> needs over the ability to run ksft on random old set of tools and have
> clean skips. IOW avoid complicating writing tests by making the author
> also responsible for testing versions of all tools.

I see. I liked Andrew's idea ("embed the date the requirement was
added into the test"), but it goes completely in the opposite
direction.

Figuring out why exactly a test failed in case of an old tool
(unexpected output passed to some pipe/parsing, exit with a non-zero
code, maybe other issues) is not always obvious. So without version
checks on the tools, I think we have to assume that the test requires
the latest version of all tools it calls (or at least a very recent
one). Which I guess is reasonable for upstream kernel development.

> The list of tools which need to be updated or installed for all
> networking tests to pass is rather long. My uneducated guess
> is all these one off SKIP patches don't amount to much. Here for
> example author is fixing one test, I'm pretty sure that far more
> tests depend on -k --json.

A quick grep found only a few more (in python scripts under
drivers/net) for -k. But (also from a quick grep) many tests seem to
use jq without checking that the command is present.

So I guess you would lean toward not accepting any such patch, not
requiring new tests to have SKIP checks, but leaving any existing
checks in? (and I suspect removing all the existing ones wouldn't
actually reduce the flow of "add check for too old $tool" patches, so
it probably doesn't make sense to do that)

-- 
Sabrina