netdev - Re: [PATCH net v2] s390/ism: fix concurrency management in ism

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250723141011.GB2459@horms.kernel.org>
Date: Wed, 23 Jul 2025 15:10:11 +0100
From: Simon Horman <horms@...nel.org>
To: Alexandra Winter <wintera@...ux.ibm.com>
Cc: David Miller <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>,
	Paolo Abeni <pabeni@...hat.com>, Eric Dumazet <edumazet@...gle.com>,
	Andrew Lunn <andrew+netdev@...n.ch>, netdev@...r.kernel.org,
	linux-s390@...r.kernel.org, Heiko Carstens <hca@...ux.ibm.com>,
	Vasily Gorbik <gor@...ux.ibm.com>,
	Alexander Gordeev <agordeev@...ux.ibm.com>,
	Christian Borntraeger <borntraeger@...ux.ibm.com>,
	Sven Schnelle <svens@...ux.ibm.com>,
	Thorsten Winkler <twinkler@...ux.ibm.com>,
	Halil Pasic <pasic@...ux.ibm.com>,
	Aliaksei Makarau <Aliaksei.Makarau@....com>,
	Mahanta Jambigi <mjambigi@...ux.ibm.com>
Subject: Re: [PATCH net v2] s390/ism: fix concurrency management in ism_cmd()

On Tue, Jul 22, 2025 at 06:18:17PM +0200, Alexandra Winter wrote:
> From: Halil Pasic <pasic@...ux.ibm.com>
> 
> The s390x ISM device data sheet clearly states that only one
> request-response sequence is allowable per ISM function at any point in
> time.  Unfortunately as of today the s390/ism driver in Linux does not
> honor that requirement. This patch aims to rectify that.
> 
> This problem was discovered based on Aliaksei's bug report which states
> that for certain workloads the ISM functions end up entering error state
> (with PEC 2 as seen from the logs) after a while and as a consequence
> connections handled by the respective function break, and for future
> connection requests the ISM device is not considered -- given it is in a
> dysfunctional state. During further debugging PEC 3A was observed as
> well.
> 
> A kernel message like
> [ 1211.244319] zpci: 061a:00:00.0: Event 0x2 reports an error for PCI function 0x61a
> is a reliable indicator of the stated function entering error state
> with PEC 2. Let me also point out that a kernel message like
> [ 1211.244325] zpci: 061a:00:00.0: The ism driver bound to the device does not support error recovery
> is a reliable indicator that the ISM function won't be auto-recovered
> because the ISM driver currently lacks support for it.
> 
> On a technical level, without this synchronization, commands (inputs to
> the FW) may be partially or fully overwritten (corrupted) by another CPU
> trying to issue commands on the same function. There is hard evidence that
> this can lead to DMB token values being used as DMB IOVAs, leading to
> PEC 2 PCI events indicating invalid DMA. But this is only one of the
> failure modes imaginable. In theory even completely losing one command
> and executing another one twice and then trying to interpret the outputs
> as if the command we intended to execute was actually executed and not
> the other one is also possible.  Frankly, I don't feel confident about
> providing an exhaustive list of possible consequences.
> 
> Fixes: 684b89bc39ce ("s390/ism: add device driver for internal shared memory")
> Reported-by: Aliaksei Makarau <Aliaksei.Makarau@....com>
> Tested-by: Mahanta Jambigi <mjambigi@...ux.ibm.com>
> Tested-by: Aliaksei Makarau <Aliaksei.Makarau@....com>
> Signed-off-by: Halil Pasic <pasic@...ux.ibm.com>
> Reviewed-by: Alexandra Winter <wintera@...ux.ibm.com>
> Signed-off-by: Alexandra Winter <wintera@...ux.ibm.com>

Thanks for the update.

Reviewed-by: Simon Horman <horms@...nel.org>