linux-kernel - Re: [PATCH 2/9] IB: add a proper completion queue abstraction

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20151125062113.GA29187@obsidianresearch.com>
Date:	Tue, 24 Nov 2015 23:21:13 -0700
From:	Jason Gunthorpe <jgunthorpe@...idianresearch.com>
To:	Caitlin Bestler <cait@...mi.com>
Cc:	Bart Van Assche <bart.vanassche@...disk.com>,
	Christoph Hellwig <hch@....de>,
	"linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
	"sagig@....mellanox.co.il" <sagig@....mellanox.co.il>,
	"axboe@...com" <axboe@...com>,
	"linux-scsi@...r.kernel.org" <linux-scsi@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 2/9] IB: add a proper completion queue abstraction

On Tue, Nov 24, 2015 at 06:47:14PM -0800, Caitlin Bestler wrote:
>         Acknowledge packet for the last packet of the request has been
>         committed to the wire (including the appropriate fields for RDMA
>         READ response).

> The target side cannot generate a response before it receives the request.
> The target side verbs cannot generate the completion before the acknowledge
> packet has been
> committed to the wire.

Sure, but, I keep saying this, the responder behavior is largely
irrelevant to what the target is able/required to do.

> Therefore the initiating side cannot receive a response before the write
> operation has completed.

Wrong. The ladder diagram would be

Requestor          Responder           Responder Verbs

SEND1 ----------->   Process
         X---------  ACK (lost)
	                     --------> recv1 CQ
			     <-------   send2 WQE
recv2 CQE <--------  SEND2 Packet
[..]
send1 CQE <--------  ACK (resent)

The Ack may be lost, causing the send CQE to arrive after the recv
CQE, even though the responder did everything in a specific order.

The fundamental issue is that the responder cannot detect the lost
ACK. The PSN of the ACK packet is part of the Requestor's PSN space,
not part of the Responders:

 9.7.5.1.1 GENERATING PSNs FOR ACKNOWLEDGE MESSAGES

 C9-95: For responses to SEND requests or RDMA WRITE requests the
  responder shall insert in the PSN field of the response the PSN of the
  most recent request packet being acknowledged.

Or stated another way, the value of the AckReq bit in SEND1 has no
impact on the contents of the SEND2 packet - thus there is no way for
the requestor to detect the loss of the ACK and hold off delivering
recv2 CQE.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/