[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-Id: <20230803202010.37149-1-nnac123@linux.ibm.com>
Date: Thu, 3 Aug 2023 15:20:06 -0500
From: Nick Child <nnac123@...ux.ibm.com>
To: netdev@...r.kernel.org
Cc: haren@...ux.ibm.com, ricklind@...ibm.com, danymadden@...ibm.com,
tlfalcon@...ux.ibm.com, bjking1@...ux.ibm.com,
Nick Child <nnac123@...ux.ibm.com>
Subject: [PATCH net 1/5] ibmvnic: Enforce stronger sanity checks on login response
Ensure that all offsets in a login response buffer are within the size
of the allocated response buffer. Any offsets or lengths that surpass
the allocation are likely the result of an incomplete response buffer.
In these cases, a full reset is necessary.
When attempting to login, the ibmvnic device will allocate a response
buffer and pass a reference to the VIOS. The VIOS will then send the
ibmvnic device a LOGIN_RSP CRQ to signal that the buffer has been filled
with data. If the ibmvnic device does not get a response in 20 seconds,
the old buffer is freed and a new login request is sent. With 2
outstanding requests, any LOGIN_RSP CRQ's could be for the older
login request. If this is the case then the login response buffer (which
is for the newer login request) could be incomplete and contain invalid
data. Therefore, we must enforce strict sanity checks on the response
buffer values.
Testing has shown that the `off_rxadd_buff_size` value is filled in last
by the VIOS and will be the smoking gun for these circumstances.
Until VIOS can implement a mechanism for tracking outstanding response
buffers and a method for mapping a LOGIN_RSP CRQ to a particular login
response buffer, the best ibmvnic can do in this situation is perform a
full reset.
Fixes: dff515a3e71d ("ibmvnic: Harden device login requests")
Signed-off-by: Nick Child <nnac123@...ux.ibm.com>
---
Hello!
This patchset is all relevant to recent bugs which came up regarding
the ibmvnic login process. Specifically, when this process times out.
ibmvnic devices are virtual devices which need to "login" to a physical
NIC at the end of its initialization process. This invloves sending a
command to the VIOS (virtual input output server, essentially the server
that this client is logging into) requesting it to fill out a DMA mapped
repsonse buffer. Once done, the VIOS sends a response informing the
client that the buffer has been filled with data.
If the VIOS does not send a response in 20 seconds then the client tries
again. If this happens then several bugs can occur. This is usually due
to the fact that there are more than one outstanding requests and no
mechanism for mapping a response CRQ to a given response buffer. Until
that mechanism is created, this patchset aims to harden this timeout
recovery process so that the device does not get stuck in an inopperable
state.
drivers/net/ethernet/ibm/ibmvnic.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c
index 763d613adbcc..996f8037c266 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -5397,6 +5397,7 @@ static int handle_login_rsp(union ibmvnic_crq *login_rsp_crq,
int num_rx_pools;
u64 *size_array;
int i;
+ u32 rsp_len;
/* CHECK: Test/set of login_pending does not need to be atomic
* because only ibmvnic_tasklet tests/clears this.
@@ -5447,6 +5448,23 @@ static int handle_login_rsp(union ibmvnic_crq *login_rsp_crq,
ibmvnic_reset(adapter, VNIC_RESET_FATAL);
return -EIO;
}
+
+ rsp_len = be32_to_cpu(login_rsp->len);
+ if (be32_to_cpu(login->login_rsp_len) < rsp_len ||
+ rsp_len <= be32_to_cpu(login_rsp->off_txsubm_subcrqs) ||
+ rsp_len <= be32_to_cpu(login_rsp->off_rxadd_subcrqs) ||
+ rsp_len <= be32_to_cpu(login_rsp->off_rxadd_buff_size) ||
+ rsp_len <= be32_to_cpu(login_rsp->off_supp_tx_desc)) {
+ /* This can happen if a login request times out and there are
+ * 2 outstanding login requests sent, the LOGIN_RSP crq
+ * could have been for the older login request. So we are
+ * parsing the newer response buffer which may be incomplete
+ */
+ dev_err(dev, "FATAL: Login rsp offsets/lengths invalid\n");
+ ibmvnic_reset(adapter, VNIC_RESET_FATAL);
+ return -EIO;
+ }
+
size_array = (u64 *)((u8 *)(adapter->login_rsp_buf) +
be32_to_cpu(adapter->login_rsp_buf->off_rxadd_buff_size));
/* variable buffer sizes are not supported, so just read the
--
2.39.3
Powered by blists - more mailing lists