netdev - Re: [PATCH net v3] rds: ib: Increment i_fastreg

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <1BC1161D-3835-4856-B171-850073C0B6F7@oracle.com>
Date: Thu, 11 Sep 2025 13:26:03 +0000
From: Haakon Bugge <haakon.bugge@...cle.com>
To: Allison Henderson <allison.henderson@...cle.com>
CC: "horms@...nel.org" <horms@...nel.org>,
        "edumazet@...gle.com"
	<edumazet@...gle.com>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "kuba@...nel.org" <kuba@...nel.org>,
        "pabeni@...hat.com" <pabeni@...hat.com>,
        "rds-devel@....oracle.com" <rds-devel@....oracle.com>,
        "stable@...r.kernel.org" <stable@...r.kernel.org>,
        "netdev@...r.kernel.org"
	<netdev@...r.kernel.org>,
        OFED mailing list <linux-rdma@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH net v3] rds: ib: Increment i_fastreg_wrs before bailing
 out

Hi Allison,


> On 10 Sep 2025, at 20:47, Allison Henderson <allison.henderson@...cle.com> wrote:
> 
> On Wed, 2025-09-10 at 13:04 +0200, Håkon Bugge wrote:
>> We need to increment i_fastreg_wrs before we bail out from
>> rds_ib_post_reg_frmr().
>> 
>> We have a fixed budget of how many FRWR operations that can be
>> outstanding using the dedicated QP used for memory registrations and
>> de-registrations. This budget is enforced by the atomic_t
>> i_fastreg_wrs. If we bail out early in rds_ib_post_reg_frmr(), we will
>> "leak" the possibility of posting an FRWR operation, and if that
>> accumulates, no FRWR operation can be carried out.
> Hi Håkon,
> 
> This sounds much clearer, thank you!
> 
>> 
>> Fixes: 1659185fb4d0 ("RDS: IB: Support Fastreg MR (FRMR) memory registration mode")
>> Fixes: 3a2886cca703 ("net/rds: Keep track of and wait for FRWR segments in use upon shutdown")
>> Cc: stable@...r.kernel.org
>> Signed-off-by: Håkon Bugge <haakon.bugge@...cle.com>
>> 
>> ---
>> 
>> v2 -> v3:
>>   * Amended commit message
>>   * Removed indentation of this section
>>   * Fixing error path from ib_post_send()
>> 
>> v1 -> v2: Added Cc: stable@...r.kernel.org
>> ---
>> net/rds/ib_frmr.c | 20 ++++++++++++--------
>> 1 file changed, 12 insertions(+), 8 deletions(-)
>> 
>> diff --git a/net/rds/ib_frmr.c b/net/rds/ib_frmr.c
>> index 28c1b00221780..395a99b5a65ca 100644
>> --- a/net/rds/ib_frmr.c
>> +++ b/net/rds/ib_frmr.c
>> @@ -133,12 +133,15 @@ static int rds_ib_post_reg_frmr(struct rds_ib_mr *ibmr)
>> 
>> ret = ib_map_mr_sg_zbva(frmr->mr, ibmr->sg, ibmr->sg_dma_len,
>> &off, PAGE_SIZE);
>> - if (unlikely(ret != ibmr->sg_dma_len))
>> - return ret < 0 ? ret : -EINVAL;
>> + if (unlikely(ret != ibmr->sg_dma_len)) {
>> + ret = ret < 0 ? ret : -EINVAL;
>> + goto out_inc;
>> + }
>> 
>> - if (cmpxchg(&frmr->fr_state,
>> -     FRMR_IS_FREE, FRMR_IS_INUSE) != FRMR_IS_FREE)
>> - return -EBUSY;
>> + if (cmpxchg(&frmr->fr_state, FRMR_IS_FREE, FRMR_IS_INUSE) != FRMR_IS_FREE) {
>> + ret = -EBUSY;
>> + goto out_inc;
>> + }
>> 
>> atomic_inc(&ibmr->ic->i_fastreg_inuse_count);
>> 
>> @@ -166,11 +169,10 @@ static int rds_ib_post_reg_frmr(struct rds_ib_mr *ibmr)
>> /* Failure here can be because of -ENOMEM as well */
>> rds_transition_frwr_state(ibmr, FRMR_IS_INUSE, FRMR_IS_STALE);
>> 
>> - atomic_inc(&ibmr->ic->i_fastreg_wrs);
>> if (printk_ratelimit())
>> pr_warn("RDS/IB: %s returned error(%d)\n",
>> __func__, ret);
>> - goto out;
> Just one nit:  This was the only place the out label was used isnt it?  If so, lets go ahead and clear it out.  

That is indeed true!

> 
>> + goto out_inc;
>> }
>> 
>> /* Wait for the registration to complete in order to prevent an invalid
>> @@ -178,9 +180,11 @@ static int rds_ib_post_reg_frmr(struct rds_ib_mr *ibmr)
>>  * being accessed while registration is still pending.
>>  */
>> wait_event(frmr->fr_reg_done, !frmr->fr_reg);
>> -
>> out:
> With that fixed, you can go ahead and add my rvb:
> Reviewed-by: Allison Henderson <allison.henderson@...cle.com>

Will do, thanks for the r-b!


Håkon

> 
> Thank you!
> 
>> + return ret;
>> 
>> +out_inc:
>> + atomic_inc(&ibmr->ic->i_fastreg_wrs);
>> return ret;
>> }