linux-kernel - Re: [PATCH] RDMA/mlx4: Spread completion vectors for proxy CQs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <A7FA2738-ACD2-42C7-A0AF-4343583CF87E@oracle.com>
Date:   Wed, 20 Feb 2019 18:46:47 +0100
From:   Håkon Bugge <haakon.bugge@...cle.com>
To:     Jason Gunthorpe <jgg@...pe.ca>
Cc:     Chuck Lever <chuck.lever@...cle.com>,
        Yishai Hadas <yishaih@...lanox.com>,
        Doug Ledford <dledford@...hat.com>, jackm@....mellanox.co.il,
        majd@...lanox.com, OFED mailing list <linux-rdma@...r.kernel.org>,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] RDMA/mlx4: Spread completion vectors for proxy CQs

> On 20 Feb 2019, at 18:14, Jason Gunthorpe <jgg@...pe.ca> wrote:
> 
> On Tue, Feb 19, 2019 at 06:32:50PM +0100, Håkon Bugge wrote:
>>   Anyway, Jason mentioned in a private email that maybe we could use the
>>   new completion API or something? I am not familiar with that one
>>   (yet).
> 
> I was thinking of the stuff in core/cq.c - but it also doesn't have
> automatic comp_vector balancing. It is the logical place to put
> something like that though..
> 
> An API to manage a bundle of CPU affine CQ's is probably what most
> ULPs really need.. (it makes little sense to create a unique CQ for
> every QP)

ULPs behave way differently. E.g. RDS creates one tx and one rx CQ per QP.

As I wrote earlier, we do not have any modify_cq() that changes the comp_vector (EQ association). We can balance #CQ associated with the EQs, but we do not know their behaviour.

So, assume 2 completion EQs, and four CQs. CQa and CQb are associated with the first EQ, the two others with the second EQ. That's the "best" we can do. But, if CQa and CQb are the only ones generating events, we will have all interrupt processing on a single CPU. But if we now could modify CQa.comp_vector to be that of the second EQ, we could achieve balance. But not sure if the drivers are able to do this at all.

> alloc_bundle()

You mean alloc a bunch of CQs? How do you know their #cqes and cq_context?

Håkon

> get_cqn_for_flow(bundle)
> alloc_qp()
> destroy_qp()
> put_cqn_for_flow(bundle)
> destroy_bundle();
> 
> Let the core code balance the cqn's and allocate (shared) CQ
> resources.
> 
> Jason