linux-kernel - Re: [PATCH 2/2] IB/mad: Use IDR for agent IDs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20180612075942.00005061@dev.mellanox.co.il>
Date:   Tue, 12 Jun 2018 07:59:42 +0300
From:   jackm <jackm@....mellanox.co.il>
To:     Jason Gunthorpe <jgg@...lanox.com>
Cc:     Leon Romanovsky <leon@...nel.org>,
        Matthew Wilcox <willy@...radead.org>,
        hans.westgaard.ry@...cle.com, Doug Ledford <dledford@...hat.com>,
        Matthew Wilcox <mawilcox@...rosoft.com>,
        linux-rdma@...r.kernel.org,
        Håkon Bugge <haakon.bugge@...cle.com>,
        Parav Pandit <parav@...lanox.com>,
        Pravin Shedge <pravin.shedge4linux@...il.com>,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH 2/2] IB/mad: Use IDR for agent IDs

On Mon, 11 Jun 2018 10:19:18 -0600
Jason Gunthorpe <jgg@...lanox.com> wrote:

> On Mon, Jun 11, 2018 at 09:19:14AM +0300, jackm wrote:
> > On Sun, 10 Jun 2018 22:42:03 -0600
> > Jason Gunthorpe <jgg@...lanox.com> wrote:
> >   
> > > Er, the spec has nothing to do with this. In Linux the TID is made
> > > unique because the core code provides 32 bits that are unique and
> > > the user provides another 32 bits that are unique. The driver
> > > cannot change any of those bits without risking non-uniquenes,
> > > which is exactly the bug mlx4 created when it stepped outside its
> > > bounds and improperly overrode bits in the TID for its own
> > > internal use.  
> > 
> > Actually, the opposite is true here.  When SRIOV is active, each VM
> > generates its *own* TIDs -- with 32 bits of agent number and 32 bits
> > of counter.  
> 
> And it does it while re-using the LRH of the host, so all VMs and the
> host are now forced to share a TID space, yes I know.
> 
> > There is a chance that two different VMs can generate the same TID!
> > Encoding the slave (VM) number in the packet actually guarantees
> > uniqueness here.  
> 
> Virtualizing the TID in the driver would be fine, but it must
> virtualize all the TIDs (even those generated by the HOST).

It DOES do so.  The host slave id is 0. Slave numbers start with 1.
If the MS byte contains a zero after paravirtualization, the MAD
was sent by the host.
In fact, ALL mads are paravirtualized -- including those to/from the host.

> 
> Just blindly assuming the host doesn't generate TID's that overlap
> with the virtualization process is a bug.
> 
Not the case, host mads are also paravirtualized.

> > There is nothing wrong with modifying the TID in a reversible way in
> > order to: a. guarantee uniqueness b. identify the VM which should
> > receive the response packet  
> 
> Sure, as long as *all* TID's sharing a LRH are vitalized like this.
> 
> > The problem was created when the agent-id numbers started to use the
> > most-significant byte (thus making the MSB slave-id addition
> > impossible).  
> 
> It hasn't always been this way? What commit?
> 
Commit: 37bfc7c1e83f1 ("IB/mlx4: SR-IOV multiplex and demultiplex MADs"):

Code snippet which replaces the MS byte is below (file drivers/infiniband/hw/mlx4/mad.c,
procedure mlx4_ib_multiplex_mad() ).
Just an aside:  Oracle noted the problem on the *host*: The host's message log
contained the warning issued on line 1529, with slave=0 (which is the hypervisor).

37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1519)  switch (tunnel->mad.mad_hdr.method) {
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1520)  case IB_MGMT_METHOD_SET:
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1521)  case IB_MGMT_METHOD_GET:
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1522)  case IB_MGMT_METHOD_REPORT:
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1523)  case IB_SA_METHOD_GET_TABLE:
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1524)  case IB_SA_METHOD_DELETE:
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1525)  case IB_SA_METHOD_GET_MULTI:
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1526)  case IB_SA_METHOD_GET_TRACE_TBL:
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1527)          slave_id = (u8 *) &tunnel->mad.mad_hdr.tid;
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1528)          if (*slave_id) {
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1529)                  mlx4_ib_warn(ctx->ib_dev, "egress mad has non-null tid msb:%d "
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1530)                               "class:%d slave:%d\n", *slave_id,
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1531)                               tunnel->mad.mad_hdr.mgmt_class, slave);
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1532)                  return;
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1533)          } else
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1534)                  *slave_id = slave;
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1535)  default:
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1536)          /* nothing */;
37bfc7c1e (Jack Morgenstein            2012-08-03 08:40:44 +0000 1537)  }

> Jason