lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Fri, 17 Dec 2010 15:13:30 -0800 From: Dimitris Michailidis <dm@...lsio.com> To: Matt Domsch <Matt_Domsch@...l.com> CC: Eilon Greenstein <eilong@...adcom.com>, Dmitry Kravkov <dmitry@...adcom.com>, "davem@...emloft.net" <davem@...emloft.net>, "netdev@...r.kernel.org" <netdev@...r.kernel.org>, "narendra_k@...l.com" <narendra_k@...l.com>, "jordan_hargrave@...l.com" <jordan_hargrave@...l.com> Subject: Re: [PATCH net-next] bnx2x: Add Nic partitioning mode (57712 devices) Matt Domsch wrote: > On Thu, Dec 09, 2010 at 04:49:25PM +0200, Eilon Greenstein wrote: >> On Mon, 2010-12-06 at 10:21 -0800, Dimitris Michailidis wrote: >>> Matt Domsch wrote: >> ... >>> /sys/class/net/<ifname>/dev_id indicates the physical port <ifname> is >>> associated with. At least a few drivers set up dev_id this way. >>> >>> >> So we are on agreement? This can satisf all needs? If so, we will add >> this scheme to the bnx2x as well. > > I don't think that's enough. Necessary, but not sufficient. > > If dev_id is a field that starts over with each PCI device (e.g. is > used to distinguish multiple ports that share the same PCI > device), that's enough to handle the Chelsio case, but not the NPAR & > SR-IOV case. My understanding is that dev_id indicates the physical port of the card associated with an interface. It does not reset when you move to a new function of the device. > > If the above is true, then a value of dev_id=0 for all 1:1 PCI Device > : Port relations is fine, leaving the three drivers that set dev_id > non-zero are all multi-port, single PCI device controllers. > > cxgb4/t4_hw.c: adap->port[i]->dev_id = j; The HW cxgb4 deals with is multi-function (actually the driver uses primarily function 4 nowadays) but it's virtualizable and the association between functions and ports very flexible. For example, you may have a 2-port card but maybe the driver will be given just (a slice of) port 1. So the driver will create one netdev with dev_id==1 and there won't be anything with dev_id 0. You cannot determine this by looking at anything PCI-related or any static table. For this driver you can get two pieces of information for an interface: - /sys/class/net/<interface>/device points to the PCI function handling the interface - /sys/class/net/<interface>/dev_id indicates the physical port of the interface You can have several interfaces with same device link and different dev_id. While the current driver doesn't do it you could also have several interfaces with different device links but same dev_id (NPAR situation, notice again that dev_ids are not per PCI function), or interfaces with different device and dev_id, or even interfaces with same device and dev_id. > mlx4/en_netdev.c: dev->dev_id = port - 1; > sfc/siena.c: efx->net_dev->dev_id = EFX_OWORD_FIELD(reg, FRF_CZ_CS_PORT_NUM) - 1; > > Is that truly how these three controllers work: they set dev_id when > there are multiple physical ports that a single PCI d/b/d/f drives? > > My naming convention of: > pci<slot>#<port> > wants to express this relationship. If I have a card with 2 PCI > devices, and 2 physical ports on each device, I have 4 ports to > describe. The dev_ids would look like: 0,1 0,1 , so I can't use that > value directly. I think they'd be 0,1,2,3 for drivers that set dev_id and 0,0,0,0 otherwise. I can make a list of PCI devices on the same card, > look at the dev_id field of each, and run a counter: > > for each slot: > int port=1; > for each pci device: > for each in net/<interface>/dev_id: > use name pci<slot>#<port> > port++ > > OK? Can someone with such a card send me tree /sys, so I can see the > tree does really look like I expect: > > /sys/devices/pci0000:00/0000:00:1c.0/0000:0b:00.0/net/eth0/dev_id = 0 > /sys/devices/pci0000:00/0000:00:1c.0/0000:0b:00.0/net/eth1/dev_id = 1 > > simply finding a net/ subdir under a PCI device, each of the > directories in net/ are interface names, with different dev_id values. This would be the common case but in general the dev_ids don't need to be consecutive or start at 0, nor does a particular dev_id need to appear just once. > Now for the partitioned devices (NPAR or SR-IOV). Here, we have > multiple PCI devices mapped to the same port. > > My naming convention of: > pci<slot>#<port>_<partition> > wants to express this relationship. > > I need a way to express which port a given partition maps to. I'm > also presuming this is a static mapping right now, that it won't > change around during runtime (ala Xsigo, which I have no solution here > for; if the mapping isn't static, this is going to get trickier). > > As dev_ids are only unique per PCI device, we would need a pointer to > the "base" device. However, in the Broadcom 57712 case, there is no > such "base" device. :-( So, using dev_id here doesn't seem like the > right approach for these devices. dev_ids can handle NPAR but I do understand that dev_id 0 is ambiguous. Two functions with dev_id 0 mean one thing for a driver that sets dev_id and a very different thing for one that doesn't. > What if we did something like this? > > /sys/devices/net_ports/port0/ > /sys/devices/pci0000:00/0000:00:1c.0/0000:0b:00.0/net/eth0/port -> > /../../../../../net_ports/port0 > /sys/devices/pci0000:00/0000:00:1c.0/0000:0b:00.1/net/eth1/port -> > /../../../../../net_ports/port0 > > > In this case, the port0 "name" is simply a way to group interfaces > into ports, it's not how ports are labeled on the chassis. If I understand you right a "port" is a group of interfaces sharing one physical port without saying which one. I think dev_id does the same and specifies which physical port. > > Do network drivers know how many ports they have? > What are the characteristics of network ports? Ideally, physical > location (PCI slot), and index within that physical location. This index is the dev_id for drivers that set it. > These > right now I'm deriving from SMBIOS and PCI, and if not explicitly > exposed, counting devices on the same slot and assigning port numbers > that way, but I would love to have explicit information from the > drivers. > > Thoughts? > > Thanks, > Matt > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists