Date:   Mon, 4 Oct 2021 12:18:59 +0000
From:   David Laight <David.Laight@...LAB.COM>
To:     "'Cufi, Carles'" <Carles.Cufi@...dicsemi.no>,
        'Florian Weimer' <fweimer@...hat.com>
CC:     "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "jukka.rissanen@...ux.intel.com" <jukka.rissanen@...ux.intel.com>,
        "johan.hedberg@...el.com" <johan.hedberg@...el.com>,
        "Lubos, Robert" <Robert.Lubos@...dicsemi.no>,
        "Bursztyka, Tomasz" <tomasz.bursztyka@...el.com>,
        "linux-toolchains@...r.kernel.org" <linux-toolchains@...r.kernel.org>
Subject: RE: Non-packed structures in IP headers

From: Cufi, Carles
> Sent: 04 October 2021 11:42
> 
> > From: David Laight <David.Laight@...LAB.COM>
> > Sent: 02 October 2021 17:55
> > From: Florian Weimer
> > > Sent: 01 October 2021 21:10
> > >
> > > * Carles Cufi:
> > >
> > > > I was looking through the structures for IPv{4,6} packet headers and
> > > > noticed that several of those that seem to be used to parse a packet
> > > > directly from the wire are not declared as packed. This surprised me
> > > > because, although I did find that provisions are made so that the
> > > > alignment of the structure, it is still technically possible for the
> > > > compiler to inject padding bytes inside those structures, since
> > > > AFAIK the C standard makes no guarantees about padding unless it's
> > > > instructed to pack the structure.
> > >
> > > The C standards do not make such guarantees, but the platform ABI
> > > standards describe struct layout and ensure that there is no padding.
> > > Linux relies on that not just for networking, but also for the
> > > userspace ABI, support for separately compiled kernel modules, and in
> > > other places.
> >
> > In particular structures are used to map hardware register blocks.
> 
> Non-padded ones? Because this again might be an issue depending on the compiler/ABI as per my
> understanding.

The ABI are usually 'sane' - at the time of writing.
So you can assume that the compiler will only add padding if it is
necessary to get a field on its correct alignment.
The most common issue is whether the ABI specifies 32bit or 64bit
alignment for 64bit items.

Most hardware register layouts are all of 'words' (usually 32bit).
So there is unlikely to be any alignment padding.
(But there may be unused/undocumented registers.)
The advantage of using a C structure (rather than constants) to
define a hardware register layout is that is much harder to end
up using offsets for the wrong register block.

> > > Sometimes there are alignment concerns in the way these structs are
> > > used, but I believe the kernel generally controls placement of the
> > > data that is being worked on, so that does not matter, either.
> > >
> > > Therefore, I do not believe this is an actual problem.
> >
> > And adding __packed forces the compiler to do byte accesses (with shifts)
> > on cpu that don't support misaligned memory accesses.
> 
> Right, I understand that using __packed involves a runtime penalty hit on memory accesses, but I
> wasn't proposing to pack those structs, just to check their size for padding at compile-time.

That is sensible, but not really needed if the structure is short
and everything is 'naturally aligned'.

> > So it really is wrong to specify __packed unless the structure can be
> > unaligned in memory, or has a 'broken' definition that has fields that
> > aren't 'naturally aligned'.
> 
> But who defines what "broken" or "natural alignment" is for each architecture? Furthermore, the C
> standard doesn't really mention any of these terms as far as I know, it just leaves complete freedom
> to the compiler to add the padding it might consider appropriate.

'natural alignment' means that a 2**n byte item is always a
multiple of 2**n mytes from the front.

> > In the latter case it is enough to mark the field that requires the
> > padding before it removed as (IIRC) __aligned(1).
> > The compiler will then remove the padding but still assume the field is
> > partially aligned - so my do two 32bit access instead of 8 8bit ones).
> 
> Interesting, I have never seen this used before, but then again I come from an (deeply) embedded
> background that tends to pack all of its structs that represent bytes that go over the wire (or
> hardware registers for that matter).

Hmmm, I get you are using constant offsets not C struts for register layouts.
Either that or you've not actually looked at the object code.

> What is also interesting in this case, is why the ethernet header is packed in Linux[2]. There don't
> seem to be any special alignment or size constraints when compared to the IP headers, since the single
> 16-bit integer is placed 12 bytes after the beginning of the struct. I assume the reason for packing
> this header is indeed alignment, and not padding. Unlike the IP headers, the kernel probably doesn't
> ensure that an ethernet header begins at an address compatible with the alignment requirements of that
> 16-bit integer, so the header needs packing not because there's a risk of padding being introduced by
> the compiler, but because the header might start at an odd memory address?

Almost certainly because the base address might be odd.
(Although odd is actually unlikely.)

Ethernet frames are annoying.
They were 'designed' before anyone thought about 32bit cpus.
So they have two 6-byte addresses followed by a 2-byte 'ethertype'
(or a 2 byte length for ISO frames).
This is then followed by the IP header - which is all 32bit fields.
So if you want the IP header aligned (which is better - even on x86)
you need to rean the ethernet frame to a 4n+2 byte boundary.

It is surprising how many systems that have cpu that require 32bit
items be on 32bit boundaries (crossing page boundaries is a PITA)
have ethernet controllers that require receive buffers be 32bit
aligned.
Even writing two bytes on junk would help.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Powered by blists - more mailing lists