linux-kernel - Re: [RFC v1 04/14] bus1: util

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CANq1E4Qmro83PSdfxpLkbMf0gEzsi=+OpJTVm9PiCD2fOaYoEw@mail.gmail.com>
Date:   Thu, 27 Oct 2016 15:31:55 +0200
From:   David Herrmann <dh.herrmann@...il.com>
To:     Arnd Bergmann <arnd@...db.de>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        linux-kernel <linux-kernel@...r.kernel.org>,
        Andy Lutomirski <luto@...capital.net>,
        Jiri Kosina <jikos@...nel.org>, Greg KH <greg@...ah.com>,
        Hannes Reinecke <hare@...e.com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Tom Gundersen <teg@...m.no>,
        Josh Triplett <josh@...htriplett.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: [RFC v1 04/14] bus1: util - fixed list utility library

Hi

On Thu, Oct 27, 2016 at 2:56 PM, Arnd Bergmann <arnd@...db.de> wrote:
> On Thursday, October 27, 2016 2:48:46 PM CEST David Herrmann wrote:
>> On Thu, Oct 27, 2016 at 2:37 PM, Peter Zijlstra <peterz@...radead.org> wrote:
>> > On Wed, Oct 26, 2016 at 09:18:00PM +0200, David Herrmann wrote:
>> >> +             e = kmalloc_array(sizeof(*e), BUS1_FLIST_BATCH + 1, gfp);
>> >
>> >> +#define BUS1_FLIST_BATCH (1024)
>> >
>> >> +struct bus1_flist {
>> >> +     union {
>> >> +             struct bus1_flist *next;
>> >> +             void *ptr;
>> >> +     };
>> >> +};
>> >
>> > So that's an allocation of 8*(1024+1), or slightly more than 2 pages.
>> >
>> > kmalloc will round up to the next power of two, so you'll end up with an
>> > allocation of 16*1024, wasting a whopping 8184 bytes per such allocation
>> > in slack space.
>> >
>> > Please consider using 1023 or something for your batch size, 511 would
>> > get you to exactly 1 page which would be even better.
>>
>> Thanks for the hint! 511 looks like the obvious choice. Maybe even
>> (PAGE_SIZE / sizeof(long) - 1). I will put a comment next to the
>> definition.
>>
>>
>
> PAGE_SIZE can be up to 64KB though, so that might lead wasting a lot
> of memory.

The bus1-flist implementation never over-allocates. It is a fixed size
list, so it only allocates as much memory as needed. The issue PeterZ
pointed out is passing suitable sizes to kmalloc(), which internally
over-allocates to power-of-2 bounds (or some similar bounds). So we
only ever waste space here if kmalloc() internally rounds up. The code
in bus1-flist allocates exactly the needed space.

Thanks
David