linux-kernel - Re: [RFC RFC PATCH] mm: convert VM flags from macros to enum

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <fa2d7db5-688c-4d04-abcd-a60f79a6bb7a@lucifer.local>
Date: Mon, 13 Oct 2025 13:57:57 +0100
From: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
To: David Hildenbrand <david@...hat.com>
Cc: Ye Liu <ye.liu@...ux.dev>, Andrew Morton <akpm@...ux-foundation.org>,
        Ye Liu <liuye@...inos.cn>, "Liam R. Howlett" <Liam.Howlett@...cle.com>,
        Vlastimil Babka <vbabka@...e.cz>, Mike Rapoport <rppt@...nel.org>,
        Suren Baghdasaryan <surenb@...gle.com>, Michal Hocko <mhocko@...e.com>,
        linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [RFC RFC PATCH] mm: convert VM flags from macros to enum

On Mon, Oct 13, 2025 at 02:31:35PM +0200, David Hildenbrand wrote:
> On 13.10.25 13:33, Lorenzo Stoakes wrote:
> > On Mon, Oct 13, 2025 at 01:12:20PM +0200, David Hildenbrand wrote:
> > > On 13.10.25 13:04, Lorenzo Stoakes wrote:
> > > > On Sat, Oct 11, 2025 at 05:30:52PM +0800, Ye Liu wrote:
> > > > > From: Ye Liu <liuye@...inos.cn>
> > > > >
> > > > > Hello MM maintainers and drgn community,
> > > > >
> > > > > This RFC proposes to convert VM_* flags from #define macros to enum
> > > > > vm_flags. The motivation comes from recent drgn development where we
> > > > > encountered difficulties in implementing VM flag parsing due to the
> > > > > current macro-based approach.
> > > >
> > > > This isn't going to work sorry, it's not valid to have flag values as an enum
> > >
> > > I don't follow, can you elaborate? IIRC, the compiler will use an integer
> > > type to back the enum that will fit all values.
> >
> > switch (flags) {
> > 	case VAL1:
> > 	case VAL2:
> > 	etc.
> > }
> >
> > Is broken (compiler will say you cover all cases when you don't...)
>
> I assume you mean theoretically, because there is no such code, right?

Right, it's a general point about why enum's are not such a great idea for this.

>
> >
> > An enum implies independent values that exhaustively describe all state, however
> > these flag values are not that - they're intended to be bit fields.
> >
>
> Observe how we use an enum for FOLL_* flags, vm_fault_reason, fault_flag and
> probably other things.

FOLL_* flags are an anonymous enum, enum fault_flag is not used as a type
anywhere, nor is vm_fault_reason. So those are both kinda weird as to why we
even name the type (they're in effect anonymous).

But also 'we do X in the kernel' doesn't mean doing X is right :)

>
> But more importantly,
>
> enum pageflags { ... :)

Your reply answers this :)

>
> --
> Cheers
>
> David / dhildenb
>