linux-kernel - Re: LTP: shmget02 fails on compat mode - 64-bit kernel and 32-bit userspace

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <9677ced3-8386-47f0-a2d6-de27ec84207e@app.fastmail.com>
Date:   Fri, 19 May 2023 12:57:30 +0200
From:   "Arnd Bergmann" <arnd@...db.de>
To:     "Naresh Kamboju" <naresh.kamboju@...aro.org>,
        "open list" <linux-kernel@...r.kernel.org>,
        "LTP List" <ltp@...ts.linux.it>, lkft-triage@...ts.linaro.org,
        "Liam R. Howlett" <Liam.Howlett@...cle.com>,
        "Davidlohr Bueso" <dbueso@...e.de>,
        "Manfred Spraul" <manfred@...orfullife.com>
Cc:     "Ard Biesheuvel" <ardb@...nel.org>,
        "Dan Carpenter" <dan.carpenter@...aro.org>,
        "Anders Roxell" <anders.roxell@...aro.org>,
        chrubis <chrubis@...e.cz>, "Petr Vorel" <pvorel@...e.cz>,
        "Martin Doucha" <mdoucha@...e.cz>
Subject: Re: LTP: shmget02 fails on compat mode - 64-bit kernel and 32-bit userspace

On Fri, May 19, 2023, at 11:17, Naresh Kamboju wrote:
> LTP running on compat mode where the tests run on
> 64-bit kernel and 32-bit userspace are noticed on a list of failures.
>
> What would be the best way to handle this rare combination of failures ?
>
> * arm64: juno-r2-compat, qemu_arm64-compat and qemu_x86_64-compat
>     - shmget02
>
> Reported-by: Linux Kernel Functional Testing <lkft@...aro.org>
>
> tst_hugepage.c:83: TINFO: 0 hugepage(s) reserved
> tst_test.c:1558: TINFO: Timeout per run is 0h 02m 30s
> tst_kconfig.c:87: TINFO: Parsing kernel config '/proc/config.gz'
> shmget02.c:95: TPASS: shmget(1644199826, 2048, 1024) : ENOENT (2)
> shmget02.c:95: TPASS: shmget(1627422610, 2048, 1536) : EEXIST (17)
> <4>[   84.678150] __vm_enough_memory: pid: 513, comm: shmget02, not
> enough memory for the allocation
> shmget02.c:95: TPASS: shmget(1644199826, 0, 1536) : EINVAL (22)
> shmget02.c:95: TFAIL: shmget(1644199826, 4278190080, 1536) expected
> EINVAL: ENOMEM (12)

Adding Liam Howlett, Davidlohr Bueso and Manfred Spraul to Cc, they
have worked on the shm code in the past few years.

This is the line

	{&shmkey1, SHMMAX + 1, IPC_CREAT | IPC_EXCL, 0, 0, EINVAL},

from

https://github.com/linux-test-project/ltp/blob/04e8f2f4fd949/testcases/kernel/syscalls/ipc/shmget/shmget02.c#LL59C1-L59C61

right?

I think this is a result of SHMMAX being defined as
#define SHMMAX (ULONG_MAX - (1UL << 24)), so the kernel would
likely use a large 64-bit value here, while the 32-bit user
space uses a much smaller limit.

The expected return code likely comes from

static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
{
...
        if (size < SHMMIN || size > ns->shm_ctlmax)
                return -EINVAL;
        
but if ns->shm_ctlmax is probably set to the 64-bit value here.
It would then trigger the accounting limit in __shmem_file_setup():

        if (shmem_acct_size(flags, size))
                return ERR_PTR(-ENOMEM);

My feeling is that the kernel in this case works as expected,
and I wouldn't see this as a bug. On the other hand, this
can probably be addressed in the kernel by adding a check for
compat tasks like

--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -714,7 +714,8 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
        char name[13];
        vm_flags_t acctflag = 0;
 
-       if (size < SHMMIN || size > ns->shm_ctlmax)
+       if (size < SHMMIN || size > ns->shm_ctlmax ||
+          in_compat_syscall() && size > COMPAT_SHMMAX))
                return -EINVAL;
 
        if (numpages << PAGE_SHIFT < size)


      Arnd