linux-kernel - Tasks blocking on POSIX message queue using wrong priority?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:   Fri, 10 Nov 2017 21:12:20 +0000
From:   Jonathan Haws <Jonathan.Haws@....usu.edu>
To:     "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Tasks blocking on POSIX message queue using wrong priority?

Hello all,

I'm looking at the file ipc/mqueue.c and I found what I believe to be a
bug, but I would love it if someone corrected me or confirmed this.

In the wq_add() function, it appears that the task is added to the wait
queue using the static priority, which I believe equates to the
processes' niceness value.  However, this means that real-time threads
(which share a processes niceness value I believe) get added to the
queue in FIFO order (unless of course the user adjusts the niceness
value, but that gets tricky for a multithreaded process :) ).

static void wq_add(struct mqueue_inode_info *info, int sr,
                        struct ext_wait_queue *ewp)
{
        struct ext_wait_queue *walk;

        ewp->task = current;

        list_for_each_entry(walk, &info->e_wait_q[sr].list, list) {
                if (walk->task->static_prio <= current->static_prio) {
                        list_add_tail(&ewp->list, &walk->list);
                        return;
                }
        }
        list_add_tail(&ewp->list, &info->e_wait_q[sr].list);
}

It seems that to have this work properly for RT threads as well all
that would be required is to have access to a function similar to the
effective_prio() call in kernel/sched/core.c.  Am I missing something,
or would that add things to the wait queue in task priority order?

Of course I could be missing some detail about how task priorities get
set.  I do have test code I can send along that fills a POSIX message
queue, spawns a few threads of varying priorities - each posting to the
queue with a message priority equal to the thread priority, and then
the reader thread pulls messages out and prints the message priority.
 I'm seeing that the order in which threads are waking is always FIFO
order, supporting my earlier argument about niceness values.

What am I missing or is this a very subtle bug in message queue
handling?

Thanks!
Jon