linux-kernel - Re: [RESEND RFC] SIGOOM Proposal

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Y7+se9vpA7JB3Mmm@mit.edu>
Date:   Thu, 12 Jan 2023 01:45:15 -0500
From:   "Theodore Ts'o" <tytso@....edu>
To:     Arseniy Lesin <emptiedsoul@...dclanz.org>
Cc:     linux-kernel@...r.kernel.org
Subject: Re: [RESEND RFC] SIGOOM Proposal

On Thu, Jan 12, 2023 at 07:51:45AM +0300, Arseniy Lesin wrote:
> 
> 2. Proposal
> ==================
> 
> 2.1. The SIGOOM Signal
> ------------------
> 
> I propose the addition of new signal: SIGOOM (Out-Of-Memory SIGnal)

AIX had a similar SIGDANGER signal which was sent to all processes
when memory was low.  By default, it was ignored, but processes that
were aware of it could use this as an opportunity to shrink their
memory footprint.

> The technical detail of this addition is a bit unpleasant: there is
> actually no room for new signals! 
> 
> Numbers 1-31 are already assigned, every signal with number > SIGRTMIN 
> (currently 32) is considered realtime and queued accordingly. 
> 
> Adding SIGOOM as signal #32 by shifting SIGRTMIN to 33 can do a trick,
> but this will almost certainly break compatibility (namely, with glibc
> threading)
> 
> I propose adding SIGOOM as signal #65 (after SIGRTMAX), but we should
> clarify some checks in kernel/signal.c (possibly in other places too,
> where signal number is tested against being realtime) and possibly add a
> such-like macro:
> 
> #define SIG_IS_REALTIME(signum) (((signum) > SIGRTMIN) && ((signum) < SIGRTMAX))

It's actually worse than this.  The problem is space in the signal
mask.  From the signal(7) man page:

   Signal mask and pending signals

       A signal may be blocked, which means that it will not be
       delivered until it is later unblocked.  Between the time when
       it is generated and when it is deliv‐ ered a signal is said to
       be pending.

       Each thread in a process has an independent signal mask, which
       indicates the set of signals that the thread is currently
       blocking.  A thread can manipulate its signal mask using
       pthread_sigmask(3).  In a traditional single-threaded ap‐
       plication, sigprocmask(2) can be used to manipulate the signal
       mask.

The signal mask is stored in the signal set structure (sigset_t /
kernel_sigset_t).    Later in that same man page:

       The addition of real-time signals required the widening of the
       signal set structure (sigset_t) from 32 to 64 bits.
       Consequently, various system calls were superseded by new
       system calls that supported the larger signal sets.  The old
       and new system calls are as follows:

       Linux 2.0 and earlier   Linux 2.2 and later
       sigaction(2)            rt_sigaction(2)
       sigpending(2)           rt_sigpending(2)
       sigprocmask(2)          rt_sigprocmask(2)
       sigreturn(2)            rt_sigreturn(2)
       sigsuspend(2)           rt_sigsuspend(2)
       sigtimedwait(2)         rt_sigtimedwait(2)

This is why adding a new signal is _hard_, whether it's
SIGDANGER/SIGOOM, or the SIGINFO from the people who want BSD-style
control-T support.

						- Ted