| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <202010250759.F9745E0B6@keescook>
Date: Sun, 25 Oct 2020 17:19:43 -0700
From: Kees Cook <keescook@...omium.org>
To: "Michael Kerrisk (man-pages)" <mtk.manpages@...il.com>
Cc: Tycho Andersen <tycho@...ho.pizza>,
Sargun Dhillon <sargun@...gun.me>,
Christian Brauner <christian@...uner.io>,
linux-man <linux-man@...r.kernel.org>,
lkml <linux-kernel@...r.kernel.org>,
Aleksa Sarai <cyphar@...har.com>, Jann Horn <jannh@...gle.com>,
Alexei Starovoitov <ast@...nel.org>, wad@...omium.org,
bpf@...r.kernel.org, Song Liu <songliubraving@...com>,
Daniel Borkmann <daniel@...earbox.net>,
Andy Lutomirski <luto@...capital.net>,
Linux Containers <containers@...ts.linux-foundation.org>,
Giuseppe Scrivano <gscrivan@...hat.com>,
Robert Sesek <rsesek@...gle.com>
Subject: Re: For review: seccomp_user_notif(2) manual page
On Thu, Oct 15, 2020 at 01:24:03PM +0200, Michael Kerrisk (man-pages) wrote:
> On 10/1/20 1:39 AM, Kees Cook wrote:
> > I'll comment more later, but I've run out of time today and I didn't see
> > anyone mention this detail yet in the existing threads... :)
>
> Later never came :-). But, I hope you may have comments for the
> next draft, which I will send out soon.
Later is now, and Soon approaches!
I finally caught up and read through this whole thread. Thank you all
for the bug fix[1], and I'm looking forward to more[2]. :)
For my reply I figured I'd base it on the current draft, so here's a
simulated quote based on the seccomp_user_notif branch of
https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git
through commit 71101158fe330af5a26552447a0bb433b69e15b7
$ COLUMNS=75 man --nh --nj man2/seccomp_user_notif.2 | sed 's/^/> /'
On Sun, Oct 25, 2020 at 01:54:05PM +0100, Michael Kerrisk (man-pages) wrote:
> SECCOMP_USER_NOTIF(2) Linux Programmer's Manual SECCOMP_USER_NOTIF(2)
>
> NAME
> seccomp_user_notif - Seccomp user-space notification mechanism
>
> SYNOPSIS
> #include <linux/seccomp.h>
> #include <linux/filter.h>
> #include <linux/audit.h>
>
> int seccomp(unsigned int operation, unsigned int flags, void *args);
>
> #include <sys/ioctl.h>
>
> int ioctl(int fd, SECCOMP_IOCTL_NOTIF_RECV,
> struct seccomp_notif *req);
> int ioctl(int fd, SECCOMP_IOCTL_NOTIF_SEND,
> struct seccomp_notif_resp *resp);
> int ioctl(int fd, SECCOMP_IOCTL_NOTIF_ID_VALID, __u64 *id);
>
> DESCRIPTION
> This page describes the user-space notification mechanism provided
> by the Secure Computing (seccomp) facility. As well as the use of
> the SECCOMP_FILTER_FLAG_NEW_LISTENER flag, the
> SECCOMP_RET_USER_NOTIF action value, and the
> SECCOMP_GET_NOTIF_SIZES operation described in seccomp(2), this
> mechanism involves the use of a number of related ioctl(2)
> operations (described below).
>
> Overview
> In conventional usage of a seccomp filter, the decision about how
> to treat a system call is made by the filter itself. By contrast,
> the user-space notification mechanism allows the seccomp filter to
> delegate the handling of the system call to another user-space
> process. Note that this mechanism is explicitly not intended as a
> method implementing security policy; see NOTES.
>
> In the discussion that follows, the thread(s) on which the seccomp
> filter is installed is (are) referred to as the target, and the
> process that is notified by the user-space notification mechanism
> is referred to as the supervisor.
>
> A suitably privileged supervisor can use the user-space
> notification mechanism to perform actions on behalf of the target.
> The advantage of the user-space notification mechanism is that the
> supervisor will usually be able to retrieve information about the
> target and the performed system call that the seccomp filter
> itself cannot. (A seccomp filter is limited in the information it
> can obtain and the actions that it can perform because it is
> running on a virtual machine inside the kernel.)
>
> An overview of the steps performed by the target and the
> supervisor is as follows:
>
> 1. The target establishes a seccomp filter in the usual manner,
> but with two differences:
>
> • The seccomp(2) flags argument includes the flag
> SECCOMP_FILTER_FLAG_NEW_LISTENER. Consequently, the return
> value of the (successful) seccomp(2) call is a new
nit: extra space
> "listening" file descriptor that can be used to receive
> notifications. Only one "listening" seccomp filter can be
> installed for a thread.
I like this limitation, but I expect that it'll need to change in the
future. Even with LSMs, we see the need for arbitrary stacking, and the
idea of there being only 1 supervisor will eventually break down. Right
now there is only 1 because only container managers are using this
feature. But if some daemon starts using it to isolate some thread,
suddenly it might break if a container manager is trying to listen to it
too, etc. I expect it won't be needed soon, but I do think it'll change.
>
> • In cases where it is appropriate, the seccomp filter returns
> the action value SECCOMP_RET_USER_NOTIF. This return value
> will trigger a notification event.
>
> 2. In order that the supervisor can obtain notifications using the
> listening file descriptor, (a duplicate of) that file
> descriptor must be passed from the target to the supervisor.
Yet another reason to have an "activate on exec" mode for seccomp. With
no_new_privs _not_ being delayed in such a way, I think it'd be safe to
add. The supervisor would get the fd immediately, and then once it
fork/execed suddenly the whole thing would activate, and no fd passing
needed.
The "on exec" boundary is really only needed for oblivious targets. For
a coordinated target, I've thought it might be nice to have an arbitrary
"go" point, where the target could call seccomp() with something like a
SECCOMP_ACTIVATE_DELAYED_FILTERS operation. This lets any process
initialization happen that might need to do things that would be blocked
by filters, etc.
Before:
fork
install some filters that don't block initialization
exec
do some initialization
install more filters, maybe block exec, seccomp
run
After:
fork
install delayed filters
exec
do some initialization
activate delayed filters
run
In practice, the two-stage filter application has been fine, if
sometimes a bit complex (e.g. for user_notif, "do some initialization"
includes figuring out how to pass the fd back to the supervisor, etc).
> One way in which this could be done is by passing the file
> descriptor over a UNIX domain socket connection between the
> target and the supervisor (using the SCM_RIGHTS ancillary
> message type described in unix(7)).
>
> 3. The supervisor will receive notification events on the
> listening file descriptor. These events are returned as
> structures of type seccomp_notif. Because this structure and
> its size may evolve over kernel versions, the supervisor must
> first determine the size of this structure using the seccomp(2)
> SECCOMP_GET_NOTIF_SIZES operation, which returns a structure of
> type seccomp_notif_sizes. The supervisor allocates a buffer of
> size seccomp_notif_sizes.seccomp_notif bytes to receive
> notification events. In addition,the supervisor allocates
> another buffer of size seccomp_notif_sizes.seccomp_notif_resp
> bytes for the response (a struct seccomp_notif_resp structure)
> that it will provide to the kernel (and thus the target).
>
> 4. The target then performs its workload, which includes system
> calls that will be controlled by the seccomp filter. Whenever
> one of these system calls causes the filter to return the
> SECCOMP_RET_USER_NOTIF action value, the kernel does not (yet)
> execute the system call; instead, execution of the target is
> temporarily blocked inside the kernel (in a sleep state that is
> interruptible by signals) and a notification event is generated
> on the listening file descriptor.
>
> 5. The supervisor can now repeatedly monitor the listening file
> descriptor for SECCOMP_RET_USER_NOTIF-triggered events. To do
> this, the supervisor uses the SECCOMP_IOCTL_NOTIF_RECV ioctl(2)
> operation to read information about a notification event; this
> operation blocks until an event is available. The operation
> returns a seccomp_notif structure containing information about
> the system call that is being attempted by the target.
>
> 6. The seccomp_notif structure returned by the
> SECCOMP_IOCTL_NOTIF_RECV operation includes the same
> information (a seccomp_data structure) that was passed to the
> seccomp filter. This information allows the supervisor to
> discover the system call number and the arguments for the
> target's system call. In addition, the notification event
> contains the ID of the thread that triggered the notification.
Should "cookie" be at least named here, just to provide a bit more
context for when it is mentioned in 8 below? E.g.:
... In addition, the notification event
contains the triggering thread's ID and a unique cookie to be
used in subsequent SECCOMP_IOCTL_NOTIF_ID_VALID and
SECCOMP_IOCTL_NOTIF_SEND operations.
>
> The information in the notification can be used to discover the
> values of pointer arguments for the target's system call.
> (This is something that can't be done from within a seccomp
> filter.) One way in which the supervisor can do this is to
> open the corresponding /proc/[tid]/mem file (see proc(5)) and
> read bytes from the location that corresponds to one of the
> pointer arguments whose value is supplied in the notification
> event. (The supervisor must be careful to avoid a race
> condition that can occur when doing this; see the description
> of the SECCOMP_IOCTL_NOTIF_ID_VALID ioctl(2) operation below.)
> In addition, the supervisor can access other system information
> that is visible in user space but which is not accessible from
> a seccomp filter.
>
> 7. Having obtained information as per the previous step, the
> supervisor may then choose to perform an action in response to
> the target's system call (which, as noted above, is not
> executed when the seccomp filter returns the
> SECCOMP_RET_USER_NOTIF action value).
>
> One example use case here relates to containers. The target
> may be located inside a container where it does not have
> sufficient capabilities to mount a filesystem in the
> container's mount namespace. However, the supervisor may be a
> more privileged process that does have sufficient capabilities
> to perform the mount operation.
>
> 8. The supervisor then sends a response to the notification. The
> information in this response is used by the kernel to construct
> a return value for the target's system call and provide a value
> that will be assigned to the errno variable of the target.
>
> The response is sent using the SECCOMP_IOCTL_NOTIF_SEND
> ioctl(2) operation, which is used to transmit a
> seccomp_notif_resp structure to the kernel. This structure
> includes a cookie value that the supervisor obtained in the
> seccomp_notif structure returned by the
> SECCOMP_IOCTL_NOTIF_RECV operation. This cookie value allows
> the kernel to associate the response with the target.
Describing where the cookie came from seems like it should live in 6
above. A reader would have to take this new info and figure out where
SECCOMP_IOCTL_NOTIF_RECV was described and piece it together. With the
suggestion to 6 above, maybe:
... This structure
must include the cookie value that the supervisor obtained in
the seccomp_notif structure returned by the
SECCOMP_IOCTL_NOTIF_RECV operation, which allows the kernel
to associate the response with the target.
>
> 9. Once the notification has been sent, the system call in the
> target thread unblocks, returning the information that was
> provided by the supervisor in the notification response.
>
> As a variation on the last two steps, the supervisor can send a
> response that tells the kernel that it should execute the target
> thread's system call; see the discussion of
> SECCOMP_USER_NOTIF_FLAG_CONTINUE, below.
>
> ioctl(2) operations
> The following ioctl(2) operations are provided to support seccomp
> user-space notification. For each of these operations, the first
> (file descriptor) argument of ioctl(2) is the listening file
> descriptor returned by a call to seccomp(2) with the
> SECCOMP_FILTER_FLAG_NEW_LISTENER flag.
>
> SECCOMP_IOCTL_NOTIF_RECV
> This operation is used to obtain a user-space notification
> event. If no such event is currently pending, the
> operation blocks until an event occurs. The third ioctl(2)
> argument is a pointer to a structure of the following form
> which contains information about the event. This structure
> must be zeroed out before the call.
>
> struct seccomp_notif {
> __u64 id; /* Cookie */
> __u32 pid; /* TID of target thread */
Should we rename this variable from pid to tid? Yes it's UAPI, but yay for
anonymous unions:
struct seccomp_notif {
__u64 id; /* Cookie */
union {
__u32 pid;
__u32 tid; /* TID of target thread */
};
__u32 flags; /* Currently unused (0) */
struct seccomp_data data; /* See seccomp(2) */
};
> __u32 flags; /* Currently unused (0) */
> struct seccomp_data data; /* See seccomp(2) */
> };
>
> The fields in this structure are as follows:
>
> id This is a cookie for the notification. Each such
> cookie is guaranteed to be unique for the
> corresponding seccomp filter.
>
> • It can be used with the
> SECCOMP_IOCTL_NOTIF_ID_VALID ioctl(2) operation to
> verify that the target is still alive.
>
> • When returning a notification response to the
> kernel, the supervisor must include the cookie
> value in the seccomp_notif_resp structure that is
> specified as the argument of the
> SECCOMP_IOCTL_NOTIF_SEND operation.
>
> pid This is the thread ID of the target thread that
> triggered the notification event.
>
> flags This is a bit mask of flags providing further
> information on the event. In the current
> implementation, this field is always zero.
>
> data This is a seccomp_data structure containing
> information about the system call that triggered the
> notification. This is the same structure that is
> passed to the seccomp filter. See seccomp(2) for
> details of this structure.
>
> On success, this operation returns 0; on failure, -1 is
> returned, and errno is set to indicate the cause of the
> error. This operation can fail with the following errors:
>
> EINVAL (since Linux 5.5)
> The seccomp_notif structure that was passed to the
> call contained nonzero fields.
>
> ENOENT The target thread was killed by a signal as the
> notification information was being generated, or the
> target's (blocked) system call was interrupted by a
> signal handler.
>
> SECCOMP_IOCTL_NOTIF_ID_VALID
> This operation can be used to check that a notification ID
> returned by an earlier SECCOMP_IOCTL_NOTIF_RECV operation
> is still valid (i.e., that the target still exists).
Maybe clarify a bit more, since it's covering more than just "is the
target still alive", but also "is that syscall still waiting for a
response":
is still valid (i.e., that the target still exists and
the syscall is still blocked waiting for a response).
>
> The third ioctl(2) argument is a pointer to the cookie (id)
> returned by the SECCOMP_IOCTL_NOTIF_RECV operation.
>
> This operation is necessary to avoid race conditions that
> can occur when the pid returned by the
> SECCOMP_IOCTL_NOTIF_RECV operation terminates, and that
> process ID is reused by another process. An example of
> this kind of race is the following
>
> 1. A notification is generated on the listening file
> descriptor. The returned seccomp_notif contains the TID
> of the target thread (in the pid field of the
> structure).
>
> 2. The target terminates.
>
> 3. Another thread or process is created on the system that
> by chance reuses the TID that was freed when the target
> terminated.
>
> 4. The supervisor open(2)s the /proc/[tid]/mem file for the
> TID obtained in step 1, with the intention of (say)
> inspecting the memory location(s) that containing the
> argument(s) of the system call that triggered the
> notification in step 1.
>
> In the above scenario, the risk is that the supervisor may
> try to access the memory of a process other than the
> target. This race can be avoided by following the call to
> open(2) with a SECCOMP_IOCTL_NOTIF_ID_VALID operation to
> verify that the process that generated the notification is
> still alive. (Note that if the target terminates after the
> latter step, a subsequent read(2) from the file descriptor
> may return 0, indicating end of file.)
>
> On success (i.e., the notification ID is still valid), this
> operation returns 0. On failure (i.e., the notification ID
> is no longer valid), -1 is returned, and errno is set to
> ENOENT.
>
> SECCOMP_IOCTL_NOTIF_SEND
> This operation is used to send a notification response back
> to the kernel. The third ioctl(2) argument of this
> structure is a pointer to a structure of the following
> form:
>
> struct seccomp_notif_resp {
> __u64 id; /* Cookie value */
> __s64 val; /* Success return value */
> __s32 error; /* 0 (success) or negative
> error number */
> __u32 flags; /* See below */
> };
>
> The fields of this structure are as follows:
>
> id This is the cookie value that was obtained using the
> SECCOMP_IOCTL_NOTIF_RECV operation. This cookie
> value allows the kernel to correctly associate this
> response with the system call that triggered the
> user-space notification.
>
> val This is the value that will be used for a spoofed
> success return for the target's system call; see
> below.
>
> error This is the value that will be used as the error
> number (errno) for a spoofed error return for the
> target's system call; see below.
>
> flags This is a bit mask that includes zero or more of the
> following flags:
>
> SECCOMP_USER_NOTIF_FLAG_CONTINUE (since Linux 5.5)
> Tell the kernel to execute the target's
> system call.
>
> Two kinds of response are possible:
>
> • A response to the kernel telling it to execute the
> target's system call. In this case, the flags field
> includes SECCOMP_USER_NOTIF_FLAG_CONTINUE and the error
> and val fields must be zero.
>
> This kind of response can be useful in cases where the
> supervisor needs to do deeper analysis of the target's
> system call than is possible from a seccomp filter (e.g.,
> examining the values of pointer arguments), and, having
> decided that the system call does not require emulation
> by the supervisor, the supervisor wants the system call
> to be executed normally in the target.
>
> The SECCOMP_USER_NOTIF_FLAG_CONTINUE flag should be used
> with caution; see NOTES.
>
> • A spoofed return value for the target's system call. In
> this case, the kernel does not execute the target's
> system call, instead causing the system call to return a
> spoofed value as specified by fields of the
> seccomp_notif_resp structure. The supervisor should set
> the fields of this structure as follows:
>
> + flags does not contain
> SECCOMP_USER_NOTIF_FLAG_CONTINUE.
>
> + error is set either to 0 for a spoofed "success"
> return or to a negative error number for a spoofed
> "failure" return. In the former case, the kernel
> causes the target's system call to return the value
> specified in the val field. In the later case, the
> kernel causes the target's system call to return -1,
> and errno is assigned the negated error value.
>
> + val is set to a value that will be used as the return
> value for a spoofed "success" return for the target's
> system call. The value in this field is ignored if
> the error field contains a nonzero value.
Strictly speaking, this is architecture specific, but all architectures
do it this way. Should seccomp enforce val == 0 when err != 0 ?
>
> On success, this operation returns 0; on failure, -1 is
> returned, and errno is set to indicate the cause of the
> error. This operation can fail with the following errors:
>
> EINPROGRESS
> A response to this notification has already been
> sent.
>
> EINVAL An invalid value was specified in the flags field.
>
> EINVAL The flags field contained
> SECCOMP_USER_NOTIF_FLAG_CONTINUE, and the error or
> val field was not zero.
>
> ENOENT The blocked system call in the target has been
> interrupted by a signal handler or the target has
> terminated.
>
> NOTES
> select()/poll()/epoll semantics
> The file descriptor returned when seccomp(2) is employed with the
> SECCOMP_FILTER_FLAG_NEW_LISTENER flag can be monitored using
> poll(2), epoll(7), and select(2). These interfaces indicate that
> the file descriptor is ready as follows:
>
> • When a notification is pending, these interfaces indicate that
> the file descriptor is readable. Following such an indication,
> a subsequent SECCOMP_IOCTL_NOTIF_RECV ioctl(2) will not block,
> returning either information about a notification or else
> failing with the error EINTR if the target has been killed by a
> signal or its system call has been interrupted by a signal
> handler.
>
> • After the notification has been received (i.e., by the
> SECCOMP_IOCTL_NOTIF_RECV ioctl(2) operation), these interfaces
> indicate that the file descriptor is writable, meaning that a
> notification response can be sent using the
> SECCOMP_IOCTL_NOTIF_SEND ioctl(2) operation.
>
> • After the last thread using the filter has terminated and been
> reaped using waitpid(2) (or similar), the file descriptor
> indicates an end-of-file condition (readable in select(2);
> POLLHUP/EPOLLHUP in poll(2)/ epoll_wait(2)).
I'll reply separately about the "ioctl() does not terminate when all
filters have terminated" case.
>
> Design goals; use of SECCOMP_USER_NOTIF_FLAG_CONTINUE
> The intent of the user-space notification feature is to allow
> system calls to be performed on behalf of the target. The
> target's system call should either be handled by the supervisor or
> allowed to continue normally in the kernel (where standard
> security policies will be applied).
>
> Note well: this mechanism must not be used to make security policy
> decisions about the system call, which would be inherently race-
> prone for reasons described next.
>
> The SECCOMP_USER_NOTIF_FLAG_CONTINUE flag must be used with
> caution. If set by the supervisor, the target's system call will
> continue. However, there is a time-of-check, time-of-use race
> here, since an attacker could exploit the interval of time where
> the target is blocked waiting on the "continue" response to do
> things such as rewriting the system call arguments.
>
> Note furthermore that a user-space notifier can be bypassed if the
> existing filters allow the use of seccomp(2) or prctl(2) to
> install a filter that returns an action value with a higher
> precedence than SECCOMP_RET_USER_NOTIF (see seccomp(2)).
>
> It should thus be absolutely clear that the seccomp user-space
> notification mechanism can not be used to implement a security
> policy! It should only ever be used in scenarios where a more
> privileged process supervises the system calls of a lesser
> privileged target to get around kernel-enforced security
> restrictions when the supervisor deems this safe. In other words,
> in order to continue a system call, the supervisor should be sure
> that another security mechanism or the kernel itself will
> sufficiently block the system call if its arguments are rewritten
> to something unsafe.
>
> Interaction with SA_RESTART signal handlers
> Consider the following scenario:
>
> • The target process has used sigaction(2) to install a signal
> handler with the SA_RESTART flag.
>
> • The target has made a system call that triggered a seccomp user-
> space notification and the target is currently blocked until the
> supervisor sends a notification response.
>
> • A signal is delivered to the target and the signal handler is
> executed.
>
> • When (if) the supervisor attempts to send a notification
> response, the SECCOMP_IOCTL_NOTIF_SEND ioctl(2)) operation will
> fail with the ENOENT error.
>
> In this scenario, the kernel will restart the target's system
> call. Consequently, the supervisor will receive another user-
> space notification. Thus, depending on how many times the blocked
> system call is interrupted by a signal handler, the supervisor may
> receive multiple notifications for the same system call in the
maybe "... for the same instance of a system call in the target." for
clarity?
> target.
>
> One oddity is that system call restarting as described in this
> scenario will occur even for the blocking system calls listed in
> signal(7) that would never normally be restarted by the SA_RESTART
> flag.
Does this need fixing? I imagine the correct behavior for this case
would be a response to _SEND of EINPROGRESS and the target would see
EINTR normally?
I mean, it's not like seccomp doesn't already expose weirdness with
syscall restarts. Not even arm64 compat agrees[3] with arm32 in this
regard. :(
> BUGS
> If a SECCOMP_IOCTL_NOTIF_RECV ioctl(2) operation is performed
> after the target terminates, then the ioctl(2) call simply blocks
> (rather than returning an error to indicate that the target no
> longer exists).
I want this fixed. It caused me no end of pain when building the
selftests, and ended up spawning my implementing a global test timeout
in kselftest. :P Before the usage counter refactor, there was no sane
way to deal with this, but now I think we're close[2]. I'll reply
separately about this.
>
> EXAMPLES
> The (somewhat contrived) program shown below demonstrates the use
> of the interfaces described in this page. The program creates a
> child process that serves as the "target" process. The child
> process installs a seccomp filter that returns the
> SECCOMP_RET_USER_NOTIF action value if a call is made to mkdir(2).
> The child process then calls mkdir(2) once for each of the
> supplied command-line arguments, and reports the result returned
> by the call. After processing all arguments, the child process
> terminates.
>
> The parent process acts as the supervisor, listening for the
> notifications that are generated when the target process calls
> mkdir(2). When such a notification occurs, the supervisor
> examines the memory of the target process (using /proc/[pid]/mem)
> to discover the pathname argument that was supplied to the
> mkdir(2) call, and performs one of the following actions:
I like this example! It's simple enough to be understandable and complex
enough to show the purpose of user_notif. :)
>
> • If the pathname begins with the prefix "/tmp/", then the
> supervisor attempts to create the specified directory, and then
> spoofs a return for the target process based on the return value
> of the supervisor's mkdir(2) call. In the event that that call
> succeeds, the spoofed success return value is the length of the
> pathname.
>
> • If the pathname begins with "./" (i.e., it is a relative
> pathname), the supervisor sends a
> SECCOMP_USER_NOTIF_FLAG_CONTINUE response to the kernel to say
> that the kernel should execute the target process's mkdir(2)
> call.
>
> • If the pathname begins with some other prefix, the supervisor
> spoofs an error return for the target process, so that the
> target process's mkdir(2) call appears to fail with the error
> EOPNOTSUPP ("Operation not supported"). Additionally, if the
> specified pathname is exactly "/bye", then the supervisor
> terminates.
>
> This program can be used to demonstrate various aspects of the
> behavior of the seccomp user-space notification mechanism. To
> help aid such demonstrations, the program logs various messages to
> show the operation of the target process (lines prefixed "T:") and
> the supervisor (indented lines prefixed "S:").
>
> In the following example, the target attempts to create the
> directory /tmp/x. Upon receiving the notification, the supervisor
> creates the directory on the target's behalf, and spoofs a success
> return to be received by the target process's mkdir(2) call.
>
> $ ./seccomp_unotify /tmp/x
> T: PID = 23168
>
> T: about to mkdir("/tmp/x")
> S: got notification (ID 0x17445c4a0f4e0e3c) for PID 23168
> S: executing: mkdir("/tmp/x", 0700)
> S: success! spoofed return = 6
> S: sending response (flags = 0; val = 6; error = 0)
> T: SUCCESS: mkdir(2) returned 6
>
> T: terminating
> S: target has terminated; bye
>
> In the above output, note that the spoofed return value seen by
> the target process is 6 (the length of the pathname /tmp/x),
> whereas a normal mkdir(2) call returns 0 on success.
>
> In the next example, the target attempts to create a directory
> using the relative pathname ./sub. Since this pathname starts
> with "./", the supervisor sends a SECCOMP_USER_NOTIF_FLAG_CONTINUE
> response to the kernel, and the kernel then (successfully)
> executes the target process's mkdir(2) call.
>
> $ ./seccomp_unotify ./sub
> T: PID = 23204
>
> T: about to mkdir("./sub")
> S: got notification (ID 0xddb16abe25b4c12) for PID 23204
> S: target can execute system call
> S: sending response (flags = 0x1; val = 0; error = 0)
> T: SUCCESS: mkdir(2) returned 0
>
> T: terminating
> S: target has terminated; bye
>
> If the target process attempts to create a directory with a
> pathname that doesn't start with "." and doesn't begin with the
> prefix "/tmp/", then the supervisor spoofs an error return
> (EOPNOTSUPP, "Operation not supported") for the target's mkdir(2)
> call (which is not executed):
>
> $ ./seccomp_unotify /xxx
> T: PID = 23178
>
> T: about to mkdir("/xxx")
> S: got notification (ID 0xe7dc095d1c524e80) for PID 23178
> S: spoofing error response (Operation not supported)
> S: sending response (flags = 0; val = 0; error = -95)
> T: ERROR: mkdir(2): Operation not supported
>
> T: terminating
> S: target has terminated; bye
>
> In the next example, the target process attempts to create a
> directory with the pathname /tmp/nosuchdir/b. Upon receiving the
> notification, the supervisor attempts to create that directory,
> but the mkdir(2) call fails because the directory /tmp/nosuchdir
> does not exist. Consequently, the supervisor spoofs an error
> return that passes the error that it received back to the target
> process's mkdir(2) call.
>
> $ ./seccomp_unotify /tmp/nosuchdir/b
> T: PID = 23199
>
> T: about to mkdir("/tmp/nosuchdir/b")
> S: got notification (ID 0x8744454293506046) for PID 23199
> S: executing: mkdir("/tmp/nosuchdir/b", 0700)
> S: failure! (errno = 2; No such file or directory)
> S: sending response (flags = 0; val = 0; error = -2)
> T: ERROR: mkdir(2): No such file or directory
>
> T: terminating
> S: target has terminated; bye
>
> If the supervisor receives a notification and sees that the
> argument of the target's mkdir(2) is the string "/bye", then (as
> well as spoofing an EOPNOTSUPP error), the supervisor terminates.
> If the target process subsequently executes another mkdir(2) that
> triggers its seccomp filter to return the SECCOMP_RET_USER_NOTIF
> action value, then the kernel causes the target process's system
> call to fail with the error ENOSYS ("Function not implemented").
> This is demonstrated by the following example:
>
> $ ./seccomp_unotify /bye /tmp/y
> T: PID = 23185
>
> T: about to mkdir("/bye")
> S: got notification (ID 0xa81236b1d2f7b0f4) for PID 23185
> S: spoofing error response (Operation not supported)
> S: sending response (flags = 0; val = 0; error = -95)
> S: terminating **********
> T: ERROR: mkdir(2): Operation not supported
>
> T: about to mkdir("/tmp/y")
> T: ERROR: mkdir(2): Function not implemented
>
> T: terminating
>
> Program source
> #define _GNU_SOURCE
> #include <sys/types.h>
> #include <sys/prctl.h>
> #include <fcntl.h>
> #include <limits.h>
> #include <signal.h>
> #include <stddef.h>
> #include <stdint.h>
> #include <stdbool.h>
> #include <linux/audit.h>
> #include <sys/syscall.h>
> #include <sys/stat.h>
> #include <linux/filter.h>
> #include <linux/seccomp.h>
> #include <sys/ioctl.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <unistd.h>
> #include <errno.h>
> #include <sys/socket.h>
> #include <sys/un.h>
>
> #define errExit(msg) do { perror(msg); exit(EXIT_FAILURE); \
> } while (0)
Because I love macros, you can expand this to make it take a format
string:
#define errExit(fmt, ...) do { \
char __err[64]; \
strerror_r(errno, __err, sizeof(__err)); \
fprintf(stderr, fmt ": %s\n", ##__VA_ARG__, __err); \
exit(EXIT_FAILURE); \
} while (0)
>
> /* Send the file descriptor 'fd' over the connected UNIX domain socket
> 'sockfd'. Returns 0 on success, or -1 on error. */
>
> static int
> sendfd(int sockfd, int fd)
> {
> struct msghdr msgh;
> struct iovec iov;
> int data;
> struct cmsghdr *cmsgp;
>
> /* Allocate a char array of suitable size to hold the ancillary data.
> However, since this buffer is in reality a 'struct cmsghdr', use a
> union to ensure that it is suitably aligned. */
> union {
> char buf[CMSG_SPACE(sizeof(int))];
> /* Space large enough to hold an 'int' */
> struct cmsghdr align;
> } controlMsg;
>
> /* The 'msg_name' field can be used to specify the address of the
> destination socket when sending a datagram. However, we do not
> need to use this field because 'sockfd' is a connected socket. */
>
> msgh.msg_name = NULL;
> msgh.msg_namelen = 0;
>
> /* On Linux, we must transmit at least one byte of real data in
> order to send ancillary data. We transmit an arbitrary integer
> whose value is ignored by recvfd(). */
>
> msgh.msg_iov = &iov;
> msgh.msg_iovlen = 1;
> iov.iov_base = &data;
> iov.iov_len = sizeof(int);
> data = 12345;
>
> /* Set 'msghdr' fields that describe ancillary data */
>
> msgh.msg_control = controlMsg.buf;
> msgh.msg_controllen = sizeof(controlMsg.buf);
>
> /* Set up ancillary data describing file descriptor to send */
>
> cmsgp = CMSG_FIRSTHDR(&msgh);
> cmsgp->cmsg_level = SOL_SOCKET;
> cmsgp->cmsg_type = SCM_RIGHTS;
> cmsgp->cmsg_len = CMSG_LEN(sizeof(int));
> memcpy(CMSG_DATA(cmsgp), &fd, sizeof(int));
>
> /* Send real plus ancillary data */
>
> if (sendmsg(sockfd, &msgh, 0) == -1)
> return -1;
>
> return 0;
> }
>
> /* Receive a file descriptor on a connected UNIX domain socket. Returns
> the received file descriptor on success, or -1 on error. */
>
> static int
> recvfd(int sockfd)
> {
> struct msghdr msgh;
> struct iovec iov;
> int data, fd;
> ssize_t nr;
>
> /* Allocate a char buffer for the ancillary data. See the comments
> in sendfd() */
> union {
> char buf[CMSG_SPACE(sizeof(int))];
> struct cmsghdr align;
> } controlMsg;
> struct cmsghdr *cmsgp;
>
> /* The 'msg_name' field can be used to obtain the address of the
> sending socket. However, we do not need this information. */
>
> msgh.msg_name = NULL;
> msgh.msg_namelen = 0;
>
> /* Specify buffer for receiving real data */
>
> msgh.msg_iov = &iov;
> msgh.msg_iovlen = 1;
> iov.iov_base = &data; /* Real data is an 'int' */
> iov.iov_len = sizeof(int);
>
> /* Set 'msghdr' fields that describe ancillary data */
>
> msgh.msg_control = controlMsg.buf;
> msgh.msg_controllen = sizeof(controlMsg.buf);
>
> /* Receive real plus ancillary data; real data is ignored */
>
> nr = recvmsg(sockfd, &msgh, 0);
> if (nr == -1)
> return -1;
>
> cmsgp = CMSG_FIRSTHDR(&msgh);
>
> /* Check the validity of the 'cmsghdr' */
>
> if (cmsgp == NULL ||
> cmsgp->cmsg_len != CMSG_LEN(sizeof(int)) ||
> cmsgp->cmsg_level != SOL_SOCKET ||
> cmsgp->cmsg_type != SCM_RIGHTS) {
> errno = EINVAL;
> return -1;
> }
>
> /* Return the received file descriptor to our caller */
>
> memcpy(&fd, CMSG_DATA(cmsgp), sizeof(int));
> return fd;
> }
>
> static void
> sigchldHandler(int sig)
> {
> char *msg = "\tS: target has terminated; bye\n";
>
> write(STDOUT_FILENO, msg, strlen(msg));
white space nit: extra space before "="
efficiency nit: strlen isn't needed, since it can be done with
compile-time constant constants:
char msg[] = "\tS: target has terminated; bye\n";
write(STDOUT_FILENO, msg, sizeof(msg) - 1);
(some optimization levels may already replace the strlen a sizeof - 1)
> _exit(EXIT_SUCCESS);
> }
>
> static int
> seccomp(unsigned int operation, unsigned int flags, void *args)
> {
> return syscall(__NR_seccomp, operation, flags, args);
> }
>
> /* The following is the x86-64-specific BPF boilerplate code for checking
> that the BPF program is running on the right architecture + ABI. At
> completion of these instructions, the accumulator contains the system
> call number. */
>
> /* For the x32 ABI, all system call numbers have bit 30 set */
>
> #define X32_SYSCALL_BIT 0x40000000
>
> #define X86_64_CHECK_ARCH_AND_LOAD_SYSCALL_NR \
> BPF_STMT(BPF_LD | BPF_W | BPF_ABS, \
> (offsetof(struct seccomp_data, arch))), \
> BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, AUDIT_ARCH_X86_64, 0, 2), \
> BPF_STMT(BPF_LD | BPF_W | BPF_ABS, \
> (offsetof(struct seccomp_data, nr))), \
> BPF_JUMP(BPF_JMP | BPF_JGE | BPF_K, X32_SYSCALL_BIT, 0, 1), \
> BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_KILL_PROCESS)
>
> /* installNotifyFilter() installs a seccomp filter that generates
> user-space notifications (SECCOMP_RET_USER_NOTIF) when the process
> calls mkdir(2); the filter allows all other system calls.
>
> The function return value is a file descriptor from which the
> user-space notifications can be fetched. */
>
> static int
> installNotifyFilter(void)
> {
> struct sock_filter filter[] = {
> X86_64_CHECK_ARCH_AND_LOAD_SYSCALL_NR,
>
> /* mkdir() triggers notification to user-space supervisor */
>
> BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, __NR_mkdir, 0, 1),
> BPF_STMT(BPF_RET + BPF_K, SECCOMP_RET_USER_NOTIF),
>
> /* Every other system call is allowed */
>
> BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_ALLOW),
> };
>
> struct sock_fprog prog = {
> .len = sizeof(filter) / sizeof(filter[0]),
> .filter = filter,
> };
>
> /* Install the filter with the SECCOMP_FILTER_FLAG_NEW_LISTENER flag;
> as a result, seccomp() returns a notification file descriptor. */
>
> int notifyFd = seccomp(SECCOMP_SET_MODE_FILTER,
> SECCOMP_FILTER_FLAG_NEW_LISTENER, &prog);
> if (notifyFd == -1)
> errExit("seccomp-install-notify-filter");
>
> return notifyFd;
> }
>
> /* Close a pair of sockets created by socketpair() */
>
> static void
> closeSocketPair(int sockPair[2])
> {
> if (close(sockPair[0]) == -1)
> errExit("closeSocketPair-close-0");
> if (close(sockPair[1]) == -1)
> errExit("closeSocketPair-close-1");
> }
>
> /* Implementation of the target process; create a child process that:
>
> (1) installs a seccomp filter with the
> SECCOMP_FILTER_FLAG_NEW_LISTENER flag;
> (2) writes the seccomp notification file descriptor returned from
> the previous step onto the UNIX domain socket, 'sockPair[0]';
> (3) calls mkdir(2) for each element of 'argv'.
>
> The function return value in the parent is the PID of the child
> process; the child does not return from this function. */
>
> static pid_t
> targetProcess(int sockPair[2], char *argv[])
> {
> pid_t targetPid = fork();
> if (targetPid == -1)
> errExit("fork");
>
> if (targetPid > 0) /* In parent, return PID of child */
> return targetPid;
>
> /* Child falls through to here */
>
> printf("T: PID = %ld\n", (long) getpid());
>
> /* Install seccomp filter(s) */
>
> if (prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0))
> errExit("prctl");
>
> int notifyFd = installNotifyFilter();
>
> /* Pass the notification file descriptor to the tracing process over
> a UNIX domain socket */
>
> if (sendfd(sockPair[0], notifyFd) == -1)
> errExit("sendfd");
>
> /* Notification and socket FDs are no longer needed in target */
>
> if (close(notifyFd) == -1)
> errExit("close-target-notify-fd");
>
> closeSocketPair(sockPair);
>
> /* Perform a mkdir() call for each of the command-line arguments */
>
> for (char **ap = argv; *ap != NULL; ap++) {
> printf("\nT: about to mkdir(\"%s\")\n", *ap);
>
> int s = mkdir(*ap, 0700);
> if (s == -1)
> perror("T: ERROR: mkdir(2)");
> else
> printf("T: SUCCESS: mkdir(2) returned %d\n", s);
> }
>
> printf("\nT: terminating\n");
> exit(EXIT_SUCCESS);
> }
>
> /* Check that the notification ID provided by a SECCOMP_IOCTL_NOTIF_RECV
> operation is still valid. It will no longer be valid if the process
> has terminated. This operation can be used when accessing /proc/PID
> files in the target process in order to avoid TOCTOU race conditions
> where the PID that is returned by SECCOMP_IOCTL_NOTIF_RECV terminates
> and is reused by another process. */
>
> static void
> checkNotificationIdIsValid(int notifyFd, uint64_t id)
> {
> if (ioctl(notifyFd, SECCOMP_IOCTL_NOTIF_ID_VALID, &id) == -1) {
> fprintf(stderr, "\tS: notification ID check: "
> "target has terminated!!!\n");
>
> exit(EXIT_FAILURE);
And now you can do:
errExit("\tS: notification ID check: "
"target has terminated! ioctl");
;)
> }
> }
>
> /* Access the memory of the target process in order to discover the
> pathname that was given to mkdir() */
>
> static bool
> getTargetPathname(struct seccomp_notif *req, int notifyFd,
> char *path, size_t len)
> {
> char procMemPath[PATH_MAX];
>
> snprintf(procMemPath, sizeof(procMemPath), "/proc/%d/mem", req->pid);
>
> int procMemFd = open(procMemPath, O_RDONLY);
> if (procMemFd == -1)
> errExit("Supervisor: open");
>
> /* Check that the process whose info we are accessing is still alive.
> If the SECCOMP_IOCTL_NOTIF_ID_VALID operation (performed
> in checkNotificationIdIsValid()) succeeds, we know that the
> /proc/PID/mem file descriptor that we opened corresponds to the
> process for which we received a notification. If that process
> subsequently terminates, then read() on that file descriptor
> will return 0 (EOF). */
>
> checkNotificationIdIsValid(notifyFd, req->id);
>
> /* Read bytes at the location containing the pathname argument
> (i.e., the first argument) of the mkdir(2) call */
>
> ssize_t nread = pread(procMemFd, path, len, req->data.args[0]);
> if (nread == -1)
> errExit("pread");
>
> if (nread == 0) {
> fprintf(stderr, "\tS: pread() of /proc/PID/mem "
> "returned 0 (EOF)\n");
> exit(EXIT_FAILURE);
> }
>
> if (close(procMemFd) == -1)
> errExit("close-/proc/PID/mem");
>
> /* We have no guarantees about what was in the memory of the target
> process. We therefore treat the buffer returned by pread() as
> untrusted input. The buffer should be terminated by a null byte;
> if not, then we will trigger an error for the target process. */
>
> for (int j = 0; j < nread; j++)
> if (path[j] == ' ')
This rendering typo (' ' vs '\0') ends up manifesting badly. ;) The man
source shows:
if (path[j] == \(aq\0\(aq)
I think this needs to be \\0 ?
Or it could also be a tested as:
if (strnlen(path, nread) < nread)
> return true;
>
> return false;
> }
>
> /* Handle notifications that arrive via the SECCOMP_RET_USER_NOTIF file
> descriptor, 'notifyFd'. */
>
> static void
> handleNotifications(int notifyFd)
> {
> struct seccomp_notif_sizes sizes;
> char path[PATH_MAX];
>
> /* Discover the sizes of the structures that are used to receive
> notifications and send notification responses, and allocate
> buffers of those sizes. */
>
> if (seccomp(SECCOMP_GET_NOTIF_SIZES, 0, &sizes) == -1)
> errExit("\tS: seccomp-SECCOMP_GET_NOTIF_SIZES");
>
> struct seccomp_notif *req = malloc(sizes.seccomp_notif);
> if (req == NULL)
> errExit("\tS: malloc");
>
> /* When allocating the response buffer, we must allow for the fact
> that the user-space binary may have been built with user-space
> headers where 'struct seccomp_notif_resp' is bigger than the
> response buffer expected by the (older) kernel. Therefore, we
> allocate a buffer that is the maximum of the two sizes. This
> ensures that if the supervisor places bytes into the response
> structure that are past the response size that the kernel expects,
> then the supervisor is not touching an invalid memory location. */
>
> size_t resp_size = sizes.seccomp_notif_resp;
> if (sizeof(struct seccomp_notif_resp) > resp_size)
> resp_size = sizeof(struct seccomp_notif_resp);
>
> struct seccomp_notif_resp *resp = malloc(resp_size);
> if (resp == NULL)
> errExit("\tS: malloc");
>
> /* Loop handling notifications */
>
> for (;;) {
> /* Wait for next notification, returning info in '*req' */
>
> memset(req, 0, sizes.seccomp_notif);
> if (ioctl(notifyFd, SECCOMP_IOCTL_NOTIF_RECV, req) == -1) {
> if (errno == EINTR)
> continue;
> errExit("Supervisor: ioctl-SECCOMP_IOCTL_NOTIF_RECV");
> }
>
> printf("\tS: got notification (ID %#llx) for PID %d\n",
> req->id, req->pid);
>
> /* The only system call that can generate a notification event
> is mkdir(2). Nevertheless, we check that the notified system
> call is indeed mkdir() as kind of future-proofing of this
> code in case the seccomp filter is later modified to
> generate notifications for other system calls. */
>
> if (req->data.nr != __NR_mkdir) {
> printf("\tS: notification contained unexpected "
> "system call number; bye!!!\n");
> exit(EXIT_FAILURE);
> }
>
> bool pathOK = getTargetPathname(req, notifyFd, path,
> sizeof(path));
>
> /* Prepopulate some fields of the response */
>
> resp->id = req->id; /* Response includes notification ID */
> resp->flags = 0;
> resp->val = 0;
>
> /* If the target pathname was not valid, trigger an EINVAL error;
> if the directory is in /tmp, then create it on behalf of the
> supervisor; if the pathname starts with '.', tell the kernel
> to let the target process execute the mkdir(); otherwise, give
> an error for a directory pathname in any other location. */
>
> if (!pathOK) {
> resp->error = -EINVAL;
> printf("\tS: spoofing error for invalid pathname (%s)\n",
> strerror(-resp->error));
> } else if (strncmp(path, "/tmp/", strlen("/tmp/")) == 0) {
> printf("\tS: executing: mkdir(\"%s\", %#llo)\n",
> path, req->data.args[1]);
>
> if (mkdir(path, req->data.args[1]) == 0) {
> resp->error = 0; /* "Success" */
> resp->val = strlen(path); /* Used as return value of
> mkdir() in target */
> printf("\tS: success! spoofed return = %lld\n",
> resp->val);
> } else {
>
> /* If mkdir() failed in the supervisor, pass the error
> back to the target */
>
> resp->error = -errno;
> printf("\tS: failure! (errno = %d; %s)\n", errno,
> strerror(errno));
> }
> } else if (strncmp(path, "./", strlen("./")) == 0) {
> resp->error = resp->val = 0;
> resp->flags = SECCOMP_USER_NOTIF_FLAG_CONTINUE;
> printf("\tS: target can execute system call\n");
> } else {
> resp->error = -EOPNOTSUPP;
> printf("\tS: spoofing error response (%s)\n",
> strerror(-resp->error));
> }
>
> /* Send a response to the notification */
>
> printf("\tS: sending response "
> "(flags = %#x; val = %lld; error = %d)\n",
> resp->flags, resp->val, resp->error);
>
> if (ioctl(notifyFd, SECCOMP_IOCTL_NOTIF_SEND, resp) == -1) {
> if (errno == ENOENT)
> printf("\tS: response failed with ENOENT; "
> "perhaps target process's syscall was "
> "interrupted by a signal?\n");
> else
> perror("ioctl-SECCOMP_IOCTL_NOTIF_SEND");
> }
>
> /* If the pathname is just "/bye", then the supervisor
> terminates. This allows us to see what happens if the
> target process makes further calls to mkdir(2). */
>
> if (strcmp(path, "/bye") == 0) {
> printf("\tS: terminating **********\n");
> exit(EXIT_FAILURE);
> }
> }
> }
>
> /* Implementation of the supervisor process:
>
> (1) obtains the notification file descriptor from 'sockPair[1]'
> (2) handles notifications that arrive on that file descriptor. */
>
> static void
> supervisor(int sockPair[2])
> {
> int notifyFd = recvfd(sockPair[1]);
> if (notifyFd == -1)
> errExit("recvfd");
>
> closeSocketPair(sockPair); /* We no longer need the socket pair */
>
> handleNotifications(notifyFd);
> }
>
> int
> main(int argc, char *argv[])
> {
> int sockPair[2];
>
> setbuf(stdout, NULL);
>
> if (argc < 2) {
> fprintf(stderr, "At least one pathname argument is required\n");
> exit(EXIT_FAILURE);
> }
>
> /* Create a UNIX domain socket that is used to pass the seccomp
> notification file descriptor from the target process to the
> supervisor process. */
>
> if (socketpair(AF_UNIX, SOCK_STREAM, 0, sockPair) == -1)
> errExit("socketpair");
>
> /* Create a child process--the "target"--that installs seccomp
> filtering. The target process writes the seccomp notification
> file descriptor onto 'sockPair[0]' and then calls mkdir(2) for
> each directory in the command-line arguments. */
>
> (void) targetProcess(sockPair, &argv[optind]);
>
> /* Catch SIGCHLD when the target terminates, so that the
> supervisor can also terminate. */
>
> struct sigaction sa;
> sa.sa_handler = sigchldHandler;
> sa.sa_flags = 0;
> sigemptyset(&sa.sa_mask);
> if (sigaction(SIGCHLD, &sa, NULL) == -1)
> errExit("sigaction");
>
> supervisor(sockPair);
>
> exit(EXIT_SUCCESS);
> }
>
> SEE ALSO
> ioctl(2), seccomp(2)
>
> A further example program can be found in the kernel source file
> samples/seccomp/user-trap.c.
>
> Linux 2020-10-01 SECCOMP_USER_NOTIF(2)
Thank you so much for this documentation and example! :)
-Kees
[1] https://git.kernel.org/linus/dfe719fef03d752f1682fa8aeddf30ba501c8555
[2] https://lore.kernel.org/lkml/CAG48ez3kpEDO1x_HfvOM2R9M78Ach9O_4+Pjs-vLLfqvZL+13A@mail.gmail.com/
[3] https://lore.kernel.org/lkml/CAGXu5jKzif=vp6gn5ZtrTx-JTN367qFphobnt9s=awbaafwoUw@mail.gmail.com/
--
Kees Cook
Powered by blists - more mailing lists