[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <8d6dd3013b05225541821132398cb7615cdd874e.camel@codethink.co.uk>
Date: Mon, 02 Jun 2025 16:59:11 +0200
From: Marcel Ziswiler <marcel.ziswiler@...ethink.co.uk>
To: Juri Lelli <juri.lelli@...hat.com>
Cc: luca abeni <luca.abeni@...tannapisa.it>, linux-kernel@...r.kernel.org,
Ingo Molnar <mingo@...hat.com>, Peter Zijlstra <peterz@...radead.org>,
Vineeth Pillai <vineeth@...byteword.org>
Subject: Re: SCHED_DEADLINE tasks missing their deadline with
SCHED_FLAG_RECLAIM jobs in the mix (using GRUB)
Hi Juri
On Thu, 2025-05-29 at 11:39 +0200, Juri Lelli wrote:
> Hi Marcel,
>
> On 25/05/25 21:29, Marcel Ziswiler wrote:
> > Hi Luca
> >
> > On Fri, 2025-05-23 at 21:46 +0200, luca abeni wrote:
> > > Hi Marcel,
> > >
> > > sorry, but I have some additional questions to fully understand your
> > > setup...
> >
> > No Problem, I am happy to answer any questions :)
> >
> > > On Mon, 19 May 2025 15:32:27 +0200
> > > Marcel Ziswiler <marcel.ziswiler@...ethink.co.uk> wrote:
> > > [...]
> > > > > just a quick question to better understand your setup (and check
> > > > > where the issue comes from):
> > > > > in the email below, you say that tasks are statically assigned to
> > > > > cores; how did you do this? Did you use isolated cpusets,
> > > >
> > > > Yes, we use the cpuset controller from the cgroup-v2 APIs in the
> > > > linux kernel in order to partition CPUs and memory nodes. In detail,
> > > > we use the AllowedCPUs and AllowedMemoryNodes in systemd's slice
> > > > configurations.
> > >
> > > How do you configure systemd? I am having troubles in reproducing your
> > > AllowedCPUs configuration... This is an example of what I am trying:
> > > sudo systemctl set-property --runtime custom-workload.slice AllowedCPUs=1
> > > sudo systemctl set-property --runtime init.scope AllowedCPUs=0,2,3
> > > sudo systemctl set-property --runtime system.slice AllowedCPUs=0,2,3
> > > sudo systemctl set-property --runtime user.slice AllowedCPUs=0,2,3
> > > and then I try to run a SCHED_DEADLINE application with
> > > sudo systemd-run --scope -p Slice=custom-workload.slice <application>
> >
> > We just use a bunch of systemd configuration files as follows:
> >
>
> ...
>
> > > How are you configuring the cpusets?
> >
> > See above.
> >
>
> Could you please add 'debug sched_debug sched_verbose' to your kernel
> cmdline and share the complete dmesg before starting your tests?
Sure, here you go [1].
> Also, I am attaching a script that should be able to retrieve cpuset
> information if you run it with
>
> # python3 get_cpuset_info.py > cpuset.out
>
> Could you please also do that and share the collected information?
[root@...alhost ~]# python3 get_cpuset_info.py > cpuset.out
[root@...alhost ~]# cat cpuset.out
Recursively retrieving cpuset information from /sys/fs/cgroup (cgroup v2):
Cgroup: /
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
------------------------------
Cgroup: /safety3.slice
cpuset.cpus: 3
cpuset.mems:
cpuset.cpus.effective: 3
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective: 3
cpuset.cpus.partition: root
------------------------------
Cgroup: /sys-fs-fuse-connections.mount
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /sys-kernel-debug.mount
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /dev-mqueue.mount
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /user.slice
cpuset.cpus: 0
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /monitor.slice
cpuset.cpus: 0
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /safety1.slice
cpuset.cpus: 1
cpuset.mems:
cpuset.cpus.effective: 1
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective: 1
cpuset.cpus.partition: root
------------------------------
Cgroup: /sys-kernel-tracing.mount
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /init.scope
cpuset.cpus: 0
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice
cpuset.cpus: 0
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/systemd-networkd.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/systemd-udevd.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/system-serial\x2dgetty.slice
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/boot.mount
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/var-lib-containers.mount
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/auditd.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/system-modprobe.slice
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/systemd-journald.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/systemd-nsresourced.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/sshd.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/var-tmp.mount
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/test-audio.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/tmp.mount
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/systemd-userdbd.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/test-speaker.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/systemd-oomd.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/systemd-resolved.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/dbus.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/systemd-timesyncd.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/system-getty.slice
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/systemd-logind.service
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /system.slice/system-disk\x2dstat\x2dmonitoring.slice
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
Cgroup: /safety2.slice
cpuset.cpus: 2
cpuset.mems:
cpuset.cpus.effective: 2
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective: 2
cpuset.cpus.partition: root
------------------------------
Cgroup: /dev-hugepages.mount
cpuset.cpus:
cpuset.mems:
cpuset.cpus.effective: 0
cpuset.mems.effective: 0
cpuset.cpus.exclusive:
cpuset.cpus.exclusive.effective:
cpuset.cpus.partition: member
------------------------------
> It should help us to better understand your setup and possibly reproduce
> the problem you are seeing.
Sure, I am happy to help.
> Thanks!
Thanks you!
> Juri
[1] https://pastebin.com/khFApYgf
Cheers
Marcel
Powered by blists - more mailing lists