| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CA+icZUXu15=NK8wQgy=eeu=JcOGfB4Qr6UnwzTVvcH4T1L4pUQ@mail.gmail.com>
Date: Thu, 28 May 2020 04:12:02 +0200
From: Sedat Dilek <sedat.dilek@...il.com>
To: Marco Elver <elver@...gle.com>
Cc: Arnd Bergmann <arnd@...db.de>, Will Deacon <will@...nel.org>,
Peter Zijlstra <peterz@...radead.org>,
Borislav Petkov <bp@...en8.de>,
Thomas Gleixner <tglx@...utronix.de>,
Ingo Molnar <mingo@...nel.org>,
clang-built-linux <clang-built-linux@...glegroups.com>,
"Paul E. McKenney" <paulmck@...nel.org>,
Dmitry Vyukov <dvyukov@...gle.com>,
Alexander Potapenko <glider@...gle.com>,
Andrey Konovalov <andreyknvl@...gle.com>,
kasan-dev <kasan-dev@...glegroups.com>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
Nick Desaulniers <ndesaulniers@...gle.com>,
Stephen Rothwell <sfr@...b.auug.org.au>
Subject: Re: [PATCH -tip] compiler_types.h: Optimize __unqual_scalar_typeof
compilation time
On Wed, May 27, 2020 at 9:15 PM Marco Elver <elver@...gle.com> wrote:
>
> On Wed, 27 May 2020 at 21:11, Sedat Dilek <sedat.dilek@...il.com> wrote:
> >
> > On Wed, May 27, 2020 at 3:57 PM Marco Elver <elver@...gle.com> wrote:
> > >
> > > On Wed, 27 May 2020 at 15:37, Sedat Dilek <sedat.dilek@...il.com> wrote:
> > > >
> > > > On Wed, May 27, 2020 at 3:30 PM Marco Elver <elver@...gle.com> wrote:
> > > > >
> > > > > On Wed, 27 May 2020 at 15:11, Sedat Dilek <sedat.dilek@...il.com> wrote:
> > > > > >
> > > > > > On Wed, May 27, 2020 at 2:50 PM Arnd Bergmann <arnd@...db.de> wrote:
> > > > > > >
> > > > > > > On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <sedat.dilek@...il.com> wrote:
> > > > > > > > On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <arnd@...db.de> wrote:
> > > > > > > > > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <sedat.dilek@...il.com> wrote:
> > > > > > > > > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <arnd@...db.de> wrote:
> > > > > > > > > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <elver@...gle.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > > > > > > > > of the drop I saw with gcc, compared to current mainline.
> > > > > > > > > > >
> > > > > > > > > > > Tested-by: Arnd Bergmann <arnd@...db.de>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi Arnd,
> > > > > > > > > >
> > > > > > > > > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> > > > > > > > >
> > > > > > > > > I meant v5.7.
> > > > > > > > >
> > > > > > > > > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > > > > > > > > >
> > > > > > > > > > Is there a speedup benefit also for Linux v5.7?
> > > > > > > > > > Which patches do I need?
> > > > > > > > >
> > > > > > > > > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > > > > > > > > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > > > > > > > > linux-next
> > > > > > > > > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > > > > > > > > almost back to the speed of linux-5.7.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Which clang version did you use - and have you set KCSAN kconfigs -
> > > > > > > > AFAICS this needs clang-11?
> > > > > > >
> > > > > > > I'm currently using clang-11, but I see the same problem with older
> > > > > > > versions, and both with and without KCSAN enabled. I think the issue
> > > > > > > is mostly the deep nesting of macros that leads to code bloat.
> > > > > > >
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > With clang-10:
> > > > > >
> > > > > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > > > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > > > > +HAVE_ARCH_KCSAN y
> > > > >
> > > > > Clang 10 doesn't support KCSAN (HAVE_KCSAN_COMPILER unset).
> > > > >
> > > > > > With clang-11:
> > > > > >
> > > > > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > > > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > > > > CLANG_VERSION 100001 -> 110000
> > > > > > +CC_HAS_ASM_INLINE y
> > > > > > +HAVE_ARCH_KCSAN y
> > > > > > +HAVE_KCSAN_COMPILER y
> > > > > > +KCSAN n
> > > > > >
> > > > > > Which KCSAN kconfigs did you enable?
> > > > >
> > > > > To clarify: as said in [1], KCSAN (or any other instrumentation) is no
> > > > > longer relevant to the issue here, and the compile-time regression is
> > > > > observable with most configs. The problem is due to pre-processing and
> > > > > parsing, which came about due to new READ_ONCE() and the
> > > > > __unqual_scalar_typeof() macro (which this patch optimizes).
> > > > >
> > > > > KCSAN and new ONCEs got tangled up because we first attempted to
> > > > > annotate {READ,WRITE}_ONCE() with data_race(), but that turned out to
> > > > > have all kinds of other issues (explanation in [2]). So we decided to
> > > > > drop all the KCSAN-specific bits from ONCE, and require KCSAN to be
> > > > > Clang 11. Those fixes were applied to the first version of new
> > > > > {READ,WRITE}_ONCE() in -tip, which actually restored the new ONCEs to
> > > > > the pre-KCSAN version (now that KCSAN can deal with them without
> > > > > annotations).
> > > > >
> > > > > Hope this makes more sense now.
> > > > >
> > > > > [1] https://lore.kernel.org/lkml/CANpmjNOUdr2UG3F45=JaDa0zLwJ5ukPc1MMKujQtmYSmQnjcXg@mail.gmail.com/
> > > > > [2] https://lore.kernel.org/lkml/20200521142047.169334-1-elver@google.com/
> > > > >
> > > >
> > > > Thanks, Marco.
> > > >
> > > > I pulled tip.git#locking/kcsan on top of Linux v5.7-rc7 and applied this patch.
> > > > Just wanted to try KCSAN for the first time and it will also be my
> > > > first building with clang-11.
> > > > That's why I asked.
> > >
> > > In general, CONFIG_KCSAN=y and the defaults for the other KCSAN
> > > options should be good. Depending on the size of your system, you
> > > could also tweak KCSAN runtime performance:
> > > https://lwn.net/Articles/816850/#Interacting%20with%20KCSAN%20at%20Runtime
> > > -- the defaults should be good for most systems though.
> > > Hope this helps. Any more questions, do let me know.
> > >
> >
> > Which "projects" and packages do I need?
> >
> > I have installed:
> >
> > # LC_ALL=C apt-get install llvm-11 clang-11 lld-11
> > --no-install-recommends -t llvm-toolchain -y
> >
> > # dpkg -l | grep
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261 | awk
> > '/^ii/ {print $1 " " $2 " " $3}' | column -t
> > ii clang-11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii libclang-common-11-dev
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii libclang-cpp11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii libclang1-11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii libllvm11:amd64
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii lld-11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii llvm-11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii llvm-11-runtime
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> >
> > Is that enough?
>
> Just clang-11 (and its transitive dependencies) is enough. Unsure what
> your installed binary is, likely "clang-11", so if you can do "make
> CC=clang-11 defconfig" (and check for CONFIG_HAVE_KCSAN_COMPILER)
> you're good to go.
>
I was able to build with clang-11 from apt.llvm.org.
[ build-time ]
Normally, it takes me approx. 05:00 to build with clang-10
(10.0.1-rc1) and Linux v5.7-rc7.
This time start: 21:18 and stop: 03:45 means 06:27 - took 01:27 longer.
Samsung Ultrabook 2nd generation aka Intel Sandybridge CPU with 'make -j3'.
[ diffconfig ]
BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
CLANG_VERSION 100001 -> 110000
+CC_HAS_ASM_INLINE y
+HAVE_ARCH_KCSAN y
+HAVE_KCSAN_COMPILER y
+KCSAN y
+KCSAN_ASSUME_PLAIN_WRITES_ATOMIC y
+KCSAN_DEBUG n
+KCSAN_DELAY_RANDOMIZE y
+KCSAN_EARLY_ENABLE y
+KCSAN_IGNORE_ATOMICS n
+KCSAN_INTERRUPT_WATCHER n
+KCSAN_NUM_WATCHPOINTS 64
+KCSAN_REPORT_ONCE_IN_MS 3000
+KCSAN_REPORT_RACE_UNKNOWN_ORIGIN y
+KCSAN_REPORT_VALUE_CHANGE_ONLY y
+KCSAN_SELFTEST y
+KCSAN_SKIP_WATCH 4000
+KCSAN_SKIP_WATCH_RANDOMIZE y
+KCSAN_UDELAY_INTERRUPT 20
+KCSAN_UDELAY_TASK 80
I am seeing this data-races:
root@...za:~# LC_ALL=C dmesg -T | grep 'BUG: KCSAN: data-race'
[Thu May 28 03:51:53 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:00 2020] BUG: KCSAN: data-race in mark_page_accessed
/ workingset_activation
[Thu May 28 03:52:02 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:08 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:52:10 2020] BUG: KCSAN: data-race in dd_has_work /
dd_insert_requests
[Thu May 28 03:52:11 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:13 2020] BUG: KCSAN: data-race in
page_counter_try_charge / page_counter_try_charge
[Thu May 28 03:52:15 2020] BUG: KCSAN: data-race in ep_poll_callback /
ep_send_events_proc
[Thu May 28 03:52:21 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:25 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:26 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:52:31 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:52:38 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:52:53 2020] BUG: KCSAN: data-race in dd_has_work /
dd_insert_requests
[Thu May 28 03:52:56 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:52:59 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:53:25 2020] BUG: KCSAN: data-race in
rwsem_spin_on_owner+0x102/0x1a0
[Thu May 28 03:53:25 2020] BUG: KCSAN: data-race in
page_counter_try_charge / page_counter_try_charge
[Thu May 28 03:53:39 2020] BUG: KCSAN: data-race in do_epoll_wait /
ep_poll_callback
[Thu May 28 03:53:39 2020] BUG: KCSAN: data-race in find_next_and_bit+0x30/0xd0
[Thu May 28 03:53:41 2020] BUG: KCSAN: data-race in dd_has_work /
dd_insert_requests
[Thu May 28 03:53:43 2020] BUG: KCSAN: data-race in do_epoll_wait /
ep_poll_callback
[Thu May 28 03:53:45 2020] BUG: KCSAN: data-race in dd_has_work /
dd_insert_requests
[Thu May 28 03:53:46 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:53:47 2020] BUG: KCSAN: data-race in
rwsem_spin_on_owner+0x102/0x1a0
[Thu May 28 03:54:02 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:54:11 2020] BUG: KCSAN: data-race in find_next_and_bit+0x30/0xd0
[Thu May 28 03:54:19 2020] BUG: KCSAN: data-race in
rwsem_spin_on_owner+0x102/0x1a0
[Thu May 28 03:55:00 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:56:14 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:56:50 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:56:50 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:56:52 2020] BUG: KCSAN: data-race in
tick_nohz_next_event / tick_nohz_stop_tick
[Thu May 28 03:56:58 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:57:58 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:58:00 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:58:07 2020] BUG: KCSAN: data-race in
tick_nohz_next_event / tick_nohz_stop_tick
[Thu May 28 03:58:44 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:58:49 2020] BUG: KCSAN: data-race in __bitmap_subset+0x38/0xd0
[Thu May 28 03:59:46 2020] BUG: KCSAN: data-race in
tick_nohz_next_event / tick_nohz_stop_tick
[Thu May 28 04:00:25 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 04:00:26 2020] BUG: KCSAN: data-race in
tick_nohz_next_event / tick_nohz_stop_tick
Full dmesg output and linux-config attached.
- Sedat -
View attachment "dmesg-T_5.7.0-rc7-3-amd64-clang.txt" of type "text/plain" (128379 bytes)
Download attachment "config-5.7.0-rc7-3-amd64-clang" of type "application/octet-stream" (230128 bytes)
Powered by blists - more mailing lists