| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <f7fc588b-12cf-95a8-6142-e4d112fb1689@linux.intel.com>
Date: Fri, 30 Oct 2020 21:26:36 +0800
From: "Ning, Hongyu" <hongyu.ning@...ux.intel.com>
To: "Joel Fernandes (Google)" <joel@...lfernandes.org>,
Nishanth Aravamudan <naravamudan@...italocean.com>,
Julien Desfossez <jdesfossez@...italocean.com>,
Peter Zijlstra <peterz@...radead.org>,
Tim Chen <tim.c.chen@...ux.intel.com>,
Vineeth Pillai <viremana@...ux.microsoft.com>,
Aaron Lu <aaron.lwe@...il.com>,
Aubrey Li <aubrey.intel@...il.com>, tglx@...utronix.de,
linux-kernel@...r.kernel.org
Cc: mingo@...nel.org, torvalds@...ux-foundation.org,
fweisbec@...il.com, keescook@...omium.org, kerrnel@...gle.com,
Phil Auld <pauld@...hat.com>,
Valentin Schneider <valentin.schneider@....com>,
Mel Gorman <mgorman@...hsingularity.net>,
Pawan Gupta <pawan.kumar.gupta@...ux.intel.com>,
Paolo Bonzini <pbonzini@...hat.com>, vineeth@...byteword.org,
Chen Yu <yu.c.chen@...el.com>,
Christian Brauner <christian.brauner@...ntu.com>,
Agata Gruza <agata.gruza@...el.com>,
Antonio Gomez Iglesias <antonio.gomez.iglesias@...el.com>,
graf@...zon.com, konrad.wilk@...cle.com, dfaggioli@...e.com,
pjt@...gle.com, rostedt@...dmis.org, derkling@...gle.com,
benbjiang@...cent.com,
Alexandre Chartre <alexandre.chartre@...cle.com>,
James.Bottomley@...senpartnership.com, OWeisse@...ch.edu,
Dhaval Giani <dhaval.giani@...cle.com>,
Junaid Shahid <junaids@...gle.com>, jsbarnes@...gle.com,
chris.hyser@...cle.com, Aubrey Li <aubrey.li@...ux.intel.com>,
"Paul E. McKenney" <paulmck@...nel.org>,
Tim Chen <tim.c.chen@...el.com>
Subject: Re: [PATCH v8 -tip 00/26] Core scheduling
On 2020/10/20 9:43, Joel Fernandes (Google) wrote:
> Eighth iteration of the Core-Scheduling feature.
>
> Core scheduling is a feature that allows only trusted tasks to run
> concurrently on cpus sharing compute resources (eg: hyperthreads on a
> core). The goal is to mitigate the core-level side-channel attacks
> without requiring to disable SMT (which has a significant impact on
> performance in some situations). Core scheduling (as of v7) mitigates
> user-space to user-space attacks and user to kernel attack when one of
> the siblings enters the kernel via interrupts or system call.
>
> By default, the feature doesn't change any of the current scheduler
> behavior. The user decides which tasks can run simultaneously on the
> same core (for now by having them in the same tagged cgroup). When a tag
> is enabled in a cgroup and a task from that cgroup is running on a
> hardware thread, the scheduler ensures that only idle or trusted tasks
> run on the other sibling(s). Besides security concerns, this feature can
> also be beneficial for RT and performance applications where we want to
> control how tasks make use of SMT dynamically.
>
> This iteration focuses on the the following stuff:
> - Redesigned API.
> - Rework of Kernel Protection feature based on Thomas's entry work.
> - Rework of hotplug fixes.
> - Address review comments in v7
>
> Joel: Both a CGroup and Per-task interface via prctl(2) are provided for
> configuring core sharing. More details are provided in documentation patch.
> Kselftests are provided to verify the correctness/rules of the interface.
>
> Julien: TPCC tests showed improvements with core-scheduling. With kernel
> protection enabled, it does not show any regression. Possibly ASI will improve
> the performance for those who choose kernel protection (can be toggled through
> sched_core_protect_kernel sysctl). Results:
> v8 average stdev diff
> baseline (SMT on) 1197.272 44.78312824
> core sched ( kernel protect) 412.9895 45.42734343 -65.51%
> core sched (no kernel protect) 686.6515 71.77756931 -42.65%
> nosmt 408.667 39.39042872 -65.87%
>
> v8 is rebased on tip/master.
>
> Future work
> ===========
> - Load balancing/Migration fixes for core scheduling.
> With v6, Load balancing is partially coresched aware, but has some
> issues w.r.t process/taskgroup weights:
> https://lwn.net/ml/linux-kernel/20200225034438.GA617271@z...
> - Core scheduling test framework: kselftests, torture tests etc
>
> Changes in v8
> =============
> - New interface/API implementation
> - Joel
> - Revised kernel protection patch
> - Joel
> - Revised Hotplug fixes
> - Joel
> - Minor bug fixes and address review comments
> - Vineeth
>
> create mode 100644 tools/testing/selftests/sched/config
> create mode 100644 tools/testing/selftests/sched/test_coresched.c
>
Adding 4 workloads test results for Core Scheduling v8:
- kernel under test: coresched community v8 from https://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git/log/?h=coresched-v5.9
- workloads:
-- A. sysbench cpu (192 threads) + sysbench cpu (192 threads)
-- B. sysbench cpu (192 threads) + sysbench mysql (192 threads, mysqld forced into the same cgroup)
-- C. uperf netperf.xml (192 threads over TCP or UDP protocol separately)
-- D. will-it-scale context_switch via pipe (192 threads)
- test machine setup:
CPU(s): 192
On-line CPU(s) list: 0-191
Thread(s) per core: 2
Core(s) per socket: 48
Socket(s): 2
NUMA node(s): 4
- test results:
-- workload A, no obvious performance drop in cs_on:
+----------------------+------+----------------------+------------------------+
| | ** | sysbench cpu * 192 | sysbench mysql * 192 |
+======================+======+======================+========================+
| cgroup | ** | cg_sysbench_cpu_0 | cg_sysbench_mysql_0 |
+----------------------+------+----------------------+------------------------+
| record_item | ** | Tput_avg (events/s) | Tput_avg (events/s) |
+----------------------+------+----------------------+------------------------+
| coresched_normalized | ** | 1.01 | 0.87 |
+----------------------+------+----------------------+------------------------+
| default_normalized | ** | 1 | 1 |
+----------------------+------+----------------------+------------------------+
| smtoff_normalized | ** | 0.59 | 0.82 |
+----------------------+------+----------------------+------------------------+
-- workload B, no obvious performance drop in cs_on:
+----------------------+------+----------------------+------------------------+
| | ** | sysbench cpu * 192 | sysbench cpu * 192 |
+======================+======+======================+========================+
| cgroup | ** | cg_sysbench_cpu_0 | cg_sysbench_cpu_1 |
+----------------------+------+----------------------+------------------------+
| record_item | ** | Tput_avg (events/s) | Tput_avg (events/s) |
+----------------------+------+----------------------+------------------------+
| coresched_normalized | ** | 1.01 | 0.98 |
+----------------------+------+----------------------+------------------------+
| default_normalized | ** | 1 | 1 |
+----------------------+------+----------------------+------------------------+
| smtoff_normalized | ** | 0.6 | 0.6 |
+----------------------+------+----------------------+------------------------+
-- workload C, known performance drop in cs_on since Core Scheduling v6:
+----------------------+------+---------------------------+---------------------------+
| | ** | uperf netperf TCP * 192 | uperf netperf UDP * 192 |
+======================+======+===========================+===========================+
| cgroup | ** | cg_uperf | cg_uperf |
+----------------------+------+---------------------------+---------------------------+
| record_item | ** | Tput_avg (Gb/s) | Tput_avg (Gb/s) |
+----------------------+------+---------------------------+---------------------------+
| coresched_normalized | ** | 0.46 | 0.48 |
+----------------------+------+---------------------------+---------------------------+
| default_normalized | ** | 1 | 1 |
+----------------------+------+---------------------------+---------------------------+
| smtoff_normalized | ** | 0.82 | 0.79 |
+----------------------+------+---------------------------+---------------------------+
-- workload D, new added syscall workload, performance drop in cs_on:
+----------------------+------+-------------------------------+
| | ** | will-it-scale * 192 |
| | | (pipe based context_switch) |
+======================+======+===============================+
| cgroup | ** | cg_will-it-scale |
+----------------------+------+-------------------------------+
| record_item | ** | threads_avg |
+----------------------+------+-------------------------------+
| coresched_normalized | ** | 0.2 |
+----------------------+------+-------------------------------+
| default_normalized | ** | 1 |
+----------------------+------+-------------------------------+
| smtoff_normalized | ** | 0.89 |
+----------------------+------+-------------------------------+
comments: per internal analyzing, suspected huge amount of spin_lock contention in cs_on, may lead to significant performance drop
- notes on test results record_item:
* coresched_normalized: smton, cs enabled, test result normalized by default value
* default_normalized: smton, cs disabled, test result normalized by default value
* smtoff_normalized: smtoff, test result normalized by default value
Powered by blists - more mailing lists