Date:   Thu, 10 Aug 2023 13:52:22 +0800
From:   Yin Fengwei <fengwei.yin@...el.com>
To:     Mateusz Guzik <mjguzik@...il.com>, <ying.huang@...el.com>,
        <feng.tang@...el.com>, kernel test robot <oliver.sang@...el.com>,
        Yujie Liu <yujie.liu@...el.com>,
        "Li, Philip" <philip.li@...el.com>
CC:     <oe-lkp@...ts.linux.dev>, <lkp@...el.com>,
        <linux-kernel@...r.kernel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Christian Brauner <brauner@...nel.org>
Subject: Re: hardening and other opts in kernel config used for benchmarking?

Add more LKP people.

On 8/8/23 23:25, Mateusz Guzik wrote:
> Hello,
> 
> I have no idea who should be in To: or Cc:, I grabbed names from an
> e-mail I got regarding one of previous changes.
> 
> Recently I benchmarked a change which added unconditional file
> position locking and found a minor regression from it (with profiler
> output to justify it):
> https://lore.kernel.org/linux-fsdevel/CAHk-=whJtLkYwEFTS9LcRiMjSqq_xswDeXo7hYNWT0Em6nL4Sw@mail.gmail.com/T/#m7c0cd6e913c6295732daea3c88f502bd4724ffb3
> 
> However, according to Christian the change was benchmarked by your
> machinery and no difference was found.
> 
> I briefly poked around and found that used configs have:
> CONFIG_RANDOMIZE_KSTACK_OFFSET=y
> 
> This is an optional and very expensive hardening feature, my question
> is if it was enabled on purpose. The cost comes from adding rdtsc to
> every syscall.
> 
> Looking at the rest of the config you have a mixed bag (e.g., hardened
> usercopy but *no* init_on_alloc) so I genuinely don't know.
We used a kernel config from Redhat Enterprise Linux distribution as
base config (it was from redhat kernel-4.18.0-187). And then use the
default value of new configs.


> 
> Given the high cost of the opt I would suggest removing it, as it
>From arch/Kconfig:

config RANDOMIZE_KSTACK_OFFSET                                                  
        bool "Support for randomizing kernel stack offset on syscall entry" if EXPERT
        default y                                                               
        depends on HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET                            
        depends on INIT_STACK_NONE || !CC_IS_CLANG || CLANG_VERSION >= 140000   
        help                                                                    
          The kernel stack offset can be randomized (after pt_regs) by          
          roughly 5 bits of entropy, frustrating memory corruption              
          attacks that depend on stack address determinism or                   
          cross-syscall address exposures.                                      
                                                                                
          The feature is controlled via the "randomize_kstack_offset=on/off"    
          kernel boot param, and if turned off has zero overhead due to its use 
          of static branches (see JUMP_LABEL).                                  
                                                                                
          If unsure, say Y.                                                  

I don't think we want to disable this kernel config in LKP.


> avoidably muddles the waters for single-threaded changes (one way or
> the other -- slowdowns can hide and speed ups go unnoticed).
> 
> I did not review the whole config.
> 
> Any comments? :)
IMHO, LKP cares more about the possible performance downgrade users could
observe and assume users are using distribution. So LKP sticks to a distribution
config and using the default value for new configs.


I did test the will-it-scale.read1 against the commit
   20ea1e7d13c1b544fe67c4a8dc3943bb1ab33e6f
and its parent commit:
   9e0ee0c7545c7ec012a53878e7687e05b87abc75
by using "read1_process -s 4 -t 1" on a IceLake platform with 48C/96T + 192G
memory. I don't have available Sapphire Rapid to run same test.

The config is with the CONFIG_RANDOMIZE_KSTACK_OFFSET and CONFIG_HARDENED_USERCOPY
disabled/enabled.


I found the result is not stable. stddev is around 7% ~ 8%. larger or close
the the performance difference. For average: there is no regression with
these two kernel configs disabled. Around 9% regression with these two kernel
configs enabled.

But from LKP perspective (Oliver, Yujie and Philip, correct me if I am wrong
here), this test result is not reported as regression because the stddev is
very close to the regression.


The specific result is as following:
parent without random/hardened_usercopy:      child without random/hardened_usercopy:
	4050865                                	3570609
	3732989                                	3971604
	4043509                                	3577899
	3644475                                	4106721
	4156399                                	3432816
	3633719                                	4207917
	3508474                                	3581210
	3363729                                	3548164
	3794146                                	3490865
	3449113                                	4088112
	
avg:	3737741                                avg:	3757591
stddev:	7%                                     stddev:	7%


parent with random/hardened_usercopy:         child with random/hardened_usercopy:
	3770330                               	4279949
	3824055                               	3377960
	3497524                               	3359246
	4244985                               	3462981
	3442711                               	3520535
	4418889                               	3436251
	4255512                               	3884103
	3605148                               	3480065
	3965194                               	3230780
	3654892                               	3372904

avg:	3867924                               avg:	3540477     9% regression. But with 8.7% stddev
stddev:	8.8%                                  stddev:	8.7%


Regards
Yin, Fengwei

> 
> Thanks,

Powered by blists - more mailing lists