Message-ID: <IA1PR11MB617118DAE8D72E2419558D6689AC9@IA1PR11MB6171.namprd11.prod.outlook.com>
Date:   Tue, 28 Feb 2023 06:40:10 +0000
From:   "Zhuo, Qiuxu" <qiuxu.zhuo@...el.com>
To:     Joel Fernandes <joel@...lfernandes.org>
CC:     "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Frederic Weisbecker" <frederic@...nel.org>,
        Lai Jiangshan <jiangshanlai@...il.com>,
        "linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>,
        "Paul E. McKenney" <paulmck@...nel.org>,
        "rcu@...r.kernel.org" <rcu@...r.kernel.org>
Subject: RE: [PATCH RFC v2] rcu: Add a minimum time for marking boot as
 completed

> From: Joel Fernandes <joel@...lfernandes.org>
> Sent: Monday, February 27, 2023 9:22 PM
> To: Zhuo, Qiuxu <qiuxu.zhuo@...el.com>
> Cc: linux-kernel@...r.kernel.org; Frederic Weisbecker <frederic@...nel.org>;
> Lai Jiangshan <jiangshanlai@...il.com>; linux-doc@...r.kernel.org; Paul E.
> McKenney <paulmck@...nel.org>; rcu@...r.kernel.org
> Subject: Re: [PATCH RFC v2] rcu: Add a minimum time for marking boot as
> completed
> 
> 
> 
> > On Feb 27, 2023, at 2:53 AM, Zhuo, Qiuxu <qiuxu.zhuo@...el.com> wrote:
> >
> > 
> >>
> >> From: Joel Fernandes (Google) <joel@...lfernandes.org>
> >> Sent: Saturday, February 25, 2023 11:34 AM
> >> To: linux-kernel@...r.kernel.org
> >> Cc: Joel Fernandes (Google) <joel@...lfernandes.org>; Frederic
> >> Weisbecker <frederic@...nel.org>; Lai Jiangshan
> >> <jiangshanlai@...il.com>; linux- doc@...r.kernel.org; Paul E.
> >> McKenney <paulmck@...nel.org>; rcu@...r.kernel.org
> >> Subject: [PATCH RFC v2] rcu: Add a minimum time for marking boot as
> >> completed
> >>
> >> On many systems, a great deal of boot happens after the kernel thinks
> >> the boot has completed. It is difficult to determine if the system
> >> has really booted from the kernel side. Some features like lazy-RCU
> >> can risk slowing down boot time if, say, a callback has been added
> >> that the boot synchronously depends on.
> >>
> >> Further, it is better to boot systems which pass
> >> 'rcu_normal_after_boot' to stay expedited for as long as the system is still
> booting.
> >>
> >> For these reasons, this commit adds a config option
> >> 'CONFIG_RCU_BOOT_END_DELAY' and a boot parameter
> >> rcupdate.boot_end_delay.
> >>
> >> By default, this value is 20s. A system designer can choose to
> >> specify a value here to keep RCU from marking boot completion.  The
> >> boot sequence will not be marked ended until at least boot_end_delay
> milliseconds have passed.
> >
> > Hi Joel,
> >
> > Just some thoughts on the default value of 20s, correct me if I'm wrong :-).
> >
> > Does the OS with CONFIG_PREEMPT_RT=y kernel concern more about the
> > real-time latency than the overall OS boot time?
> 
> But every system has to boot, even an RT system.

   Yes, this is true.

> >
> > If so, we might make rcupdate.boot_end_delay = 0 as the default value
> > (NOT the default 20s) for CONFIG_PREEMPT_RT=y kernels?
> 
> Could you measure how much time your RT system takes to boot before the
> application runs?

I don't have a real-time OS environment to measure the OS boot time. 
I tried to measure the OS boot time of my "CentOS Stream 8" w/o and
w/ Joel’s patch. 

My testing showed the positive result that the OS boot time was 
reduced by ~4.6% on my side after applying Joel’s patch.

For testing details, please see the below:

1) Testing environment:
    OS            : CentOS Stream 8 (non-RT OS)
    Kernel     : v6.2
    Machine : Intel Cascade Lake server (2 sockets, each with 44 logical threads)
    Qemu  args  : -cpu host -enable-kvm, -smp 88,threads=2,sockets=2, … 

2) My OS boot time definition: 
    The time from the start of the kernel boot to the shell command line 
    prompt is shown from the console. [ Different people may have
    different OS boot time definitions. ]
  
3) My measurement method (very rough method): 
    A timer in the kernel periodically prints the boot time every 100ms. 
    As soon as the shell command line prompt is shown from the console,
    we record the boot time printed by the timer, then the printed boot 
    time is the OS boot time. 
   
    The console log (mixed userspace and kernel logs) looked like this:

           [  OK  ] Started Permit User Sessions.
                        Starting Terminate Plymouth Boot Screen...
                        Starting Hold until boot process finishes up...
           [  OK  ] Started Command Scheduler.
           [    6.824466] input: ImExPS/2 Generic Explorer ...
           [    6.884685] Boot ms 6863
		...
           [    7.170920] Spectre V2 : WARNING: Unprivileged eBPF ...
           [    7.173140] Spectre V2 : WARNING: Unprivileged eBPF ...
           [    7.196741] Boot ms 7175
		...
           [    8.236757] Boot ms 8215

           CentOS Stream 8
           Kernel 6.2.0-rcu+ on an x86_64

           login: [    8.340751] Boot ms 8319
           [    8.444756] Boot ms 8423
		...

     Then the log "login: [    8.340751] Boot ms 8319" roughly showed the OS boot time was ~8.3s.

4) Measured OS boot time (in seconds)
   a) Measured 10 times w/o Joel's patch:
        8.7s, 8.4s, 8.6s, 8.2s, 9.0s, 8.7s, 8.8s, 9.3s, 8.8s, 8.3s
        The average OS boot time was: ~8.7s
   
   b) Measure 10 times w/ Joel's patch: 
        8.5s, 8.2s, 7.6s, 8.2s, 8.7s, 8.2s, 7.8s, 8.2s, 9.3s, 8.4s
        The average OS boot time was: ~8.3s.
		 
The OS boot time was reduced by : 8.7 – 8.3 = 0.4 second
The reduction percentage was       : 0.4/8.7 * 100% = 4.6%

If the testing above makes sense to you, please feel free to 
add

      Tested-by: Qiuxu Zhuo <qiuxu.zhuo@...el.com>

Thanks!
-Qiuxu

> I can change it to default 0 essentially NOOPing it, but I would rather have a
> saner default (10 seconds even), than having someone forget to tune this for
> their system.
> 
> Thanks,
> 
>  - Joel

Powered by blists - more mailing lists