linux-kernel - Re: [PATCH v1 6/8] x86/tsc: Use seqcount_latch

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 4 Sep 2020 10:03:12 +0200
From:   peterz@...radead.org
To:     "Ahmed S. Darwish" <a.darwish@...utronix.de>
Cc:     Ingo Molnar <mingo@...hat.com>, Will Deacon <will@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        "Sebastian A. Siewior" <bigeasy@...utronix.de>,
        LKML <linux-kernel@...r.kernel.org>,
        Borislav Petkov <bp@...en8.de>, x86@...nel.org,
        "H. Peter Anvin" <hpa@...or.com>
Subject: Re: [PATCH v1 6/8] x86/tsc: Use seqcount_latch_t

On Fri, Sep 04, 2020 at 09:41:42AM +0200, peterz@...radead.org wrote:
> On Thu, Aug 27, 2020 at 01:40:42PM +0200, Ahmed S. Darwish wrote:
> 
> >  __always_inline void cyc2ns_read_begin(struct cyc2ns_data *data)
> >  {
> > +	seqcount_latch_t *seqcount;
> >  	int seq, idx;
> >  
> >  	preempt_disable_notrace();
> >  
> > +	seqcount = &this_cpu_ptr(&cyc2ns)->seq;
> >  	do {
> > -		seq = this_cpu_read(cyc2ns.seq.sequence);
> > +		seq = raw_read_seqcount_latch(seqcount);
> >  		idx = seq & 1;
> >  
> >  		data->cyc2ns_offset = this_cpu_read(cyc2ns.data[idx].cyc2ns_offset);
> >  		data->cyc2ns_mul    = this_cpu_read(cyc2ns.data[idx].cyc2ns_mul);
> >  		data->cyc2ns_shift  = this_cpu_read(cyc2ns.data[idx].cyc2ns_shift);
> >  
> > -	} while (unlikely(seq != this_cpu_read(cyc2ns.seq.sequence)));
> > +	} while (read_seqcount_latch_retry(seqcount, seq));
> >  }
> 
> So I worried about this change, it obviously generates worse code. But I
> was not expecting this:
> 
> Before:
> 
> 196: 0000000000000110   189 FUNC    GLOBAL DEFAULT    1 native_sched_clock
> 
> After:
> 
> 195: 0000000000000110   399 FUNC    GLOBAL DEFAULT    1 native_sched_clock
> 
> That's _210_ bytes extra!!
> 
> If you look at the disassembly of the thing after it's a complete
> trainwreck.

The below delta fixes it again.

---
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -68,21 +68,19 @@ early_param("tsc_early_khz", tsc_early_k
 
 __always_inline void cyc2ns_read_begin(struct cyc2ns_data *data)
 {
-	seqcount_latch_t *seqcount;
 	int seq, idx;
 
 	preempt_disable_notrace();
 
-	seqcount = &this_cpu_ptr(&cyc2ns)->seq;
 	do {
-		seq = raw_read_seqcount_latch(seqcount);
+		seq = this_cpu_read(cyc2ns.seq.seqcount.sequence);
 		idx = seq & 1;
 
 		data->cyc2ns_offset = this_cpu_read(cyc2ns.data[idx].cyc2ns_offset);
 		data->cyc2ns_mul    = this_cpu_read(cyc2ns.data[idx].cyc2ns_mul);
 		data->cyc2ns_shift  = this_cpu_read(cyc2ns.data[idx].cyc2ns_shift);
 
-	} while (read_seqcount_latch_retry(seqcount, seq));
+	} while (unlikely(seq != this_cpu_read(cyc2ns.seq.seqcount.sequence)));
 }
 
 __always_inline void cyc2ns_read_end(void)