linux-kernel - Re: [linus:master] [block] e70c301fae: stress-ng.aiol.ops_per

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Z3zlHVJ+eo8rf1O+@xsang-OptiPlex-9020>
Date: Tue, 7 Jan 2025 16:26:05 +0800
From: Oliver Sang <oliver.sang@...el.com>
To: Christoph Hellwig <hch@....de>
CC: Niklas Cassel <cassel@...nel.org>, <oe-lkp@...ts.linux.dev>,
	<lkp@...el.com>, <linux-kernel@...r.kernel.org>, Jens Axboe
	<axboe@...nel.dk>, <linux-block@...r.kernel.org>,
	<virtualization@...ts.linux.dev>, <linux-nvme@...ts.infradead.org>, "Damien
 Le Moal" <dlemoal@...nel.org>, <linux-btrfs@...r.kernel.org>,
	<linux-aio@...ck.org>, <oliver.sang@...el.com>
Subject: Re: [linus:master] [block]  e70c301fae: stress-ng.aiol.ops_per_sec
 49.6% regression

hi, Christoph Hellwig,

On Fri, Jan 03, 2025 at 07:49:25AM +0100, Christoph Hellwig wrote:
> On Thu, Jan 02, 2025 at 10:49:41AM +0100, Niklas Cassel wrote:
> > > > from below information, it seems an 'ahci' to me. but since I have limited
> > > > knowledge about storage driver, maybe I'm wrong. if you want more information,
> > > > please let us know. thanks a lot!
> > > 
> > > Yes, this looks like ahci.  Thanks a lot!
> > 
> > Did this ever get resolved?
> > 
> > I haven't seen a patch that seems to address this.
> > 
> > AHCI (ata_scsi_queuecmd()) only issues a single command, so if there is any
> > reordering when issuing a batch of commands, my guess is that the problem
> > also affects SCSI / the problem is in upper layers above AHCI, i.e. SCSI lib
> > or block layer.
> 
> I started looking into this before the holidays.  blktrace shows perfectly
> sequential writes without any reordering using ahci, directly on the
> block device or using xfs and btrfs when using dd.  I also started
> looking into what the test does and got as far as checking out the
> stress-ng source tree and looking at stress-aiol.c.  AFAICS the default
> submission does simple reads and writes using increasing offsets.
> So if the test result isn't a fluke either the aio code does some
> weird reordering or btrfs does.
> 
> Oliver, did the test also show any interesting results on non-btrfs
> setups?
> 

I tried to run with ext4 fs [1] and xfs [2], seems not be able to get stable
results (%stddev is too big, even bigger than %change). seems no value from
both tests.


[1]
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime:
  gcc-12/performance/1HDD/ext4/x86_64-rhel-9.4/100%/debian-12-x86_64-20240206.cgz/lkp-icl-2sp8/aiol/stress-ng/60s

a3396b99990d8b4e e70c301faece15b618e54b613b1
---------------- ---------------------------
         %stddev     %change         %stddev
             \          |                \
    142.01 ± 17%      -4.6%     135.55 ± 18%  stress-ng.aiol.async_I/O_events_completed_per_sec
     14077 ± 14%      -3.3%      13617 ± 15%  stress-ng.aiol.ops
    233.95 ± 14%      -3.4%     225.97 ± 15%  stress-ng.aiol.ops_per_sec


[2]
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime:
  gcc-12/performance/1HDD/xfs/x86_64-rhel-9.4/100%/debian-12-x86_64-20240206.cgz/lkp-icl-2sp8/aiol/stress-ng/60s

a3396b99990d8b4e e70c301faece15b618e54b613b1
---------------- ---------------------------
         %stddev     %change         %stddev
             \          |                \
     11.97 ± 21%     +18.5%      14.19 ± 44%  stress-ng.aiol.async_I/O_events_completed_per_sec
      1498 ± 33%      +9.5%       1640 ± 49%  stress-ng.aiol.ops
     23.45 ± 34%     +10.2%      25.85 ± 52%  stress-ng.aiol.ops_per_sec