[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <3c4b9025-2667-ca8c-7427-502068b8168e@amd.com>
Date: Fri, 14 Apr 2023 12:55:14 +0530
From: Ayush Jain <ayush.jain3@....com>
To: broonie@...nel.org,
Linux Next Mailing List <linux-next@...r.kernel.org>
Cc: Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
axboe@...nel.dk, Santosh.Shukla@....com, Kalpana.Shetty@....com,
Narasimhan V <Narasimhan.V@....com>, sfr@...b.auug.org.au
Subject: Re: linux-next: Tree for Apr 13
Hello,
On 4/14/2023 10:41 AM, Jain, Ayush wrote:
> Hello Broonie,
>
> When Running fio-test on latest linux-next tree, I noticed that test hung indefinitely, Going back I see that this problem exists since
> next-20230316 release, After bisecting I landed on the following merge commit by Jens.
>
> Commit 097d3ca138f9 ("Merge branch 'for-6.4/splice' into for-next")
>
> Running perf I see following trace and call-stack for fio:
>
> Overhead Command Shared Object Symbol
> 25.08% fio [kernel.vmlinux] [k] copy_user_generic_string
> copy_user_generic_string
> __do_splice
> __x64_sys_splice
> do_syscall_64
> entry_SYSCALL_64_after_hwframe
> splice
> 0x1c44be0
> ...
>
> On a good kernel I see the following perf trace:
>
> Overhead Command Shared Object Symbol
> 49.93% fio fio [.] fio_crc32
> 7.23% fio fio [.] clock_thread_fn
> 2.10% fio [kernel.vmlinux] [k] clear_page_rep
> 1.55% fio fio [.] __fill_random_buf
> 1.35% fio [kernel.vmlinux] [k] loop_queue_rq
> 1.05% fio [kernel.vmlinux] [k] copy_user_generic_string
> ...
>
> I see some splice changes being added as the part of merge
> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/log/?h=for-6.4/splice
>
> I observe this problem on 3 EPYC system(Zen1,3,4), with the following disk architecture
>
> Zen1: nvme0n1 931.5G Samsung SSD 970 EVO Plus 1TB
> Zen4: nvme0n1 232.9G Samsung SSD 960 EVO 250GB
>
> I am running fio as follows:
>
> $fio fio-simple.job --filename=/dev/test_vg/test_lv
>
> where test_lv is mounted as follows:
>
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
> loop0 7:0 0 12.2G 0 loop
> └─md127 9:127 0 12.2G 0 raid0
> └─test_vg-test_lv 253:3 0 5.5G 0 lvm
>
> You can find fio-simple.job at
> https://github.com/avocado-framework-tests/avocado-misc-tests/blob/master/io/disk/fiotest.py.data/fio-simple.job
>
> Fio Version: fio-3.34-25-g07ed
>
Also adding to these observations
-If we create a filesystem on the raw disk -- Test completes with a Pass
-If there is no Filesystem on the raw disk(loop, nvme) -- Test hangs with the provided trace
>
> Regards
> Ayush Jain
> > On 4/13/2023 11:55 PM, broonie@...nel.org wrote:
>> Hi all,
>>
>> Changes since 20230411:
>>
>> The sh tree gained a conflict with the mm-unstable tree.
>>
>> The erofs tree gained a conflict with the vfs-idmapping tree.
>>
>> The ext4 tree gained multiple conflicts with the mm-stable tree.
>>
>> The net-next tree gained a conflict with the origin tree.
>>
>> The bpf-next tree gained a conflict with the net-net tree.
>>
>> Non-merge commits (relative to Linus' tree): 10382
>> 11028 files changed, 580165 insertions(+), 249563 deletions(-)
>>
>> ----------------------------------------------------------------------------
>>
>> I have created today's linux-next tree at
>> git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
>> (patches at http://www.kernel.org/pub/linux/kernel/next/ ). If you
>> are tracking the linux-next tree using git, you should not use "git pull"
>> to do so as that will try to merge the new linux-next release with the
>> old one. You should use "git fetch" and checkout or reset to the new
>> master.
>>
>> You can see which trees have been included by looking in the Next/Trees
>> file in the source. There is also the merge.log file in the Next
>> directory. Between each merge, the tree was built with a ppc64_defconfig
>> for powerpc, an allmodconfig for x86_64, a multi_v7_defconfig for arm
>> and a native build of tools/perf. After the final fixups (if any), I do
>> an x86_64 modules_install followed by builds for x86_64 allnoconfig,
>> powerpc allnoconfig (32 and 64 bit), ppc44x_defconfig, allyesconfig
>> and pseries_le_defconfig and i386, arm64, s390, sparc and sparc64
>> defconfig and htmldocs. And finally, a simple boot test of the powerpc
>> pseries_le_defconfig kernel in qemu (with and without kvm enabled).
>>
>> Below is a summary of the state of the merge.
>>
>> I am currently merging 357 trees (counting Linus' and 102 trees of bug
>> fix patches pending for the current merge release).
>>
>> Stats about the size of the tree over time can be seen at
>> http://neuling.org/linux-next-size.html .
>>
>> Status of my local build tests will be at
>> http://kisskb.ellerman.id.au/linux-next . If maintainers want to give
>> advice about cross compilers/configs that work, we are always open to add
>> more builds.
>>
>> Thanks to Randy Dunlap for doing many randconfig builds. And to Paul
>> Gortmaker for triage and bug fixes.
>
Regards,
Ayush Jain
Powered by blists - more mailing lists