[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <CALy5rjUMnocsh80gPB+4UgaFS-Gsz5KAFnAN8Nj7m_oyohFfvg@mail.gmail.com>
Date: Wed, 16 Oct 2024 19:09:29 +0800
From: Xiongwei Song <sxwbruce@...il.com>
To: cem@...nel.org, djwong@...nel.org, linux-xfs@...r.kernel.org
Cc: linux-kernel@...r.kernel.org
Subject: XFS performance degradation during running cp command with big test file
Dear Experts,
We are facing a performance degradation on the XFS partition. We
was trying to copy a big file(200GB ~ 250GB) from a path to /dev/null,
when performing cp command to 60s ~ 90s, the reading speed was
suddenly down. At the beginning, the reading speed was around
1080MB/s, 60s later the speed was down to around 350MB/s. This
problem is only found with XFS + Thick LUN.
The test environment:
Storage Model: Dell unity XT 380 Think/Thin LUN
Linux Version: 4.12.14
The steps to run test:
1) Create a xfs partition with following commands
parted -a opt /dev/sdb mklabel gpt mkpart sdb xfs 0% 100%
mkfs.xfs /dev/sdbx
mount /dev/sdbx /xfs
2) Create a ~200GB file named fileA in the partition.
3) Run cp command to copy the file created in step 2. Meanwhile,
run iostat vmstat and blktrace to capture logs.
cp /xfs/fileA /dev/null
To narrow down this issue, we also did some experiments
below to compare:
1) Run the test with dd command with XFS + Thick LUN
dd if=/xfs/fileA of=/dev/null bs=32k status=progress
Result: also meet performance degradation
Speed: around 650MB/S
Speed has changed to around 350MB/S since the 60s ~ 90s of cp run.
2) Run the test with dd command with raw device with XFS + Thick LUN
dd if=/dev/sdbx of=/dev/null bs=32k status=progress
Results: No performance degradation
Speed: around 520MB/s
3) Run run test with ext4 + Think LUN
cp /xfs/fileA /dev/null
Results: No performance degradation
Speed: around 1080MB/s
4) Run the test with cp with XFS + Thin LUN
cp /xfs/fileA /dev/null
Result: No performance degradation
Speed: around 500MB/s
5) Run the test with dd with XFS + Thin LUN
dd if=/xfs/fileA of=/dev/null bs=32k status=progress
Result: No performance degradation
Speed: around 500MB/s
It seems the issue only can be triggered with XFS + Thick LUN,
no matter dd or cp to read the test file. We would like to learn
if there is something special with XFS in this test situation?
Is it known?
Do you have any thoughts or suggestions? Also, do you need vmstat
or iostat logs or blktrace or any other logs to address this issue?
Thank you in advance.
Regards,
Bruce
Powered by blists - more mailing lists