linux-kernel - rm -f * on large files very slow on XFS + MD RAID 6 volume of 15x 4TB of HDDs (52TB)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 21 Apr 2014 22:58:53 +0200
From:	Speedy Milan <speedy.milan@...il.com>
To:	linux-kernel@...r.kernel.org
Cc:	Ivan Pantovic <gyro.ivan@...il.com>
Subject: rm -f * on large files very slow on XFS + MD RAID 6 volume of 15x 4TB
 of HDDs (52TB)

I want to report very slow deletion of 24 50GB files (in total 12 TB),
all present in the same folder.

OS is CentOS 6.4, with upgraded kernel 3.13.1.

The hardware is a Supermicro server with 15x 4TB WD Se drives in MD
RAID 6, totalling 52TB of free space.

XFS is formated directly on the RAID volume, without LVM layers.

Deletion was done with rm -f * command, and it took upwards of 1 hour
to delete the files.

File system was filled completely prior to deletion.

rm was mostly waiting (D state), probably for kworker threads, and
iostat was showing big HDD utilization numbers and very low throughput
so it looked like a random HDD workload was in effect.

rm was the only IO task at the moment.

I found it strange that deleting large files should take such a long
time, so I thought it would be wise to report.

Thanks,
Milan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/