linux-kernel - Re: rm -f * on large files very slow on XFS + MD RAID 6 volume of 15x 4TB of HDDs (52TB)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140423082538.GR18672@dastard>
Date:	Wed, 23 Apr 2014 18:25:38 +1000
From:	Dave Chinner <david@...morbit.com>
To:	Ivan Pantovic <gyro.ivan@...il.com>
Cc:	Speedy Milan <speedy.milan@...il.com>,
	linux-kernel@...r.kernel.org, xfs@....sgi.com
Subject: Re: rm -f * on large files very slow on XFS + MD RAID 6 volume of
 15x 4TB of HDDs (52TB)

On Wed, Apr 23, 2014 at 09:23:41AM +0200, Ivan Pantovic wrote:
> 
> >[root@...ve-b ~]# xfs_db -r /dev/md0
> >xfs_db> frag
> >actual 11157932, ideal 11015175, fragmentation factor 1.28%
> >xfs_db>
> 
> this is current level of fragmentation ... is it bad?

http://xfs.org/index.php/XFS_FAQ#Q:_The_xfs_db_.22frag.22_command_says_I.27m_over_50.25._Is_that_bad.3F

> some say over 1% is candidate for defrag? ...

Some say that over 70% is usually not a problem:

http://www.mythtv.org/wiki/XFS_Filesystem#Defragmenting_XFS_Partitions

i.e. the level that becomes are problem is highly workload specific.
So, you can't read *anything* in that number without know exactly
what is in your filesystem, how the application(s) interact with it
and so on. 

Besides, I was asking specifically about the files you removed, not
the files that remain in the filesystem. Given that you have 11
million inodes in the filesystem, you probably removed the only
significantly large files in the filesystem....

So, the files your removed are now free space, so free space
fragmentation is what we need to look at.  i.e. use the freesp
command to dump the histogram and summary of the free space...

Cheers,

Dave.
-- 
Dave Chinner
david@...morbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/