linux-kernel - Kernel fs/ext3 dir

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <CAD501LUf52VuoSqR--_94e6XvNqkANEmoTx_7ro_4SBfHFjDXg@mail.gmail.com>
Date:	Fri, 14 Aug 2015 16:31:25 +0700
From:	Stefan Knecht <knecht@...hian.com>
To:	linux-kernel@...r.kernel.org
Subject: Kernel fs/ext3 dir_index internals

Hello all

I would like to know if my understanding of the process is correct and
would be glad if someone who knows the internals could confirm or
deny.

I'm not a regular reader of the list, so please CC my email address in
any replies.

My question is about the internal mechanics of the directory index on
an ext3 file system. I understand the basics of how it works and what
its purpose is, but I encountered the following event which got me
curious.

We have a dir_index on ext3 in use on an EL5 system running
2.6.18-371.4.1, where we saw a warning in syslog of the index running
full on one directory. Listing the directory contents showed over 9.1
millions files, which was apparently enough to fill the index. We
deleted all the files, and checked the directory contents using "ls"
and it showed just 1 file. 15 or so minutes later, another check
showed another 9.1 million files, and another alert in syslog that the
directory index is full again. The files were not newly created, they
were old files (some 3 months old).

That prompted me to investigate. A couple email threads and posts on
sites such as serverfault later, I came to the following theory:

- if present, the index is used whenever the contents of a directory
are accessed
- if the index is full, it may not reflect what's actually in the directory
- once space in the index is freed, the index is refreshed / re-populated

This would mean that:

-  "ls" would not show all the files in the directory - only what's
currently in the index
- a loop over the directory in PERL, such as this: perl -e
'for(<*>){((stat)[9]<(unlink))}' would only delete what's in the index

Is my understanding correct?

I'd appreciate some feedback as this has quite a few people puzzled.

Best regards and a good weekend!

Stefan

-- 

Stefan Knecht

Solutions Architect

Pythian - Love your data

knecht@...hian.com

Phone: +1-866-798-4426x1331

www.pythian.com

-- 

--

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/