linux-kernel - Re: File system for scratch space (in HPC cluster)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <3f258920-7fbc-22c6-03ed-152db872644c@molgen.mpg.de>
Date:   Fri, 25 Oct 2019 10:33:10 +0200
From:   Paul Menzel <pmenzel@...gen.mpg.de>
To:     "Theodore Y. Ts'o" <tytso@....edu>,
        Boaz Harrosh <openosd@...il.com>
Cc:     linux-fsdevel@...r.kernel.org,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Donald Buczek <buczek@...gen.mpg.de>
Subject: Re: File system for scratch space (in HPC cluster)

Dear Boaz, dear Theodore,


Thank you for your replies.

On 2019-10-24 22:34, Theodore Y. Ts'o wrote:
> On Thu, Oct 24, 2019 at 06:01:05PM +0300, Boaz Harrosh wrote:
>>> You could use ext4 in nojournal mode.  If you want to make sure that
>>> fsync() doesn't force a cache flush, you can mount with the nobarrier
>>> mount option.

Yeah, that is the current settings we use.

>> And open the file with O_TMPFILE|O_EXCL so there is no metadata as well.
> 
> O_TMPFILE means that there is no directory entry created.  The
> pathname passed to the open system call is the directory specifying
> the file system where the temporary file will be created.

Interesting.

The main problem is, that we can’t control what the users put into the
cluster, so a mount option is needed.

> This may or may not be what the original poster wanted, depending on
> whether by "scratch file" he meant a file which could be opened by
> pathname by another, subsequent process or not.

Yeah, the scientists send often scripts, where they access the files
by subsequent processes.


Kind regards,

Paul


Download attachment "smime.p7s" of type "application/pkcs7-signature" (5174 bytes)