linux-kernel - Re: File system for scratch space (in HPC cluster)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <c09eccc9-c067-c121-e9ae-8e3f32d8c80b@molgen.mpg.de>
Date:   Fri, 25 Oct 2019 10:35:34 +0200
From:   Paul Menzel <pmenzel@...gen.mpg.de>
To:     Andreas Dilger <adilger@...ger.ca>
Cc:     Linux FS-devel Mailing List <linux-fsdevel@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Donald Buczek <buczek@...gen.mpg.de>
Subject: Re: File system for scratch space (in HPC cluster)

Dear Andreas,


On 2019-10-24 19:51, Andreas Dilger wrote:
> On Oct 24, 2019, at 4:43 AM, Paul Menzel <pmenzel@...gen.mpg.de> 
> wrote:

>> In our cluster, we offer scratch space for temporary files. As 
>> these files are temporary, we do not need any safety requirements
>> – especially not those when the system crashes or shuts down. So
>> no `sync` is for example needed.
>> 
>> Are there file systems catering to this need? I couldn’t find any? 
>> Maybe I missed some options for existing file systems.
> 
> How big do you need the scratch filesystem to be?  Is it local to
> the node or does it need to be shared between nodes?

In this case local.

> If it needs to be large and shared between nodes then Lustre is 
> typically used for this.  If it is local and relatively small you 
> could consider using tmpfs backed by swab on an NVMe flash device 
> (M.2 or U.2, Optane if you can afford it) inside the node.
> 
> That way you get RAM-like performance for many files, with a larger 
> capacity than RAM when needed (tmpfs can use swap).
> 
> You might consider to mount a new tmpfs filesystem per job (no 
> formatting is needed for tmpfs), and then unmount it when the job is 
> done, so that the old files are automatically cleaned up.
That is a good idea, but probably not practical for 10 TB. Out of
curiosity, what is the limit for “relatively small” in your
experience?


Kind regards,

Paul


Download attachment "smime.p7s" of type "application/pkcs7-signature" (5174 bytes)