linux-ext4 - Re: Transparent compression with ext4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <b8663f69-cdaf-4c05-b99f-cd4105023264@wiesinger.com>
Date: Wed, 22 Jan 2025 08:29:09 +0100
From: Gerhard Wiesinger <lists@...singer.com>
To: "Theodore Ts'o" <tytso@....edu>
Cc: linux-ext4@...r.kernel.org
Subject: Re: Transparent compression with ext4 - especially with zstd

On 21.01.2025 20:33, Theodore Ts'o wrote:
> On Tue, Jan 21, 2025 at 07:47:24PM +0100, Gerhard Wiesinger wrote:
>> We are talking in some scenarios about some factors of diskspace. E.g. in
>> my database scenario with PostgreSQL around 85% of disk space can be saved
>> (e.g. around factor 7).
> Worse, using a transparent compression breaks the ACID properties of
> the database.  If you crash or have a power failure while rewriting
> the 64k compression cluster, all or part of that 64k compression
> cluster can be corrupted.  And if your customers care about (their)
> data integrity, the fact that you cheaped out on disk space might not
> be something that would impress them terribly.
>
BTW: Why does it break the ACID properties?

Typically the transaction log will be (and have to be) flushed/synced to 
disk (fsync). If that's ok everything is fine and all DB transactions 
can be forwared if necessary. If that fails the last transaction is not 
recorded.

I also don't see any compression related. That can also happen without 
compression.

Any clarification?

Ciao,

Gerhard