linux-ext4 - Re: New service e2scrub

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190318214717.GB3859@mit.edu>
Date:   Mon, 18 Mar 2019 17:47:17 -0400
From:   "Theodore Ts'o" <tytso@....edu>
To:     Paul Menzel <pmenzel@...gen.mpg.de>
Cc:     "Darrick J. Wong" <darrick.wong@...cle.com>,
        linux-ext4@...r.kernel.org
Subject: Re: New service e2scrub_reap

On Mon, Mar 18, 2019 at 12:24:55PM +0100, Paul Menzel wrote:
> Dear Ted, dear Darrick,
> 
> On Debian Sid/unstable, I noticed the new service `scrub/e2scrub_reap.service`
> installed in the default target [1][2].
> 
> $ systemctl status -o short-precise e2scrub_reap.service
> ● e2scrub_reap.service - Remove Stale Online ext4 Metadata Check Snapshots
>    Loaded: loaded (/lib/systemd/system/e2scrub_reap.service; enabled; vendor preset: enabled)
>    Active: inactive (dead) since Mon 2019-03-18 12:17:13 CET; 1min 1s ago
>      Docs: man:e2scrub_all(8)
>   Process: 447 ExecStart=/sbin/e2scrub_all -A -r (code=exited, status=0/SUCCESS)
>  Main PID: 447 (code=exited, status=0/SUCCESS)
> 
> Mar 18 12:17:08.223560 plumpsklo systemd[1]: Starting Remove Stale Online ext4 Metadata Check Snapshots...
> Mar 18 12:17:13.996465 plumpsklo systemd[1]: e2scrub_reap.service: Succeeded.
> Mar 18 12:17:13.996808 plumpsklo systemd[1]: Started Remove Stale Online ext4 Metadata Check Snapshots.

Yeah, that's unfortunate.  I'm seeing a similar time on my (fairly
high-end) laptop:

# time e2scrub_all -A -r 

real	0m4.356s
user	0m0.677s
sys	0m1.285s


We should be able to fix this in general by avoiding the use of lsblk
at all, and in the case of e2scrub -r, just simply iterating over the
output of:

lvs --name-prefixes -o vg_name,lv_name,lv_path,origin -S lv_role=snapshot

(which takes about a fifth of a second on my laptop and it should be
even faster if there are no LVM volumes on the system)

And without the -r option, we should just be able to do this:

lvs --name-prefixes -o vg_name,lv_name,lv_path -S lv_active=active,lv_role=public

Right now we're calling lvs for every single block device emitted by
lsblk, and from what I can tell, we can do a much better job
optimizing e2scrub_all.

> Reading the manual, the switch `-r` “removes e2scrub snapshots but do not
> check anything”.
> 
> Does this have to be done during boot-up, or could it be done after the
> default target was reached, or even during shutting down?

This shouldn't be blocking any other targets, I think there should be
a way to configure the unit file so that it runs in parallel with the
other systemd units.  My systemd-fu is not super strong, so I'll have
to do some investigating to see how we can fix this.

Regards,

					- Ted