linux-ext4 - Re: [RFCv1 18/72] libext2fs: Add support to get average group count

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <83D3872A-E269-477D-8D2E-CA2AF80D658A@dilger.ca>
Date:   Wed, 14 Dec 2022 14:24:50 -0700
From:   Andreas Dilger <adilger@...ger.ca>
To:     "Ritesh Harjani (IBM)" <ritesh.list@...il.com>
Cc:     Theodore Ts'o <tytso@....edu>, linux-ext4@...r.kernel.org,
        Harshad Shirwadkar <harshadshirwadkar@...il.com>,
        Wang Shilong <wangshilong1991@...il.com>, Li Xi <lixi@....com>
Subject: Re: [RFCv1 18/72] libext2fs: Add support to get average group count

On Nov 7, 2022, at 5:21 AM, Ritesh Harjani (IBM) <ritesh.list@...il.com> wrote:
> 
> From: Wang Shilong <wshilong@....com>
> 
> number of threads in pfsck should not exceed flex bg numbers.
> This patch adds the support in libext2fs to calculate
> ext2fs_get_avg_group() which returns an average group
> count which each thread has to scan.
> 
> fs->fs_num_threads will be set by the client, in this case e2fsck.
> No. of threads will be passed along with -m option while running e2fsck.
> That will also set fs->fs_num_threads, which will help in controlling
> the amount of memory consumed to maintain in memory data structures (per
> thread) in case of multiple parallel threads (pfsck) to avoid oom.
> 
> Signed-off-by: Wang Shilong <wshilong@....com>
> Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@...il.com>
> ---
> lib/ext2fs/ext2fs.h | 32 +++++++++++++++++++++++++++++++-
> 1 file changed, 31 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> index b1505f95..6b4926ce 100644
> --- a/lib/ext2fs/ext2fs.h
> +++ b/lib/ext2fs/ext2fs.h
> @@ -279,10 +279,11 @@ struct struct_ext2_filsys {
> 	int				cluster_ratio_bits;
> 	__u16				default_bitmap_type;
> 	__u16				pad;
> +	__u32				fs_num_threads;

From later cleanup patch, fs_num_threads should just use "__u16 pad" field.

Otherwise, looks OK.


> 	/*
> 	 * Reserved for future expansion
> 	 */
> -	__u32				reserved[5];
> +	__u32				reserved[4];
> 
> 	/*
> 	 * Reserved for the use of the calling application.
> @@ -2231,6 +2232,35 @@ ext2fs_orphan_block_tail(ext2_filsys fs, char *buf)
> 		sizeof(struct ext4_orphan_block_tail));
> }
> 
> +static dgrp_t ext2fs_get_avg_group(ext2_filsys fs)
> +{
> +#ifdef HAVE_PTHREAD
> +	dgrp_t average_group;
> +	unsigned flexbg_size;
> +
> +	if (fs->fs_num_threads <= 1)
> +		return fs->group_desc_count;
> +
> +	average_group = fs->group_desc_count / fs->fs_num_threads;
> +	if (average_group <= 1)
> +		return 1;
> +
> +	if (ext2fs_has_feature_flex_bg(fs->super)) {
> +		int times = 1;
> +
> +		flexbg_size = 1 << fs->super->s_log_groups_per_flex;
> +		if (average_group % flexbg_size) {
> +			times = average_group / flexbg_size;
> +			average_group = times * flexbg_size;
> +		}
> +	}
> +
> +	return average_group;
> +#else
> +	return fs->group_desc_count;
> +#endif
> +}
> +
> #undef _INLINE_
> #endif
> 
> --
> 2.37.3
> 


Cheers, Andreas






Download attachment "signature.asc" of type "application/pgp-signature" (874 bytes)