linux-kernel - Re: [f2fs-dev] [PATCH 3/5] f2fs: control the memory footprint used by ino entries

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 9 Nov 2014 21:20:09 -0800
From:	Jaegeuk Kim <jaegeuk@...nel.org>
To:	Changman Lee <cm224.lee@...sung.com>
Cc:	linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org,
	linux-f2fs-devel@...ts.sourceforge.net
Subject: Re: [f2fs-dev] [PATCH 3/5] f2fs: control the memory footprint used
 by ino entries

On Mon, Nov 10, 2014 at 12:28:34PM +0900, Changman Lee wrote:
> On Sat, Nov 08, 2014 at 11:36:07PM -0800, Jaegeuk Kim wrote:
> > This patch adds to control the memory footprint used by ino entries.
> > This will conduct best effort, not strictly.
> > 
> > Signed-off-by: Jaegeuk Kim <jaegeuk@...nel.org>
> > ---
> >  fs/f2fs/node.c    | 28 ++++++++++++++++++++++------
> >  fs/f2fs/node.h    |  3 ++-
> >  fs/f2fs/segment.c |  3 ++-
> >  3 files changed, 26 insertions(+), 8 deletions(-)
> > 
> > diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
> > index 44b8afe..4ea2c47 100644
> > --- a/fs/f2fs/node.c
> > +++ b/fs/f2fs/node.c
> > @@ -31,22 +31,38 @@ bool available_free_memory(struct f2fs_sb_info *sbi, int type)
> >  {
> >  	struct f2fs_nm_info *nm_i = NM_I(sbi);
> >  	struct sysinfo val;
> > +	unsigned long avail_ram;
> >  	unsigned long mem_size = 0;
> >  	bool res = false;
> >  
> >  	si_meminfo(&val);
> > -	/* give 25%, 25%, 50% memory for each components respectively */
> > +
> > +	/* only uses low memory */
> > +	avail_ram = val.totalram - val.totalhigh;
> > +
> > +	/* give 25%, 25%, 50%, 50% memory for each components respectively */
> 
> Hi Jaegeuk,
> 
> The memory usage of nm_i should be 100% but it's 125%.
> Mistake or intended?

I contemplated whether this 100% was an exact number that we expected.
The answer was NO, since this number was just an estimated one.
There were no strict constrains to limit memory footprints even the previous
codes were used whatever 25%, 25%, and 50%.

So, here, I'd like to add additional threshold for INO_ENTRIES on a basis
of the given threshold.
In addition, I don't want to add any complex equations to satisfy 100% at all.
It's meaningless.

Thanks,

> 
> >  	if (type == FREE_NIDS) {
> > -		mem_size = (nm_i->fcnt * sizeof(struct free_nid)) >> 12;
> > -		res = mem_size < ((val.totalram * nm_i->ram_thresh / 100) >> 2);
> > +		mem_size = (nm_i->fcnt * sizeof(struct free_nid)) >>
> > +							PAGE_CACHE_SHIFT;
> > +		res = mem_size < ((avail_ram * nm_i->ram_thresh / 100) >> 2);
> >  	} else if (type == NAT_ENTRIES) {
> > -		mem_size = (nm_i->nat_cnt * sizeof(struct nat_entry)) >> 12;
> > -		res = mem_size < ((val.totalram * nm_i->ram_thresh / 100) >> 2);
> > +		mem_size = (nm_i->nat_cnt * sizeof(struct nat_entry)) >>
> > +							PAGE_CACHE_SHIFT;
> > +		res = mem_size < ((avail_ram * nm_i->ram_thresh / 100) >> 2);
> >  	} else if (type == DIRTY_DENTS) {
> >  		if (sbi->sb->s_bdi->dirty_exceeded)
> >  			return false;
> >  		mem_size = get_pages(sbi, F2FS_DIRTY_DENTS);
> > -		res = mem_size < ((val.totalram * nm_i->ram_thresh / 100) >> 1);
> > +		res = mem_size < ((avail_ram * nm_i->ram_thresh / 100) >> 1);
> > +	} else if (type == INO_ENTRIES) {
> > +		int i;
> > +
> > +		if (sbi->sb->s_bdi->dirty_exceeded)
> > +			return false;
> > +		for (i = 0; i <= UPDATE_INO; i++)
> > +			mem_size += (sbi->ino_num[i] * sizeof(struct ino_entry))
> > +							>> PAGE_CACHE_SHIFT;
> > +		res = mem_size < ((avail_ram * nm_i->ram_thresh / 100) >> 1);
> >  	}
> >  	return res;
> >  }
> > diff --git a/fs/f2fs/node.h b/fs/f2fs/node.h
> > index acb71e5..d10b644 100644
> > --- a/fs/f2fs/node.h
> > +++ b/fs/f2fs/node.h
> > @@ -106,7 +106,8 @@ static inline void raw_nat_from_node_info(struct f2fs_nat_entry *raw_ne,
> >  enum mem_type {
> >  	FREE_NIDS,	/* indicates the free nid list */
> >  	NAT_ENTRIES,	/* indicates the cached nat entry */
> > -	DIRTY_DENTS	/* indicates dirty dentry pages */
> > +	DIRTY_DENTS,	/* indicates dirty dentry pages */
> > +	INO_ENTRIES,	/* indicates inode entries */
> >  };
> >  
> >  struct nat_entry_set {
> > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
> > index 16721b5d..e094675 100644
> > --- a/fs/f2fs/segment.c
> > +++ b/fs/f2fs/segment.c
> > @@ -276,7 +276,8 @@ void f2fs_balance_fs_bg(struct f2fs_sb_info *sbi)
> >  {
> >  	/* check the # of cached NAT entries and prefree segments */
> >  	if (try_to_free_nats(sbi, NAT_ENTRY_PER_BLOCK) ||
> > -				excess_prefree_segs(sbi))
> > +			excess_prefree_segs(sbi) ||
> > +			available_free_memory(sbi, INO_ENTRIES))
> >  		f2fs_sync_fs(sbi->sb, true);
> >  }
> >  
> > -- 
> > 2.1.1
> > 
> > 
> > ------------------------------------------------------------------------------
> > _______________________________________________
> > Linux-f2fs-devel mailing list
> > Linux-f2fs-devel@...ts.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/