linux-kernel - Re: [PATCH 1/2] f2fs: detect host-managed SMR by feature flag

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <CAJVOszCrULuMVwhcoaSnP=noa4SajON2UDyz3pVdG=jSPn0P_A@mail.gmail.com>
Date:   Wed, 24 Aug 2016 15:31:39 -0500
From:   Shaun Tancheff <shaun.tancheff@...gate.com>
To:     Jaegeuk Kim <jaegeuk@...nel.org>
Cc:     Damien Le Moal <Damien.LeMoal@...t.com>, schmorp@...morp.de,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
        "linux-f2fs-devel@...ts.sourceforge.net" 
        <linux-f2fs-devel@...ts.sourceforge.net>,
        Josh Bingaman <josh.bingaman@...gate.com>
Subject: Re: [PATCH 1/2] f2fs: detect host-managed SMR by feature flag

Hi Jaegeuk,

On Wed, Jun 15, 2016 at 11:32 AM, Jaegeuk Kim <jaegeuk@...nel.org> wrote:
> On Wed, Jun 15, 2016 at 04:21:13AM +0000, Damien Le Moal wrote:
>> On 6/15/16 03:45, Jaegeuk Kim wrote:
>> > If mkfs.f2fs gives a feature flag for host-managed SMR, we can set mode=lfs
>> > by default.
>> >
>> > Signed-off-by: Jaegeuk Kim <jaegeuk@...nel.org>
>> > ---
>> >  fs/f2fs/f2fs.h  | 21 +++++++++++++++++++++
>> >  fs/f2fs/super.c | 14 ++++++++------
>> >  2 files changed, 29 insertions(+), 6 deletions(-)
>> >
>> > diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
>> > index 32be19e..1a19c02 100644
>> > --- a/fs/f2fs/f2fs.h
>> > +++ b/fs/f2fs/f2fs.h
>> > @@ -133,6 +133,7 @@ struct f2fs_mount_info {
>> >  };
>> >
>> >  #define F2FS_FEATURE_ENCRYPT       0x0001
>> > +#define F2FS_FEATURE_HMSMR 0x0002
>> >
>> >  #define F2FS_HAS_FEATURE(sb, mask)                                 \
>> >     ((F2FS_SB(sb)->raw_super->feature & cpu_to_le32(mask)) != 0)
>> > @@ -2354,6 +2355,26 @@ static inline int f2fs_sb_has_crypto(struct super_block *sb)
>> >     return F2FS_HAS_FEATURE(sb, F2FS_FEATURE_ENCRYPT);
>> >  }
>> >
>> > +static inline int f2fs_sb_mounted_hmsmr(struct super_block *sb)
>> > +{
>> > +   return F2FS_HAS_FEATURE(sb, F2FS_FEATURE_HMSMR);
>> > +}
>> > +
>> > +static inline void set_opt_mode(struct f2fs_sb_info *sbi, unsigned int mt)
>> > +{
>> > +   clear_opt(sbi, ADAPTIVE);
>> > +   clear_opt(sbi, LFS);
>> > +
>> > +   switch (mt) {
>> > +   case F2FS_MOUNT_ADAPTIVE:
>> > +           set_opt(sbi, ADAPTIVE);
>> > +           break;
>> > +   case F2FS_MOUNT_LFS:
>> > +           set_opt(sbi, LFS);
>> > +           break;
>> > +   }
>> > +}
>> > +
>> >  static inline bool f2fs_may_encrypt(struct inode *inode)
>> >  {
>> >  #ifdef CONFIG_F2FS_FS_ENCRYPTION
>> > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
>> > index edc736d..71b6066 100644
>> > --- a/fs/f2fs/super.c
>> > +++ b/fs/f2fs/super.c
>> > @@ -515,12 +515,10 @@ static int parse_options(struct super_block *sb, char *options)
>> >                             return -ENOMEM;
>> >                     if (strlen(name) == 8 &&
>> >                                     !strncmp(name, "adaptive", 8)) {
>> > -                           set_opt(sbi, ADAPTIVE);
>> > -                           clear_opt(sbi, LFS);
>> > +                           set_opt_mode(sbi, F2FS_MOUNT_ADAPTIVE);
>> >                     } else if (strlen(name) == 3 &&
>> >                                     !strncmp(name, "lfs", 3)) {
>> > -                           clear_opt(sbi, ADAPTIVE);
>> > -                           set_opt(sbi, LFS);
>> > +                           set_opt_mode(sbi, F2FS_MOUNT_LFS);
>> >                     } else {
>> >                             kfree(name);
>> >                             return -EINVAL;
>> > @@ -980,7 +978,10 @@ static void default_options(struct f2fs_sb_info *sbi)
>> >     set_opt(sbi, EXTENT_CACHE);
>> >     sbi->sb->s_flags |= MS_LAZYTIME;
>> >     set_opt(sbi, FLUSH_MERGE);
>> > -   set_opt(sbi, ADAPTIVE);
>> > +   if (f2fs_sb_mounted_hmsmr(sbi->sb))
>> > +           set_opt_mode(sbi, F2FS_MOUNT_LFS);
>> > +   else
>> > +           set_opt_mode(sbi, F2FS_MOUNT_ADAPTIVE);
>>
>> Strictly speaking, host-aware drives would not require the LFS mode
>> as any zone of the disk can be randomly written and so the occasional
>> in-place updates of the adaptive mode would be OK. However, randomly
>> writing host-aware drives can lead to a lot of background activity on
>> the drive side for internal management of the accumulated random writes,
>> with a potential performance drop over time (this is highly dependent on
>> the drive FW implementation though).
>>
>> So I think it would be good to also enable the LFS mode by default for
>> host-aware devices, unless the user has explicitly specified mount with
>> adaptive mode (if the user "knows" that the drive FW handles very
>> efficiently random writes).
>
> Agreed. Would HASMR also provide the same geometry, zone information, in mkfs?
> Cause I haven't tested it so far.

Yes. Host Aware SMR reports geometry in the same way. As far as I can tell,
your current mkfs for f2fs works correctly for HASMR.

> IIRC, we discussed about HASMR with Marc Lehmann last year, and he gave the
> below link which sums up.
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__comments.gmane.org_gmane.linux.file-2Dsystems.f2fs_2854&d=CwIBAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=Wg5NqlNlVTT7Ugl8V50qIHLe856QW0qfG3WVYGOrWzA&m=3Uw29rZTMJsKnUE9bBduFOdp3OiEwJhpgbJ9GmyO_SU&s=YNxrXKMJpd-BPHnYG8ctqg6ct58p2yqO3BRKlbjBO_s&e=
> https://urldefense.proofpoint.com/v2/url?u=http-3A__blog.schmorp.de_2015-2D10-2D08-2Dsmr-2Darchive-2Ddrives-2Dfast-2Dnow.html&d=CwIBAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=Wg5NqlNlVTT7Ugl8V50qIHLe856QW0qfG3WVYGOrWzA&m=3Uw29rZTMJsKnUE9bBduFOdp3OiEwJhpgbJ9GmyO_SU&s=SC_RkdkfDQZTeIccq0o1zE4siI_RgDL4UhiVc1VzENY&e=

I think above applies to 'drive managed' SMR where the upper level
has no insight into the underlying zone layout.

>> So how about also introducing a F2FS_FEATURE_HASMR feature flag to
>> handle these different cases ?
>
> Yes, so once I can retrieve the zone information from the device, surely I'll
> add that.

I have posted some patches that will allow you to retrieve the zone
information for both HA and HM drives.

Can you see if this would meet your needs?

The data is returned to the BIO is in the same (drive) format as SG_IO.

>> Also, I think that the DISCARD option must be enabled by default for
>> HMSMR disks. Otherwise, zones write pointer will never get reset.
>> The same applies to HASMR devices mounted with the LFS mode.
>> (In any case, the discard handling does not look like it will always
>> align to the device zone size, which will fail on a zoned disk (discard
>> granularity is the zone size). I may be missing something though. Still
>> checking.)
>
> Yup, will do. BTW, I remember I fixed zone-aligned discard issue before.
> Could you please check the latest dev-test branch in f2fs.git?
> I'm used to rebase that branch occasionally when I changed the previous patches.

I think you will need to have the have access to the zone information.

Regards,
Shaun Tancheff