Message-ID: <87ecqcpujw.fsf@wotan.olymp>
Date: Wed, 05 Nov 2025 11:50:43 +0000
From: Luis Henriques <luis@...lia.com>
To: Amir Goldstein <amir73il@...il.com>
Cc: "Darrick J. Wong" <djwong@...nel.org>,  Bernd Schubert
 <bernd@...ernd.com>,  "Theodore Ts'o" <tytso@....edu>,  Miklos Szeredi
 <miklos@...redi.hu>,  Bernd Schubert <bschubert@....com>,
  linux-fsdevel@...r.kernel.org,  linux-kernel@...r.kernel.org,  Kevin Chen
 <kchen@....com>,
    Matt Harvey <mharvey@...ptrading.com>
Subject: Re: [RFC] Another take at restarting FUSE servers

Hi Amir,

On Wed, Nov 05 2025, Amir Goldstein wrote:

> On Tue, Nov 4, 2025 at 3:52 PM Luis Henriques <luis@...lia.com> wrote:

<...>

>> > fuse_entry_out was extended once and fuse_reply_entry()
>> > sends the size of the struct.
>>
>> So, if I'm understanding you correctly, you're suggesting to extend
>> fuse_entry_out to add the new handle (a 'size' field + the actual handle).
>
> Well it depends...
>
> There are several ways to do it.
> I would really like to get Miklos and Bernd's opinion on the preferred way.

Sure, all feedback is welcome!

> So far, it looks like the client determines the size of the output args.
>
> If we want the server to be able to write a different file handle size
> per inode that's going to be a bigger challenge.
>
> I think it's plenty enough if server and client negotiate a max file handle
> size and then the client always reserves enough space in the output
> args buffer.
>
> One more thing to ask is what is "the actual handle".
> If "the actual handle" is the variable sized struct file_handle then
> the size is already available in the file handle header.

Actually, this is exactly what I was trying to mimic for my initial
attempt.  However, I was not going to do any size negotiation but instead
define a maximum size for the handle.  See below.

> If it is not, then I think some sort of type or version of the file handles
> encoding should be negotiated beyond the max handle size.

In my initial stab at this I was going to take a very simple approach and
hard-code a maximum size for the handle.  This would have the advantage of
allowing the server to use different sizes for different inodes (though
I'm not sure how useful that would be in practice).  So, in summary, I
would define the new handle like this:

/* Same value as MAX_HANDLE_SZ */
#define FUSE_MAX_HANDLE_SZ 128

struct fuse_file_handle {
	uint32_t	size;
	uint32_t	padding;
	char		handle[FUSE_MAX_HANDLE_SZ];
};

and this struct would be included in fuse_entry_out.

There's probably a problem with having this (big) fixed size increase to
fuse_entry_out, but maybe that could be fixed once I have all the other
details sorted out.  Hopefully I'm not oversimplifying the problem,
skipping the need for negotiating a handle size.

>> That's probably a good idea.  I was working towards having the
>> LOOKUP_HANDLE to be similar to LOOKUP, but extending it so that it would
>> include:
>>
>>  - An extra inarg: the parent directory handle.  (To be honest, I'm not
>>    really sure this would be needed.)
>
> Yes, I think you need extra inarg.
> Why would it not be needed?
> The problem is that you cannot know if the parent node id in the lookup
> command is stale after server restart.

Ah, of course.  Hence the need for this extra inarg.

> The thing is that the kernel fuse inode will need to store the file handle,
> much the same as an NFS client stores the file handle provided by the
> NFS server.
>
> FYI, fanotify has an optimized way to store file handles in
> struct fanotify_fid_event - small file handles are stored inline
> and larger file handles can use an external buffer.
>
> But fuse does not need to support any size of file handles.
> For first version we could definitely simplify things by limiting the size
> of supported file handles, because server and client need to negotiate
> the max file handle size anyway.

I'll definitely need to have a look at how fanotify does that.  But I
guess that if my simplistic approach with a static array is acceptable for
now, I'll stick with it for the initial attempt to implement this, and
eventually revisit it later to do something more clever.

>>  - An extra outarg: for the actual handle.
>>
>> With your suggestion, only the extra inarg would be required.
>>
>
> Yes, either extra arg or just an extended size of fuse_entry_out
> negotiated at init time.
>
> TBH it seems cleaner to add 2nd outarg to all the commands,
> but CREATE already has a 2nd arg and 2nd arg does not solve
> READDIRPLUS.

Right.  I'm more and more convinced that extending fuse_entry_out is the
way to go.

>> > However fuse_reply_create() sends it with fuse_open_out
>> > appended
>>
>> This one should be fine...
>>
>> > and fuse_add_direntry_plus() does not seem to write
>> > record size at all, so server and client will need to agree on the
>> > size of fuse_entry_out and this would need to be backward compat.
>> > If both server and client declare support for FUSE_LOOKUP_HANDLE
>> > it should be fine (?).
>>
>> ... yeah, this could be a bit trickier.  But I'll need to go look into it.
>>
>> Thanks a lot for your comments, Amir.  I was trying to get an RFC out
>> soon(ish) to get early feedback, hoping to prevent me following wrong
>> paths.
>>
>
> Disclaimer, following my advice may well lead you down wrong paths..
> Best to wait for confirmation from Miklos and Bernd if you want to have
> more certainty...

Haha thanks for the warning :-)

And again, thanks a lot for your feedback, Amir.

Cheers,
-- 
Luís

Powered by blists - more mailing lists