Message-ID: <ee159a27-9006-4ec7-a6e9-9d9d1bdf3872@huaweicloud.com>
Date: Wed, 28 May 2025 17:58:34 +0800
From: Zizhi Wo <wozizhi@...weicloud.com>
To: Gao Xiang <hsiangkao@...ux.alibaba.com>,
 Zizhi Wo <wozizhi@...weicloud.com>, netfs@...ts.linux.dev,
 dhowells@...hat.com, jlayton@...nel.org, brauner@...nel.org
Cc: jefflexu@...ux.alibaba.com, zhujia.zj@...edance.com,
 linux-erofs@...ts.ozlabs.org, linux-fsdevel@...r.kernel.org,
 linux-kernel@...r.kernel.org, libaokun1@...wei.com, yangerkun@...wei.com,
 houtao1@...wei.com, yukuai3@...wei.com
Subject: Re: [QUESTION] cachefiles: Recovery concerns with on-demand loading
 after unexpected power loss



在 2025/5/28 17:25, Gao Xiang 写道:
> 
> 
> On 2025/5/28 16:53, Zizhi Wo wrote:
>>
>>
>> 在 2025/5/28 16:35, Gao Xiang 写道:
>>> Hi Zizhi,
>>>
>>> On 2025/5/28 16:07, Zizhi Wo wrote:
>>>> Currently, in on-demand loading mode, cachefiles first calls
>>>> cachefiles_create_tmpfile() to generate a tmpfile, and only during 
>>>> the exit
>>>> process does it call 
>>>> cachefiles_commit_object->cachefiles_commit_tmpfile to
>>>> create the actual dentry and making it visible to users.
>>>>
>>>> If the cache write is interrupted unexpectedly (e.g., by system 
>>>> crash or
>>>> power loss), during the next startup process, 
>>>> cachefiles_look_up_object()
>>>> will determine that no corresponding dentry has been generated and will
>>>> recreate the tmpfile and pull the complete data again!
>>>>
>>>> The current implementation mechanism appears to provide per-file 
>>>> atomicity.
>>>> For scenarios involving large image files (where significant amount of
>>>> cache data needs to be written), this re-pulling process after an
>>>> interruption seems considerable overhead?
>>>>
>>>> In previous kernel versions, cache dentry were generated during the
>>>> LOOK_UP_OBJECT process of the object state machine. Even if power 
>>>> was lost
>>>> midway, the next startup process could continue pulling data based 
>>>> on the
>>>> previously downloaded cache data on disk.
>>>>
>>>> What would be the recommended way to handle this situation? Or am I
>>>> thinking about this incorrectly? Would appreciate any feedback and 
>>>> guidance
>>>> from the community.
>>>
>>> As you can see, EROFS fscache feature was marked as deprecated
>>> since per-content hooks already support the same use case.
>>>
>>> the EROFS fscache support will be removed after I make
>>> per-content hooks work in erofs-utils, which needs some time
>>> because currently I don't have enough time to work on the
>>> community stuff.
>>>
>>> Thanks,
>>> Gao Xiang
>>
>> Thanks for your reply.
>>
>> Indeed, the subsequent implementations have moved to using fanotify.
>> Moreover, based on evaluation, this approach could indeed lead to
>> performance improvements.
>>
>> However, in our current use case, we are still working with a kernel
>> version that only supports the fscache-based approach, so this issue
>> still exists for us. :(
> 
> Since it's deprecated (because that fscache improvement will
> take much long time to upstream and netfs dependency is
> redundant in addition to new pre-content hooks), could you
> improve it downstream directly?
> 
> Or if you have some simple proposal you could post, but no one
> avoids you to use fscache downstream but it seems pre-content
> hooks are more cleaner for this use case..
> 
> Thanks,
> Gao Xiang
> 

I understand. We'll have some internal discussions to explore possible
solutions. Our initial intention in bringing this up was also to hear if
the community has any suggestions or different perspectives on this
issue, which could help guide our future improvements (although fanotify 
is for future and fscache seems deprecated). For example, as you
mentioned fanotify — that was quite insightful.

Thanks,
Zizhi Wo

> 
>>
>> Thanks,
>> Zizhi Wo
>>
>>>
>>>>
>>>> Thanks,
>>>> Zizhi Wo
>>>
> 
> 

Powered by blists - more mailing lists