[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <116d12db-5c24-9a57-79a9-b7a02a0bb4c2@redhat.com>
Date: Fri, 18 Nov 2022 13:24:10 +0800
From: Xiubo Li <xiubli@...hat.com>
To: Luís Henriques <lhenriques@...e.de>
Cc: Ilya Dryomov <idryomov@...il.com>,
Jeff Layton <jlayton@...nel.org>, ceph-devel@...r.kernel.org,
linux-kernel@...r.kernel.org
Subject: Re: [PATCH] ceph: make sure directories aren't complete after setting
crypt context
On 18/11/2022 02:02, Luís Henriques wrote:
> Xiubo Li <xiubli@...hat.com> writes:
>
>> On 17/11/2022 16:03, Xiubo Li wrote:
>>> On 16/11/2022 23:37, Luís Henriques wrote:
>>>> When setting a directory's crypt context, __ceph_dir_clear_complete() needs
>>>> to be used otherwise, if it was complete before, any old dentry that's still
>>>> around will be valid.
>>>>
>>>> Signed-off-by: Luís Henriques <lhenriques@...e.de>
>>>> ---
>>>> Hi!
>>>>
>>>> Here's a simple way to trigger the bug this patch is fixing:
>>>>
>>>> # cd /cephfs
>>>> # ls mydir
>>>> nKRhofOAVNsAwVLvDw7a0c9ypsjbZfK3n0Npnmni6j0
>>>> # ls mydir/nKRhofOAVNsAwVLvDw7a0c9ypsjbZfK3n0Npnmni6j0/
>>>> Cyuer5xT+kBlEPgtwAqSj0WK2taEljP5vHZ,D8VXCJ8
>>>> u+46b2XVCt7Obpz0gznZyNLRj79Q2l4KmkwbKOzdQKw
>>>> # fscrypt unlock mydir
>>>> # touch /mnt/test/mydir/mysubdir/file
>>>> touch: cannot touch '/mnt/test/mydir/mysubdir/file': No such file or
>>>> directory
>>>>
>>>> fs/ceph/crypto.c | 4 ++++
>>>> 1 file changed, 4 insertions(+)
>>>>
>>>> diff --git a/fs/ceph/crypto.c b/fs/ceph/crypto.c
>>>> index 35a2ccfe6899..dc1557967032 100644
>>>> --- a/fs/ceph/crypto.c
>>>> +++ b/fs/ceph/crypto.c
>>>> @@ -87,6 +87,10 @@ static int ceph_crypt_get_context(struct inode *inode,
>>>> void *ctx, size_t len)
>>>> return -ERANGE;
>>>> memcpy(ctx, cfa->cfa_blob, ctxlen);
>>>> +
>>>> + /* Directory isn't complete anymore */
>>>> + if (S_ISDIR(inode->i_mode) && __ceph_dir_is_complete(ci))
>>>> + __ceph_dir_clear_complete(ci);
>>> Hi Luis,
>>>
>>> Good catch!
>>>
>>> BTW, why do this in the ceph_crypt_get_context() ? As my understanding is that
>>> we should mark 'mydir' as incomplete when unlocking it. While as I remembered
>>> the unlock operation will do:
>>>
>>>
>>> Step1: get_encpolicy via 'mydir' as ctx
>>> Step2: rm_enckey of ctx from the superblock
>>>
>> Sorry, it should be add_enckey.
>>> Since I am still running the test cases for the file lock patches, so I didn't
>>> catch logs to confirm the above steps yet.
>>>
>>> If I am right IMO then we should mark the dir as incomplete in the Step2
>>> instead, because for non-unlock operations they may also do the Step1.
>>>
>> Your patch will work. But probably we should do this just around
>> __fscrypt_prepare_readdir() or fscrypt_prepare_readdir() instead ? We need to
>> detect that once the 'inode->i_crypt_info' changed then mark the dir as
>> incomplete.
>>
>> For now for the lock operation it will evict the inode, which will help do this
>> for us already. But for unlock case, we need to handle it by ourself.
> OK, that makes sense and to be honest I thought that there should be
> another place for doing this. Unfortunately, I didn't found it: in the
> test case I have the fscrypt_prepare_readdir() isn't called:
>
> # cd /cephfs
> # ls mydir
> nKRhofOAVNsAwVLvDw7a0c9ypsjbZfK3n0Npnmni6j0
> # ls mydir/nKRhofOAVNsAwVLvDw7a0c9ypsjbZfK3n0Npnmni6j0/
> Cyuer5xT+kBlEPgtwAqSj0WK2taEljP5vHZ,D8VXCJ8 u+46b2XVCt7Obpz0gznZyNLRj79Q2l4KmkwbKOzdQKw
>
> At this point readdir was executed, of course. And
> __ceph_dir_set_complete() is also used to indicate that we have the full
> contents. However, executing the following commands won't result in any
> new readdir():
>
> # fscrypt unlock mydir
> # touch /mnt/test/mydir/mysubdir/file
>
> and since the encryption key is set at the sb level, I couldn't find a way
> to detect changes in inode->i_crypt_info. ceph_d_revalidate() is invoked
> but at that point I don't thing we have a way to know what is changing.
>
> Any ideas?
# ls mydir/
zy94Zt01M90xwYq+nxJsEvea+HYq49mqVgrUBkYrJAU
# ls mydir/zy94Zt01M90xwYq+nxJsEvea+HYq49mqVgrUBkYrJAU/
24iNa8ICYc6nZZIkL,4n1sOtp9KiaTGtpq8PCZDQ6LU
ivG4rQYzg6YENagbTZaV2kh2sVouEdlkmBZPfgVDl48
I have added the debug logs in all the places just like:
@@ -784,7 +786,9 @@ static struct dentry *ceph_lookup(struct inode *dir,
struct dentry *dentry,
return ERR_PTR(-ENAMETOOLONG);
if (IS_ENCRYPTED(dir)) {
+ printk("%s fscrypt_has_encryption_key(dir): %d ======\n",
__func__, fscrypt_has_encryption_key(dir));
err = __fscrypt_prepare_readdir(dir);
+ printk("%s fscrypt_has_encryption_key(dir): %d ======\n",
__func__, fscrypt_has_encryption_key(dir));
if (err)
return ERR_PTR(err);
if (!fscrypt_has_encryption_key(dir)) {
# fscrypt unlock mydir
Enter custom passphrase for protector "l":
"mydir" is now unlocked and ready for use.
And after "mydir/" was unlocked and then when doing:
# touch mydir/dir/file2
And when lookup the "dir/" dentry I can see the "mydir/" encryption key
changed:
709 <7>[79125.023676] ceph: __ceph_caps_issued_mask ino 0x10000000004
cap 000000004dc11892 issued pAsLsXsFs (mask As)
710 <7>[79125.023687] ceph: __touch_cap 000000007071b095 cap
000000004dc11892 mds0
711 <7>[79125.023823] ceph: lookup 000000007071b095 dentry
00000000f97501b2 'dir'
712 <4>[79125.023838] ceph_lookup fscrypt_has_encryption_key(dir): 0 ======
713 <4>[79125.024186] ceph_lookup fscrypt_has_encryption_key(dir): 1 ======
714 <7>[79125.024194] ceph: dir 000000007071b095 flags are 0x0
715 <7>[79125.024269] ceph: do_request on 00000000a93fafef
I am thinking could we just make __fscrypt_prepare_readdir(), which will
return 0 when the key is already set or successfully set, to return 1
instead of 0 to mark that the key changed ?
Thanks!
- Xiubo
>
> Cheers,
Powered by blists - more mailing lists