lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <a05dfccc-33ff-4857-b68d-ddd64cae11d0@linux.dev>
Date:   Tue, 24 May 2022 08:52:30 +0800
From:   Jackie Liu <liu.yun@...ux.dev>
To:     Dave Chinner <david@...morbit.com>,
        liuzhengyuan <liuzhengyuan@...inos.cn>,
        胡海 <huhai@...inos.cn>, zhangshida@...inos.cn
Cc:     darrick.wong@...cle.com, linux-xfs@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Re: [BUG report] security_inode_alloc return -ENOMEM let xfs shutdown



在 2022/5/24 上午7:20, Dave Chinner 写道:
> On Mon, May 23, 2022 at 04:51:50PM +0800, Jackie Liu wrote:
>> Hello Maintainer and developer.
>>
>>     Syzkaller report an filesystem shutdown for me, It's very easy to
>> trigger and also exists on the latest kernel version 5.18-rc7.
> 
> Shutdown is a perfectly reasonable way to handle a failure that we
> can't recover cleanly from.
> 
>> dmesg shows:
>>
>> [  285.725893] FAULT_INJECTION: forcing a failure.
>>                 name failslab, interval 1, probability 0, space 0, times 0
>> [  285.729625] CPU: 7 PID: 18034 Comm: syz-executor Not tainted 4.19.90-43+
>> #7
>> [  285.731420] Source Version: b62cabdd86181d386998660ebf34ca653addd6c9
>> [  285.733051] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0
>> 02/06/2015
>> [  285.734796] Call trace:
>> [  285.735614]  dump_backtrace+0x0/0x3e0
>> [  285.736609]  show_stack+0x2c/0x38
>> [  285.737525]  dump_stack+0x164/0x1fc
>> [  285.738489]  should_fail+0x5c0/0x688
>> [  285.739555]  __should_failslab+0x118/0x180
>> [  285.740725]  should_failslab+0x2c/0x78
>> [  285.741808]  kmem_cache_alloc_trace+0x270/0x410
>> [  285.743120]  security_inode_alloc+0x100/0x1a8
>> [  285.744356]  inode_init_always+0x48c/0xa28
>> [  285.745524]  xfs_iget_cache_hit+0x9c0/0x2f28
>> [  285.746739]  xfs_iget+0x33c/0x9e0
>> [  285.747708]  xfs_ialloc+0x218/0x11c0
>> [  285.748752]  xfs_dir_ialloc+0xe8/0x480
>> [  285.749832]  xfs_create+0x5bc/0x1220
>> [  285.750871]  xfs_generic_create+0x42c/0x568
>> [  285.752053]  xfs_vn_mknod+0x48/0x58
>> [  285.753067]  xfs_vn_create+0x40/0x50
>> [  285.754106]  lookup_open+0x960/0x1580
>> [  285.755176]  do_last+0xd44/0x2180
>> [  285.756149]  path_openat+0x1a0/0x6d0
>> [  285.757187]  do_filp_open+0x14c/0x208
>> [  285.758245]  do_sys_open+0x340/0x470
>> [  285.759289]  __arm64_sys_openat+0x98/0xd8
>> [  285.760438]  el0_svc_common+0x230/0x3f0
>> [  285.761541]  el0_svc_handler+0x144/0x1a8
>> [  285.762674]  el0_svc+0x8/0x1b0
>> [  285.763737] security_inode_alloc:796
>> [  285.764733] inode_init_always:202
>> [  285.765669] xfs_create:1213
>> [  285.766485] XFS (dm-0): Internal error xfs_trans_cancel at line 1046 of
>> file fs/xfs/xfs_trans.c.  Caller xfs_create+0x700/0x1220
>> [  285.769503] CPU: 7 PID: 18034 Comm: syz-executor Not tainted 4.19.90-43+
>> #7
>> [  285.771275] Source Version: b62cabdd86181d386998660ebf34ca653addd6c9
>> [  285.772892] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0
>> 02/06/2015
>> [  285.774625] Call trace:
>> [  285.775335]  dump_backtrace+0x0/0x3e0
>> [  285.776324]  show_stack+0x2c/0x38
>> [  285.777236]  dump_stack+0x164/0x1fc
>> [  285.778188]  xfs_error_report+0xdc/0xe0
>> [  285.779292]  xfs_trans_cancel+0x490/0x878
>> [  285.780439]  xfs_create+0x700/0x1220
>> [  285.781477]  xfs_generic_create+0x42c/0x568
>> [  285.782673]  xfs_vn_mknod+0x48/0x58
>> [  285.783687]  xfs_vn_create+0x40/0x50
>> [  285.784724]  lookup_open+0x960/0x1580
>> [  285.785782]  do_last+0xd44/0x2180
>> [  285.786760]  path_openat+0x1a0/0x6d0
>> [  285.787791]  do_filp_open+0x14c/0x208
>> [  285.788844]  do_sys_open+0x340/0x470
>> [  285.789880]  __arm64_sys_openat+0x98/0xd8
>> [  285.791039]  el0_svc_common+0x230/0x3f0
>> [  285.792139]  el0_svc_handler+0x144/0x1a8
>> [  285.793260]  el0_svc+0x8/0x1b0
>> [  285.794283] XFS (dm-0): xfs_do_force_shutdown(0x8) called from line 1047
>> of file fs/xfs/xfs_trans.c.  Return address = 00000000a4a366b9
>> [  285.816187] XFS (dm-0): Corruption of in-memory data detected. Shutting
>> down filesystem
>> [  285.818476] XFS (dm-0): Please umount the filesystem and rectify the
>> problem(s)
> 
> Yup, that's a shutdown with a dirty transaction because memory
> allocation failed in the middle of a transaction. XFS can not
> tolerate memory allocation failure within the scope of a dirty
> transactions and, in practice, this almost never happens. Indeed,
> I've never seen this allocation from security_inode_alloc():
> 
> int lsm_inode_alloc(struct inode *inode)
> {
>          if (!lsm_inode_cache) {
>                  inode->i_security = NULL;
>                  return 0;
>          }
> 
>>>>>>    inode->i_security = kmem_cache_zalloc(lsm_inode_cache, GFP_NOFS);
>          if (inode->i_security == NULL)
>                  return -ENOMEM;
>          return 0;
> }
> 
> fail in all my OOM testing. Hence, to me, this is a theoretical
> failure as I've never, ever seen this allocation fail in production
> or test systems, even when driving them hard into OOM with excessive
> inode allocation and triggering the OOM killer repeatedly until the
> system kills init....
> 
> Hence I don't think there's anything we need to change here right
> now. If users start hitting this, then we're going to have add new
> memalloc_nofail_save/restore() functionality to XFS transaction
> contexts. But until then, I don't think we need to worry about
> syzkaller intentionally hitting this shutdown.

Thanks Dave.

   In the actual test, the x86 or arm64 device test will trigger this 
error more easily when FAILSLAB is turned on. After our internal 
discussion, we can try again through such a patch. Anyway, thank you for 
your reply.

diff --git a/fs/xfs/xfs_icache.c b/fs/xfs/xfs_icache.c
index ceee27b70384..360304409c0c 100644
--- a/fs/xfs/xfs_icache.c
+++ b/fs/xfs/xfs_icache.c
@@ -435,6 +435,7 @@ xfs_iget_cache_hit(
                                 wake_up_bit(&ip->i_flags, __XFS_INEW_BIT);
                         ASSERT(ip->i_flags & XFS_IRECLAIMABLE);
                         trace_xfs_iget_reclaim_fail(ip);
+                       error = -EAGAIN;
                         goto out_error;
                 }

@@ -503,7 +504,7 @@ xfs_iget_cache_miss(

         ip = xfs_inode_alloc(mp, ino);
         if (!ip)
-               return -ENOMEM;
+               return -EAGAIN;

         error = xfs_iread(mp, tp, ip, flags);
         if (error)


--
BR, Jackie Liu

> 
> Cheers,
> 
> Dave.
> 

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ