lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Fri, 22 Apr 2022 21:15:01 +0800 From: "zhangfei.gao@...mail.com" <zhangfei.gao@...mail.com> To: Jean-Philippe Brucker <jean-philippe@...aro.org> Cc: Fenghua Yu <fenghua.yu@...el.com>, Dave Hansen <dave.hansen@...el.com>, Joerg Roedel <joro@...tes.org>, Ravi V Shankar <ravi.v.shankar@...el.com>, Tony Luck <tony.luck@...el.com>, Ashok Raj <ashok.raj@...el.com>, Peter Zijlstra <peterz@...radead.org>, Dave Hansen <dave.hansen@...ux.intel.com>, x86 <x86@...nel.org>, linux-kernel <linux-kernel@...r.kernel.org>, iommu <iommu@...ts.linux-foundation.org>, Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>, Andy Lutomirski <luto@...nel.org>, Josh Poimboeuf <jpoimboe@...hat.com>, Thomas Gleixner <tglx@...utronix.de>, will@...nel.org, robin.murphy@....com, zhangfei.gao@...aro.org Subject: Re: Re: [PATCH v4 05/11] iommu/sva: Assign a PASID to mm on PASID allocation and free it on mm exit Hi, Jean On 2022/4/22 下午6:11, Jean-Philippe Brucker wrote: > On Fri, Apr 22, 2022 at 05:03:10PM +0800, zhangfei.gao@...mail.com wrote: > [...] >>> Have tested, still got some issue with our openssl-engine. >>> >>> 1. If openssl-engine does not register rsa, nginx works well. >>> >>> 2. If openssl-engine register rsa, nginx also works, but ioasid is not >>> freed when nginx stop. >>> >>> IMPLEMENT_DYNAMIC_BIND_FN(bind_fn) >>> bind_fn >>> ENGINE_set_RSA(e, rsa_methods()) >>> >>> destroy_fn >>> >>> If ENGINE_set_RSA is set, nginx start and stop will NOT call destroy_fn. >>> Even rsa_methods is almost new via RSA_meth_new. >>> >>> In 5.18-rcx, this caused ioasid not freed in nginx start and stop. >>> In 5.17, though destroy_fn is not called, but ioasid is freed when nginx >>> stop, so not noticed this issue before. >> 1. uacce_fops_release >> In 5.16 or 5.17 >> In fact, we aslo has the issue: openssl engine does not call destroy_fn -> >> close(uacce_fd) >> But system will automatically close all opened fd, >> so uacce_fops_release is also called and free ioasid. >> >> Have one experiment, not call close fd >> >> log: open uacce fd but no close >> [ 2583.471225] dump_backtrace+0x0/0x1a0 >> [ 2583.474876] show_stack+0x20/0x30 >> [ 2583.478178] dump_stack_lvl+0x8c/0xb8 >> [ 2583.481825] dump_stack+0x18/0x34 >> [ 2583.485126] uacce_fops_release+0x44/0xdc >> [ 2583.489117] __fput+0x78/0x240 >> [ 2583.492159] ____fput+0x18/0x28 >> [ 2583.495288] task_work_run+0x88/0x160 >> [ 2583.498936] do_notify_resume+0x214/0x490 >> [ 2583.502927] el0_svc+0x58/0x70 >> [ 2583.505968] el0t_64_sync_handler+0xb0/0xb8 >> [ 2583.510132] el0t_64_sync+0x1a0/0x1a4 >> [ 2583.582292] uacce_fops_release q=00000000d6674128 >> >> In 5.18, since refcount was add. >> The opened uacce fd was not closed automatically by system >> So we see the issue. >> >> log: open uacce fd but no close >> [ 106.360140] uacce_fops_open q=00000000ccc38d74 >> [ 106.364929] ioasid_alloc ioasid=1 >> [ 106.368585] iommu_sva_alloc_pasid pasid=1 >> [ 106.372943] iommu_sva_bind_device handle=000000006cca298a >> // ioasid is not free > I'm trying to piece together what happens from the kernel point of view. > > * master process with mm A opens a queue fd through uacce, which calls > iommu_sva_bind_device(dev, A) -> PASID 1 > > * master forks and exits. Child (daemon) gets mm B, inherits the queue fd. > The device is still bound to mm A with PASID 1, since the queue fd is > still open. > We discussed this before, but I don't remember where we left off. The > child can't use the queue because its mappings are not copied on fork(), > and the queue is still bound to the parent mm A. The child either needs to > open a new queue or take ownership of the old one with a new uacce ioctl. Yes, currently nginx aligned with the case. Child process (worker process) reopen uacce, Master process (do init) open uacce, iommu_sva_bind_device(dev, A) -> PASID 1 Master process fork Child (daemon) and exit. Child (daemon) does not use PASID 1 any more, only fork and manage worker process. Worker process reopen uacce, iommu_sva_bind_device(dev, B) PASID 2 So it is expected. > Is that the "IMPLEMENT_DYNAMIC_BIND_FN()" you mention, something out of > tree? This operation should unbind from A before binding to B, no? > Otherwise we leak PASID 1. In 5.16 PASID 1 from master is hold until nginx service stop. nginx start master: iommu_sva_alloc_pasid mm->pasid=1 // master process lynx https start: iommu_sva_alloc_pasid mm->pasid=2 //worker process nginx stop: from fops_release iommu_sva_free_pasid mm->pasid=2 // worker process iommu_sva_free_pasid mm->pasid=1 // master process Have one silly question. kerne driver fops_open iommu_sva_bind_device fops_release iommu_sva_unbind_device application main() fd = open return; Application exit but not close(fd), is it expected fops_release will be called automatically by system? On 5.17 fops_release is called automatically, as well as iommu_sva_unbind_device. On 5.18-rc1. fops_release is not called, have to manually call close(fd) Since nginx may have a issue, it does not call close(fd) when nginx -s quit. Thanks
Powered by blists - more mailing lists