linux-kernel - Re: [RFCv3 PATCH 1/6] uacce: Add documents for WarpDrive/uacce

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20181119182752.GA4890@ziepe.ca>
Date:   Mon, 19 Nov 2018 11:27:52 -0700
From:   Jason Gunthorpe <jgg@...pe.ca>
To:     Jerome Glisse <jglisse@...hat.com>
Cc:     Leon Romanovsky <leon@...nel.org>,
        Kenneth Lee <liguozhu@...ilicon.com>,
        Tim Sell <timothy.sell@...sys.com>, linux-doc@...r.kernel.org,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Zaibo Xu <xuzaibo@...wei.com>, zhangfei.gao@...mail.com,
        linuxarm@...wei.com, haojian.zhuang@...aro.org,
        Christoph Lameter <cl@...ux.com>,
        Hao Fang <fanghao11@...wei.com>,
        Gavin Schenk <g.schenk@...elmann.de>,
        RDMA mailing list <linux-rdma@...r.kernel.org>,
        Zhou Wang <wangzhou1@...ilicon.com>,
        Doug Ledford <dledford@...hat.com>,
        Uwe Kleine-König 
        <u.kleine-koenig@...gutronix.de>,
        David Kershner <david.kershner@...sys.com>,
        Kenneth Lee <nek.in.cn@...il.com>,
        Johan Hovold <johan@...nel.org>,
        Cyrille Pitchen <cyrille.pitchen@...e-electrons.com>,
        Sagar Dharia <sdharia@...eaurora.org>,
        Jens Axboe <axboe@...nel.dk>, guodong.xu@...aro.org,
        linux-netdev <netdev@...r.kernel.org>,
        Randy Dunlap <rdunlap@...radead.org>,
        linux-kernel@...r.kernel.org, Vinod Koul <vkoul@...nel.org>,
        linux-crypto@...r.kernel.org,
        Philippe Ombredanne <pombredanne@...b.com>,
        Sanyog Kale <sanyog.r.kale@...el.com>,
        "David S. Miller" <davem@...emloft.net>,
        linux-accelerators@...ts.ozlabs.org
Subject: Re: [RFCv3 PATCH 1/6] uacce: Add documents for WarpDrive/uacce

On Mon, Nov 19, 2018 at 11:48:54AM -0500, Jerome Glisse wrote:

> Just to comment on this, any infiniband driver which use umem and do
> not have ODP (here ODP for me means listening to mmu notifier so all
> infiniband driver except mlx5) will be affected by same issue AFAICT.
> 
> AFAICT there is no special thing happening after fork() inside any of
> those driver. So if parent create a umem mr before fork() and program
> hardware with it then after fork() the parent might start using new
> page for the umem range while the old memory is use by the child. The
> reverse is also true (parent using old memory and child new memory)
> bottom line you can not predict which memory the child or the parent
> will use for the range after fork().
> 
> So no matter what you consider the child or the parent, what the hw
> will use for the mr is unlikely to match what the CPU use for the
> same virtual address. In other word:
> 
> Before fork:
>     CPU parent: virtual addr ptr1 -> physical address = 0xCAFE
>     HARDWARE:   virtual addr ptr1 -> physical address = 0xCAFE
> 
> Case 1:
>     CPU parent: virtual addr ptr1 -> physical address = 0xCAFE
>     CPU child:  virtual addr ptr1 -> physical address = 0xDEAD
>     HARDWARE:   virtual addr ptr1 -> physical address = 0xCAFE
> 
> Case 2:
>     CPU parent: virtual addr ptr1 -> physical address = 0xBEEF
>     CPU child:  virtual addr ptr1 -> physical address = 0xCAFE
>     HARDWARE:   virtual addr ptr1 -> physical address = 0xCAFE

IIRC this is solved in IB by automatically calling
madvise(MADV_DONTFORK) before creating the MR.

MADV_DONTFORK
  .. This is useful to prevent copy-on-write semantics from changing the
  physical location of a page if the parent writes to it after a
  fork(2) ..

Jason