linux-kernel - Re: [PATCH v2 00/25] AMDKFD kernel driver

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140722074053.GI15237@phenom.ffwll.local>
Date:	Tue, 22 Jul 2014 09:40:53 +0200
From:	Daniel Vetter <daniel@...ll.ch>
To:	Jerome Glisse <j.glisse@...il.com>,
	Oded Gabbay <oded.gabbay@....com>,
	Christian König <deathsimple@...afone.de>,
	David Airlie <airlied@...ux.ie>,
	Alex Deucher <alexdeucher@...il.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	John Bridgman <John.Bridgman@....com>,
	Joerg Roedel <joro@...tes.org>,
	Andrew Lewycky <Andrew.Lewycky@....com>,
	Michel Dänzer <michel.daenzer@....com>,
	Ben Goz <Ben.Goz@....com>,
	Alexey Skidanov <Alexey.Skidanov@....com>,
	Evgeny Pinchuk <Evgeny.Pinchuk@....com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"dri-devel@...ts.freedesktop.org" <dri-devel@...ts.freedesktop.org>,
	linux-mm <linux-mm@...ck.org>
Subject: Re: [PATCH v2 00/25] AMDKFD kernel driver

On Tue, Jul 22, 2014 at 09:28:51AM +0200, Daniel Vetter wrote:
> On Mon, Jul 21, 2014 at 03:03:07PM -0400, Jerome Glisse wrote:
> > On Mon, Jul 21, 2014 at 09:41:29PM +0300, Oded Gabbay wrote:
> > > On 21/07/14 21:22, Daniel Vetter wrote:
> > > > On Mon, Jul 21, 2014 at 7:28 PM, Oded Gabbay <oded.gabbay@....com> wrote:
> > > >>> I'm not sure whether we can do the same trick with the hw scheduler. But
> > > >>> then unpinning hw contexts will drain the pipeline anyway, so I guess we
> > > >>> can just stop feeding the hw scheduler until it runs dry. And then unpin
> > > >>> and evict.
> > > >> So, I'm afraid but we can't do this for AMD Kaveri because:
> > > > 
> > > > Well as long as you can drain the hw scheduler queue (and you can do
> > > > that, worst case you have to unmap all the doorbells and other stuff
> > > > to intercept further submission from userspace) you can evict stuff.
> > > 
> > > I can't drain the hw scheduler queue, as I can't do mid-wave preemption.
> > > Moreover, if I use the dequeue request register to preempt a queue
> > > during a dispatch it may be that some waves (wave groups actually) of
> > > the dispatch have not yet been created, and when I reactivate the mqd,
> > > they should be created but are not. However, this works fine if you use
> > > the HIQ. the CP ucode correctly saves and restores the state of an
> > > outstanding dispatch. I don't think we have access to the state from
> > > software at all, so it's not a bug, it is "as designed".
> > > 
> > 
> > I think here Daniel is suggesting to unmapp the doorbell page, and track
> > each write made by userspace to it and while unmapped wait for the gpu to
> > drain or use some kind of fence on a special queue. Once GPU is drain we
> > can move pinned buffer, then remap the doorbell and update it to the last
> > value written by userspace which will resume execution to the next job.
> 
> Exactly, just prevent userspace from submitting more. And if you have
> misbehaving userspace that submits too much, reset the gpu and tell it
> that you're sorry but won't schedule any more work.
> 
> We have this already in i915 (since like all other gpus we're not
> preempting right now) and it works. There's some code floating around to
> even restrict the reset to _just_ the offending submission context, with
> nothing else getting corrupted.
> 
> You can do all this with the doorbells and unmapping them, but it's a
> pain. Much easier if you have a real ioctl, and I haven't seen anyone with
> perf data indicating that an ioctl would be too much overhead on linux.
> Neither in this thread nor internally here at intel.

Aside: Another reason why the ioctl is better than the doorbell is
integration with other drivers. Yeah I know this is about compute, but
sooner or later someone will want to e.g. post-proc video frames between
the v4l capture device and the gpu mpeg encoder. Or something else fancy.

Then you want to be able to somehow integrate into a cross-driver fence
framework like android syncpts, and you can't do that without an ioctl for
the compute submissions.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/