Message-ID: <20150115184726.GA23247@obsidianresearch.com>
Date:	Thu, 15 Jan 2015 11:47:26 -0700
From:	Jason Gunthorpe <jgunthorpe@...idianresearch.com>
To:	atull <atull@...nsource.altera.com>
Cc:	gregkh@...uxfoundation.org,
	One Thousand Gnomes <gnomes@...rguk.ukuu.org.uk>,
	Pavel Machek <pavel@...x.de>, hpa@...or.com, monstr@...str.eu,
	michal.simek@...inx.com, rdunlap@...radead.org,
	linux-kernel@...r.kernel.org, devicetree@...r.kernel.org,
	pantelis.antoniou@...sulko.com, robh+dt@...nel.org,
	grant.likely@...aro.org, iws@...o.caltech.edu,
	linux-doc@...r.kernel.org, broonie@...nel.org, philip@...ister.org,
	rubini@...dd.com, s.trumtrar@...gutronix.de, jason@...edaemon.net,
	kyle.teske@...com, nico@...aro.org, balbi@...com,
	m.chehab@...sung.com, davidb@...eaurora.org, rob@...dley.net,
	davem@...emloft.net, cesarb@...arb.net, sameo@...ux.intel.com,
	akpm@...ux-foundation.org, linus.walleij@...aro.org,
	pawel.moll@....com, mark.rutland@....com,
	ijc+devicetree@...lion.org.uk, galak@...eaurora.org,
	devel@...verdev.osuosl.org, delicious.quinoa@...il.com,
	dinguyen@...nsource.altera.com, yvanderv@...nsource.altera.com
Subject: Re: [PATCH v8 2/4] fpga manager: add sysfs interface document

On Thu, Jan 15, 2015 at 10:34:39AM -0600, atull wrote:

> This is great!  The way I had it working was using Pantelis' devicetree
> configfs interface.

I figured you were very close to this already in your overlay work..
 
> The DT fragment described the FPGA logic and included a filename
> for firmware to load. In another branch of this thread, I see discussion
> starting on what the overlay should look like and whether it could somehow
> contain the DT itself.

It is a novel idea, my concern would be that embedding the FPGA in the
DT makes it permanent unswappable kernel memory.

Not having the kernel hold the FPGA is best for many uses.

Having the kernel hold the FPGA as a swppable file handle/mmap of some
sort is next best (obviously the fs must be operating during resume)

Unswappable kernel memory is the worst choice

> Long ago this driver started out with a /dev interface.  It didn't have
> an ioctl yet at that point, but programming the fpga was by opening
> the devnode and writing to it.  Greg KH preferred sysfs or configfs
> over adding another ioctl:

I think to justify the ioctls you need a reason to have the context
of a FD. sysfs is stateless, so if my process dies the kernel doesn't
know.

But now that we are talking about adding locking and ownership
concepts a FD is the natural anchor for that in user space.

Ie, if I open the dev node, program a FPGA and then crash the kernel
doesn't attach drivers, and immediately de-programs the
chip. Userspace has to make it all the way through to the DT bind
before the FPGA lifetime would exceed the FD.

> https://lkml.org/lkml/2013/10/8/677

I think Greg's reply makes sense in the context of the question being
asked. Thinking of the FPGA as lockable ref counted kind of resource
changes the question somewhat.

Identifying the ioctls needed would probably clarify things. My
rough start would be 

- Get status: programed, not programmed, error?
  Bitfile type? (eg Xilinx has nearly every permutation of bit/byte
  ordering)
- Start Program with with some kind of context (ie this a new bit
  file, partial reconfiguration basis X, partial reconfiguration
  overlay on X)
- for (;;) write() to do programming
- Get Error to return detailed failure information (CRC error,
  auth error, etc)
- Hand over to a DT overlay (how does this work?) Lock transfers
  from FD to kernel

-  .. something something VFIO .. ?

Where start program is refused if the FPGA is already locked, and
  locks it 
Where start program -> close() returns the FPGA back to reset and
  unlocks
Where start program -> hand over -> close() keeps the FPGA loaded with
 kernel drivers attached and fpga locked (remove the overlay to
 de-program and unlock)

Not sure exactly how to tie together DT overlays with the FPGA state,
but that seems the natural combination..

Not sure about partial reconfiguration - clearly the kernel needs to
know and check that the bitfiles are of the correct family, I wonder
if the approach should be to program a basis on the FPGA which then
creates a new FPGA device in the system that can accept the partial
reconfiguration - this way the locking makes sense, the basis is
locked by the kernel and devices and the overlay remains
lockable/swappable/whatever by a dedicated /dev/ node ??

Just thinking aloud, I've never had a use case for partial
reconfiguration.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists