[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <Z0X7UE/ZfM6fgL1r@lizhi-Precision-Tower-5810>
Date: Tue, 26 Nov 2024 11:46:08 -0500
From: Frank Li <Frank.li@....com>
To: Manivannan Sadhasivam <manivannan.sadhasivam@...aro.org>
Cc: Krzysztof Wilczyński <kw@...ux.com>,
Kishon Vijay Abraham I <kishon@...nel.org>,
Bjorn Helgaas <bhelgaas@...gle.com>, Arnd Bergmann <arnd@...db.de>,
Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
linux-kernel@...r.kernel.org, linux-pci@...r.kernel.org,
imx@...ts.linux.dev, Niklas Cassel <cassel@...nel.org>,
dlemoal@...nel.org, maz@...nel.org, tglx@...utronix.de,
jdmason@...zu.us
Subject: Re: [PATCH v8 4/6] PCI: endpoint: pci-epf-test: Add doorbell test
support
On Tue, Nov 26, 2024 at 09:55:23AM +0530, Manivannan Sadhasivam wrote:
> On Mon, Nov 25, 2024 at 02:17:04PM -0500, Frank Li wrote:
> > On Sun, Nov 24, 2024 at 01:26:45PM +0530, Manivannan Sadhasivam wrote:
> > > On Sat, Nov 16, 2024 at 09:40:44AM -0500, Frank Li wrote:
> > > > Add three registers: doorbell_bar, doorbell_addr, and doorbell_data,
> > > > along with doorbell_done. Use pci_epf_alloc_doorbell() to allocate a
> > >
> > > I don't see 'doorbell_done' defined anywhere.
> > >
> > > > doorbell address space.
> > > >
> > > > Enable the Root Complex (RC) side driver to trigger pci-epc-test's doorbell
> > > > callback handler by writing doorbell_data to the mapped doorbell_bar's
> > > > address space.
> > > >
> > > > Set doorbell_done in the doorbell callback to indicate completion.
> > > >
> > >
> > > Same here.
> > >
> > > > To avoid broken compatibility, add new command COMMAND_ENABLE_DOORBELL
> > >
> > > 'avoid breaking compatibility between host and endpoint,...'
> > >
> > > > and COMMAND_DISABLE_DOORBELL. Host side need send COMMAND_ENABLE_DOORBELL
> > > > to map one bar's inbound address to MSI space. the command
> > > > COMMAND_DISABLE_DOORBELL to recovery original inbound address mapping.
> > > >
> > > > Host side new driver Host side old driver
> > > >
> > > > EP: new driver S F
> > > > EP: old driver F F
> > >
> > > So the last case of old EP and host drivers will fail?
> >
> > doorbell test will fail if old EP.
> >
>
> How come there would be doorbell test if it is an old host driver?
fail by unsupport ioctrl(). It should "implicit default behavior".
Frank
>
> > >
> > > >
> > > > S: If EP side support MSI, 'pcitest -B' return success.
> > > > If EP side doesn't support MSI, the same to 'F'.
> > > >
> > > > F: 'pcitest -B' return failure, other case as usual.
> > > >
> > > > Tested-by: Niklas Cassel <cassel@...nel.org>
> > > > Signed-off-by: Frank Li <Frank.Li@....com>
> > > > ---
> > > > Change from v7 to v8
> > > > - rename to pci_epf_align_inbound_addr_lo_hi()
> > > >
> > > > Change from v6 to v7
> > > > - use help function pci_epf_align_addr_lo_hi()
> > > >
> > > > Change from v5 to v6
> > > > - rename doorbell_addr to doorbell_offset
> > > >
> > > > Chagne from v4 to v5
> > > > - Add doorbell free at unbind function.
> > > > - Move msi irq handler to here to more complex user case, such as differece
> > > > doorbell can use difference handler function.
> > > > - Add Niklas's code to handle fixed bar's case. If need add your signed-off
> > > > tag or co-developer tag, please let me know.
> > > >
> > > > change from v3 to v4
> > > > - remove revid requirement
> > > > - Add command COMMAND_ENABLE_DOORBELL and COMMAND_DISABLE_DOORBELL.
> > > > - call pci_epc_set_bar() to map inbound address to MSI space only at
> > > > COMMAND_ENABLE_DOORBELL.
> > > > ---
> > > > drivers/pci/endpoint/functions/pci-epf-test.c | 117 ++++++++++++++++++++++++++
> > > > 1 file changed, 117 insertions(+)
> > > >
> > > > diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
> > > > index ef6677f34116e..410b2f4bb7ce7 100644
> > > > --- a/drivers/pci/endpoint/functions/pci-epf-test.c
> > > > +++ b/drivers/pci/endpoint/functions/pci-epf-test.c
> > > > @@ -11,12 +11,14 @@
> > > > #include <linux/dmaengine.h>
> > > > #include <linux/io.h>
> > > > #include <linux/module.h>
> > > > +#include <linux/msi.h>
> > > > #include <linux/slab.h>
> > > > #include <linux/pci_ids.h>
> > > > #include <linux/random.h>
> > > >
> > > > #include <linux/pci-epc.h>
> > > > #include <linux/pci-epf.h>
> > > > +#include <linux/pci-ep-msi.h>
> > > > #include <linux/pci_regs.h>
> > > >
> > > > #define IRQ_TYPE_INTX 0
> > > > @@ -29,6 +31,8 @@
> > > > #define COMMAND_READ BIT(3)
> > > > #define COMMAND_WRITE BIT(4)
> > > > #define COMMAND_COPY BIT(5)
> > > > +#define COMMAND_ENABLE_DOORBELL BIT(6)
> > > > +#define COMMAND_DISABLE_DOORBELL BIT(7)
> > > >
> > > > #define STATUS_READ_SUCCESS BIT(0)
> > > > #define STATUS_READ_FAIL BIT(1)
> > > > @@ -39,6 +43,11 @@
> > > > #define STATUS_IRQ_RAISED BIT(6)
> > > > #define STATUS_SRC_ADDR_INVALID BIT(7)
> > > > #define STATUS_DST_ADDR_INVALID BIT(8)
> > > > +#define STATUS_DOORBELL_SUCCESS BIT(9)
> > > > +#define STATUS_DOORBELL_ENABLE_SUCCESS BIT(10)
> > > > +#define STATUS_DOORBELL_ENABLE_FAIL BIT(11)
> > > > +#define STATUS_DOORBELL_DISABLE_SUCCESS BIT(12)
> > > > +#define STATUS_DOORBELL_DISABLE_FAIL BIT(13)
> > > >
> > > > #define FLAG_USE_DMA BIT(0)
> > > >
> > > > @@ -74,6 +83,9 @@ struct pci_epf_test_reg {
> > > > u32 irq_type;
> > > > u32 irq_number;
> > > > u32 flags;
> > > > + u32 doorbell_bar;
> > > > + u32 doorbell_offset;
> > > > + u32 doorbell_data;
> > > > } __packed;
> > > >
> > > > static struct pci_epf_header test_header = {
> > > > @@ -642,6 +654,63 @@ static void pci_epf_test_raise_irq(struct pci_epf_test *epf_test,
> > > > }
> > > > }
> > > >
> > > > +static void pci_epf_enable_doorbell(struct pci_epf_test *epf_test, struct pci_epf_test_reg *reg)
> > > > +{
> > > > + enum pci_barno bar = reg->doorbell_bar;
> > > > + struct pci_epf *epf = epf_test->epf;
> > > > + struct pci_epc *epc = epf->epc;
> > > > + struct pci_epf_bar db_bar;
> > >
> > > db_bar = {};
> > >
> > > > + struct msi_msg *msg;
> > > > + size_t offset;
> > > > + int ret;
> > > > +
> > > > + if (bar < BAR_0 || bar == epf_test->test_reg_bar || !epf->db_msg) {
> > >
> > > What is the need of BAR check here and below? pci_epf_alloc_doorbell() should've
> > > allocated proper BAR already.
> >
> > Not check it at call pci_epf_alloc_doorbell() because it optional feature.
>
> What is 'optional feature' here? allocating doorbell?
Not all platform support ITS. If hardware have not ITS support,
pci_epf_alloc_doorbell() will return fail.
>
> > It return failure when it actually use it.
> >
>
> So host can call pci_epf_enable_doorbell() without pci_epf_alloc_doorbell()?
Suppose yes because host don't know if EP support doorbell.
>
> > >
> > > > + reg->status |= STATUS_DOORBELL_ENABLE_FAIL;
> > > > + return;
> > > > + }
> > > > +
> > > > + msg = &epf->db_msg[0].msg;
> > > > + ret = pci_epf_align_inbound_addr_lo_hi(epf, bar, msg->address_lo, msg->address_hi,
> > > > + &db_bar.phys_addr, &offset);
> > > > +
> > > > + if (ret) {
> > > > + reg->status |= STATUS_DOORBELL_ENABLE_FAIL;
> > > > + return;
> > > > + }
> > > > +
> > > > + reg->doorbell_offset = offset;
> > > > +
> > > > + db_bar.barno = bar;
> > > > + db_bar.size = epf->bar[bar].size;
> > > > + db_bar.flags = epf->bar[bar].flags;
> > > > + db_bar.addr = NULL;
> > >
> > > Not needed if you initialize above.
> > >
> > > > +
> > > > + ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, &db_bar);
> > > > + if (!ret)
> > > > + reg->status |= STATUS_DOORBELL_ENABLE_SUCCESS;
> > > > + else
> > > > + reg->status |= STATUS_DOORBELL_ENABLE_FAIL;
> > > > +}
> > > > +
> > >
> > > [...]
> > >
> > > > static const struct pci_epc_event_ops pci_epf_test_event_ops = {
> > > > .epc_init = pci_epf_test_epc_init,
> > > > .epc_deinit = pci_epf_test_epc_deinit,
> > > > @@ -921,12 +1010,34 @@ static int pci_epf_test_bind(struct pci_epf *epf)
> > > > if (ret)
> > > > return ret;
> > > >
> > > > + ret = pci_epf_alloc_doorbell(epf, 1);
> > > > + if (!ret) {
> > > > + struct pci_epf_test_reg *reg = epf_test->reg[test_reg_bar];
> > > > + struct msi_msg *msg = &epf->db_msg[0].msg;
> > > > + enum pci_barno bar;
> > > > +
> > > > + bar = pci_epc_get_next_free_bar(epc_features, test_reg_bar + 1);
> > >
> > > NO_BAR check?
> >
> > This is optional feature, It should check when use it.
> >
>
> NO. Why would you call request_irq() if the doorbell BAR is not available? It
> doesn't make sense.
Maybe reasonable now. But it will be not true if we support map two
seperate memory regions to a bar in future. Anyway I can add check here.
Frank
>
> - Mani
>
> --
> மணிவண்ணன் சதாசிவம்
Powered by blists - more mailing lists