Message-ID: <aX019VTWjMlPX8qp@fedora>
Date: Fri, 30 Jan 2026 23:51:33 +0100
From: Niklas Cassel <cassel@...nel.org>
To: Koichiro Den <den@...inux.co.jp>
Cc: Aksh Garg <a-garg7@...com>, linux-pci@...r.kernel.org,
	jingoohan1@...il.com, mani@...nel.org, lpieralisi@...nel.org,
	kwilczynski@...nel.org, robh@...nel.org, bhelgaas@...gle.com,
	Zhiqiang.Hou@....com, gustavo.pimentel@...opsys.com,
	linux-kernel@...r.kernel.org, s-vadapalli@...com,
	danishanwar@...com
Subject: Re: [PATCH v4 2/3] PCI: dwc: ep: Add per-PF BAR and inbound ATU
 mapping support

Hello Koichiro,

On Sat, Jan 31, 2026 at 02:16:37AM +0900, Koichiro Den wrote:
> While looking at this, I spotted a subtle corner case that the missing
> 'return' has been masking.
> 
> Consider the following re-programming sequence for a BAR:
> 
>   1) The initial set_bar() programs the BAR in BAR Match Mode
>   2) The second set_bar() switches the BAR to Address Match Mode
>      - Ad part of this, set_bar() calls dw_pcie_ep_clear_ib_maps() before
>        re-programming.
>   3) The third set_bar() switches back to BAR Match Mode.
> 
> In step (3), the current behavior depends on how the caller handles the
> struct pci_epf_bar passed to set_bar()
> 
>   a) If the caller passes a freshly prepared epf_bar with .num_submap = 0,
>      dw_pcie_ep_clear_ib_maps() is called as expected.
> 
>   b) If the caller updates the same epf_bar in place (i.e. reused the
>      object that passed in step (2)) and simply updates .num_submap back to
>      0, dw_pcie_ep_clear_ib_maps() is NOT called because the condition in
>      [1] evaluates to false. The recently merged (my) pci-epf-test code
>      follows this patter, see [2].
> 
> The point is that ep_func->epf_bar[bar] only stores a pointer, and the
> actual object is owned by the API caller. Since struct pci_epf embeds the
> BAR descriptors, updating an epf_bar in place (pattern (b)) seems quite
> natural and, in my view, should be supported.

(cut)

> Requiring callers to always pass a new epf_bar (pattern (a)) feels
> contrary to the current API/data-structure design.

I disagree.

I previously suggested that you should look at pci_epf_test_enable_doorbell()
and pci_epf_test_disable_doorbell().

If we look at the workflow in pci-epf-test for the doorbell test case:

1)
pci_epf_test_epc_init() calls pci_epf_test_set_bar() which calls
pci_epc_set_bar(..., &epf->bar[bar]);

2)
pci_epf_test_enable_doorbell()
uses a new struct pci_epf_bar (epf_test->db_bar)

copies the barno, size and flags for the BAR is will dynamically change:
epf_test->db_bar.barno = bar;
epf_test->db_bar.size = epf->bar[bar].size;
epf_test->db_bar.flags = epf->bar[bar].flags;
calls pci_epc_set_bar(..., &epf_test->db_bar);

3)
pci_epf_test_disable_doorbell() calls 
pci_epc_set_bar(..., &epf->bar[bar]);
with the original unmodified struct pci_epf_bar.


Your new test case however, reuses epf->bar[bar] in step 1, 2, 3.

I think the solution is to change your test case so that you use same
workflow as the doorbell test case. Just add a new struct pci_epf_bar
to struct pci_epf_test where you store the temporary BAR.


I'm not a big fan of your suggested code changes.

I think adding the early return in dw_pcie_ep_clear_ib_maps() and
modifying your test case to use a new epf_bar are the only things
that should be fixed as soon as possible.


> One downside is that this introduces a behavioral change in a specific
> failure case that already existed prior to this series.
> 
> Concretely, this only affects the traditional reprogramming path where a
> BAR configured in BAR Match Mode is updated again to BAR Match Mode. If
> such an update fails (for example, because the new epf_bar->phys_addr is
> not aligned to region_align), the behavior changes as follows:
> 
>   - Before this change: the previously programmed BAR Match Mode mapping
>     remains in place.
>   - After this change: the existing mapping is torn down first, so no mapping
>     remains after the failure.
> 
> I don't think this change is a major issue, since the caller will still
> observe the failure and should handle the error accordingly in either case.
> That said, I'd appreciate feedback if retaining the old BAR Match Mode
> mapping on such failure scenario is considered important for the existing
> update path.

You have found a potential problem with the API though, but like you said,
this problem was there even before your changes, so this problem can take
more time to be addressed.

Since the DWC EP driver only stores pointers to struct pci_epf_bar BARs
in set_bar(), it is theoretically possible for the caller to modify this
struct after calling set_bar(). This means that any future safety check
against values in ep_func->epf_bar (e.g. [1]) is unreliable, as the caller
could have modified the struct after calling set_bar().

[1] https://github.com/torvalds/linux/blob/v6.19-rc7/drivers/pci/controller/dwc/pcie-designware-ep.c#L368-L370

I think the solution to this is to modify struct dw_pcie_ep_func:

@@ -472,7 +472,7 @@ struct dw_pcie_ep_func {
        u8                      msi_cap;        /* MSI capability offset */
        u8                      msix_cap;       /* MSI-X capability offset */
        u8                      bar_to_atu[PCI_STD_NUM_BARS];
-       struct pci_epf_bar      *epf_bar[PCI_STD_NUM_BARS];
+       struct pci_epf_bar      epf_bar[PCI_STD_NUM_BARS];
 
        /* Only for Address Match Mode inbound iATU */
        u32                     *ib_atu_indexes[PCI_STD_NUM_BARS];



And then modify set_bar() to something like:

@@ -588,7 +588,7 @@ static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
        if (ret)
                return ret;
 
-       ep_func->epf_bar[bar] = epf_bar;
+       memcpy(&ep_func->epf_bar[bar], epf_bar, sizeof *epf_bar);
 
        return 0;
 }


But that would also mean that we need to modify the if (ep_func->epf_bar[bar])
with something like if (ep_func->epf_bar_in_use[bar])

Or... I guess we could keep it as pointers, but use kmemdup() in set_bar()
to get our own immutable copy, and then kfree() the pointer in clear_bar()
(and in set_bar() before kmemduping the memory, if the DWC driver already
has a local copy (i.e. dynamically changing the BAR without first calling
clear_bar()))


Kind regards,
Niklas

Powered by blists - more mailing lists