| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <dc83957f-1fc6-be30-a14f-3fa4858f04ee@huawei.com>
Date: Mon, 29 Aug 2022 17:28:40 +0800
From: "liaochang (A)" <liaochang1@...wei.com>
To: Marc Zyngier <maz@...nel.org>
CC: <tglx@...utronix.de>, <samuel@...lland.org>, <brgl@...ev.pl>,
<andy.shevchenko@...il.com>, <mikelley@...rosoft.com>,
<lvjianmin@...ngson.cn>, <mark.rutland@....com>,
<linux-kernel@...r.kernel.org>, <john.garry@...wei.com>
Subject: Re: [PATCH 1/2] genirq: Record dangling hwirq number into struct
irq_data
在 2022/8/28 0:15, Marc Zyngier 写道:
> On Thu, 25 Aug 2022 07:08:18 +0100,
> Liao Chang <liaochang1@...wei.com> wrote:
>>
>> Following interrupt allocation process lead to some interrupts are
>> mapped in the low-level domain(Arm ITS), but they are never been mapped
>> at the higher level.
>>
>> irq_domain_alloc_irqs_hierarchy(.., nr_irqs, ...)
>> its_irq_domain_alloc(..., nr_irqs, ...)
>> its_alloc_device_irq(..., nr_irqs, ...)
>> bitmap_find_free_region(..., get_count_order(nr_irqs))
>>
>> Since ITS domain find a region of zero bits, the length of which must
>> aligned to power of two. If nr_irqs is 30, the length of zero bits is
>> actually 32, but only first 30 bits are really mapped.
>>
>> On teardown, low-level domain only free these interrupts that actually
>> mapped, and leave last interrupts dangling in the ITS domain. Thus the
>> ITS device resources are never freed. On device driver reload, dangling
>> interrupts prevent ITS domain from allocating enough resource.
>>
>> irq_domain_free_irqs_hierarchy(..., nr_irqs, ...)
>> its_irq_domain_free(..., irq_base + i, 1)
>> bitmap_release_region(..., irq_base + i, get_count_order(1))
>>
>> John reported this problem to LKML and Marc provided a solution and fix
>> it in the generic code, see the discussion from Link tag. Marc's patch
>> fix John's problem, but does not take care of some corner case, look one
>> example below.
>>
>> Step1: 32 interrupts allocated in LPI domain, but return the first 30 to
>> higher driver.
>>
>> 111111111111111111111111111111 11
>> |<------------0~29------------>|30,31|
>>
>> Step2: interrupt #16~28 are released one by one, then #0~15 and #29~31
>> still be there.
>>
>> 1111111111111111 0000000000000 1 11
>> |<-----0~15----->|<---16~28--->|29|30,31|
>>
>> Step#: on driver teardown, generic code will invoke ITS domain code
>> twice. The first time, #0~15 will be released, the second one, only #29
>> will be released(1 align to power of two).
>>
>> 0000000000000000 0000000000000 0 11
>> |<-----0~15----->|<---16~28--->|29|30,31|
>>
>> In short summary, the dangling problem stems from the number of released
>> hwirq is less than the one of allocated hwirq in ITS domain. In order to
>> fix this problem, make irq_data record the number of allocated but
>> unmapped hwirq. If hwirq followed by some unmapped bits, ITS domain
>> record the number of unmapped bits to the last irq_data mapped to higher
>> level, when the last hwirq followed by unmapped hwirq is released, some
>> dangling bit will be clear eventualy, look back the trivial example
>> above.
>>
>> Step1: record '2' into the irq_data.dangling of #29 hwirq.
>>
>> 111111111111111111111111111111 11
>> |<------------0~29------------>|30,31|
>> dangling: 000000000000000000000000000002
>>
>> Step2: no change
>>
>> 1111111111111111 0000000000000 1 11
>> |<-----0~15----->|<---16~28--->|29|30,31|
>> dangling: 0000000000000000 0000000000000 2
>>
>> Step3: ITS domain will release #30~31 since the irq_data.dangling of #29
>> is '2'.
>>
>> 0000000000000000 0000000000000 0 00
>> |<-----0~15----->|<---16~28--->|29|30,31|
>> dangling: 0000000000000000 0000000000000 2
>>
>> Fixes: 4615fbc3788dd ("genirq/irqdomain: Don't try to free an interrupt
>> that has no mapping")
>> Reported-by: John Garry <john.garry@...wei.com>
>> Signed-off-by: Liao Chang <liaochang1@...wei.com>
>> Link: https://lore.kernel.org/lkml/3d3d0155e66429968cb4f6b4feeae4b3@kernel.org/
>> ---
>> include/linux/irq.h | 10 ++++++++++
>> 1 file changed, 10 insertions(+)
>>
>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>> index c3eb89606c2b..c48f10a0c230 100644
>> --- a/include/linux/irq.h
>> +++ b/include/linux/irq.h
>> @@ -167,6 +167,10 @@ struct irq_common_data {
>> * @mask: precomputed bitmask for accessing the chip registers
>> * @irq: interrupt number
>> * @hwirq: hardware interrupt number, local to the interrupt domain
>> + * @dangling: amount of dangling hardware interrupt, Arm ITS allocate
>> + * hardware interrupt more than expected, aligned to power
>> + * of two, so that unsued interrupt number become dangling.
>> + * Use this field to record dangling bits follwoing @hwirq.
>> * @common: point to data shared by all irqchips
>> * @chip: low level interrupt hardware access
>> * @domain: Interrupt translation domain; responsible for mapping
>> @@ -180,6 +184,7 @@ struct irq_data {
>> u32 mask;
>> unsigned int irq;
>> unsigned long hwirq;
>> + unsigned long dangling;
>> struct irq_common_data *common;
>> struct irq_chip *chip;
>> struct irq_domain *domain;
>
>
> There is no way I will put ITS-specific hacks in the core, and I
> really don't think this is the correct way to address this. Also, why
> track this sort of thing on a per-interrupt basis, while this is
> obviously a device-level allocation?
Thanks for comment, my older version is introduce a dangling bitmap in ITS
irq_domain at the cose of one more bitmap. such as like this:
struct event_lpi_map {
unsigned long *lpi_map;
+ unsigned long *dangling_map;
...
}
+dev->event.map.dangling_map = bitmap_alloc(nevc);
Obviously this dangling bitmap is likely to be sparse, which means most of bits
are wasted, hence I decide to record dangling bits into the per-interrupt object.
Well, it sounds a bad idea...
>
> The real issue is that there is currently no way for the ITS code to
> know when we're done with *all* the interrupts of a device. This is
> what needs fixing.
Let my understand what you mean, you are saying let **device driver** record these mapped
and dangling interrupts, and free these dangling interrupts explicitly when driver is uninstall.
>
> M.
>
--
BR,
Liao, Chang
Powered by blists - more mailing lists