linux-kernel - Re: [PATCH] irqchip: gicv3-its: Use NUMA aware memory allocation for ITS tables

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <2a715559-3759-5bd4-1346-bc95f023093b@codeaurora.org>
Date:   Mon, 3 Jul 2017 10:15:15 -0500
From:   Shanker Donthineni <shankerd@...eaurora.org>
To:     Marc Zyngier <marc.zyngier@....com>,
        Ganapatrao Kulkarni <gpkulkarni@...il.com>
Cc:     Jason Cooper <jason@...edaemon.net>,
        Vikram Sethi <vikrams@...eaurora.org>,
        linux-kernel <linux-kernel@...r.kernel.org>,
        Jayachandran C <jnair@...iumnetworks.com>,
        "ganapatrao.kulkarni@...ium.com" <ganapatrao.kulkarni@...ium.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        linux-arm-kernel <linux-arm-kernel@...ts.infradead.org>
Subject: Re: [PATCH] irqchip: gicv3-its: Use NUMA aware memory allocation for
 ITS tables

Hi Marc,

On 07/03/2017 09:53 AM, Marc Zyngier wrote:
> Hi Shanker,
> 
> On 03/07/17 15:24, Shanker Donthineni wrote:
>> Hi Marc,
>>
>> On 06/30/2017 03:51 AM, Marc Zyngier wrote:
>>> On 30/06/17 04:01, Ganapatrao Kulkarni wrote:
>>>> On Fri, Jun 30, 2017 at 8:04 AM, Ganapatrao Kulkarni
>>>> <gpkulkarni@...il.com> wrote:
>>>>> Hi Shanker,
>>>>>
>>>>> On Sun, Jun 25, 2017 at 9:16 PM, Shanker Donthineni
>>>>> <shankerd@...eaurora.org> wrote:
>>>>>> The NUMA node information is visible to ITS driver but not being used
>>>>>> other than handling errata. This patch allocates the memory for ITS
>>>>>> tables from the corresponding NUMA node using the appropriate NUMA
>>>>>> aware functions.
>>>>
>>>> IMHO, the description would have been more constructive?
>>>>
>>>> "All ITS tables are mapped by default to NODE 0 memory.
>>>> Adding changes to allocate memory from respective NUMA NODES of ITS devices.
>>>> This will optimize tables access and avoids unnecessary inter-node traffic."
>>>
>>> But more importantly, I'd like to see figures showing the actual benefit
>>> of this per-node allocation. Given that both of you guys have access to
>>> such platforms, please show me the numbers!
>>>
>>
>> I'll share the actual results which shows the improvement whenever 
>> available on our next chips. Current version of Qualcomm qdf2400 doesn't 
>> support multi socket configuration to capture results and share with you. 
>>
>> Do you see any other issues with this patch apart from the performance 
>> improvements. I strongly believe this brings the noticeable improvement 
>> in numbers on systems where it has multi node memory/CPU configuration.
> 
> I agree that it *could* show an improvement, but it very much depends on
> how often the ITS misses in its caches. For this kind of patches, I want
> to see two things:
> 

Just imagine systems with hundreds of PCI-SRIOV virtual functions and
assigning some of them to virtual machines, and systems with GICv4 feature. 
There should be a lot of cache misses on ITS VCPU, DEVICE and COLLECTION 
lookups. And also VLPI patches that you've posted for comments are forcing 
to use VLPI feature for each VM irrespective of pass-through device assignment. 


> 1) It brings a measurable benefit on NUMA platforms
> 2) it doesn't adversely impact non-NUMA systems
> 
It should not affect the ITS hardware behavior non-NUMA based system since 
software always allocate memory from a single (default) NUMA node.

> I can deal with (2), but I have no way of evaluating (1), mostly for the
> lack of an infrastructure exercising multiple ITSs at the same time.
> 
Agree with you, but it takes some time for me to provide the test results. 

> Thanks,
> 
> 	M.
> 

-- 
Shanker Donthineni
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.