linux-kernel - Re: Plumbers 2018 - Performance and Scalability Microconference

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <0f49f722-1759-f097-ff46-4ec7286dc69e@oracle.com>
Date:   Mon, 10 Sep 2018 20:38:35 -0400
From:   Daniel Jordan <daniel.m.jordan@...cle.com>
To:     John Hubbard <jhubbard@...dia.com>, linux-kernel@...r.kernel.org,
        "linux-mm@...ck.org" <linux-mm@...ck.org>
Cc:     Aaron Lu <aaron.lu@...el.com>, alex.kogan@...cle.com,
        akpm@...ux-foundation.org, boqun.feng@...il.com, brouer@...hat.com,
        dave@...olabs.net, dave.dice@...cle.com,
        Dhaval Giani <dhaval.giani@...cle.com>, ktkhai@...tuozzo.com,
        ldufour@...ux.vnet.ibm.com, Pavel.Tatashin@...rosoft.com,
        paulmck@...ux.vnet.ibm.com, shady.issa@...cle.com,
        tariqt@...lanox.com, tglx@...utronix.de, tim.c.chen@...el.com,
        vbabka@...e.cz, longman@...hat.com, yang.shi@...ux.alibaba.com,
        shy828301@...il.com, Huang Ying <ying.huang@...el.com>,
        subhra.mazumdar@...cle.com,
        Steven Sistare <steven.sistare@...cle.com>, jwadams@...gle.com,
        ashwinch@...gle.com, sqazi@...gle.com,
        Shakeel Butt <shakeelb@...gle.com>, walken@...gle.com,
        rientjes@...gle.com, junaids@...gle.com,
        Neha Agarwal <nehaagarwal@...gle.com>
Subject: Re: Plumbers 2018 - Performance and Scalability Microconference

On 9/8/18 12:13 AM, John Hubbard wrote:
> I'm interested in the first 3 of those 4 topics, so if it doesn't conflict with HMM topics or
> fix-gup-with-dma topics, I'd like to attend.

Great, we'll add your name to the list.

> GPUs generally need to access large chunks of
> memory, and that includes migrating (dma-copying) pages around.
> 
> So for example a multi-threaded migration of huge pages between normal RAM and GPU memory is an
> intriguing direction (and I realize that it's a well-known topic, already). Doing that properly
> (how many threads to use?) seems like it requires scheduler interaction.

Yes, in past discussions of multithreading kernel work, there's been some discussion of a scheduler API that could answer "are there idle CPUs we could use to multithread?".

Instead of adding an interface, though, we could just let the scheduler do something it already knows how to do: prioritize.

Additional threads used to parallelize kernel work could run at the lowest priority (i.e. MAX_NICE).  If the machine is heavily loaded, these extra threads simply won't run and other workloads on the system will be unaffected.

There's the issue of priority inversion if one or more of those extra threads get started and are then preempted by normal-priority tasks midway through, but the main thread doing the job can just will its priority to each worker in turn once it's finished, so at most one thread will be active on a heavily loaded system, again leaving other workloads on the system undisturbed.