linux-kernel - Linux scheduler capabilities for batch jobs.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <6f41bd4b0906010641n1b360dffs6eaed72eb24f60f5@mail.gmail.com>
Date:	Mon, 1 Jun 2009 09:41:07 -0400
From:	J Louis <handstogether8@...il.com>
To:	linux-kernel@...r.kernel.org
Subject: Linux scheduler capabilities for batch jobs.

Hello All,

I have been writing user land server code and increasingly I find
myself writing resource management code which I think would be better
handled by the scheduler.  The problem can be described as "don't run
this process if the machine is swapping."  I would think that this
would be a common enough need that it was already in the kernel.  I'm
hoping it is and I have simply overlooked it (part of the reason for
this post,) but I've looked around a good bit and most of the
scheduler enhancements have to do with real time and latency, not
batch jobs.

My problem is analogous to a parallel make.  Say I have an 8 CPU
machine, and I run "make -j8".  If the total memory of the 8 jobs
throws the machine into swap, it begins to thrash and runtime is
awful.  I believe this is aggravated by the scheduler trying to be
fair, and keeping all 8 processes running.  If it was possible to tell
the scheduler that it was OK not to be fair when scheduling these
processes, I think the total runtime could be reduced if it put some
of the processes to sleep while others completed.  Is there a way to
tell the scheduler it is allowed to do this?  Should there be?

Thank You,

Hands
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/