linux-kernel - Re: High priority tasks break SMP balancer?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20071117010352.GA13666@vmware.com>
Date:	Fri, 16 Nov 2007 17:03:52 -0800
From:	Micah Dowty <micah@...are.com>
To:	Dmitry Adamushko <dmitry.adamushko@...il.com>
Cc:	Ingo Molnar <mingo@...e.hu>, Christoph Lameter <clameter@....com>,
	Kyle Moffett <mrmacman_g4@....com>,
	Cyrus Massoumi <cyrusm@....net>,
	LKML Kernel <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...l.org>, Mike Galbraith <efault@....de>,
	Paul Menage <menage@...gle.com>,
	Peter Williams <pwil3058@...pond.net.au>
Subject: Re: High priority tasks break SMP balancer?

On Sat, Nov 17, 2007 at 12:26:41AM +0100, Dmitry Adamushko wrote:
> Let's say we change a pattern for the niced task: e.g. run for 100 ms.
> and then sleep for 300 ms. (that's ~25% of cpu load) in the loop. Any
> behavioral changes?

For consistency, I tested this using /dev/rtc. I set the rtc frequency
to 16 Hz, and replaced the main loop of my high (-19) priority thread
with:

   while (1) {
      unsigned long data;

      for (i = 0; i < 3; i++) {
         if (read(rtc, &data, sizeof data) != sizeof data) {
            perror("read");
            return 1;
         }
      }

      fcntl(rtc, F_SETFL, O_NONBLOCK);
      while (read(rtc, &data, sizeof data) < 0);
      fcntl(rtc, F_SETFL, 0);
   }

Now it's busy-looping for 62ms, and sleeping for three consecutive
62.5ms chunks totalling 187.5ms.

The results aren't quite what I was expecting. I have only observed
this so far in test cases where I have a very high wakeup frequency,
so I wasn't expecting this to work. I did, however, still observe the
problem where occasionally I get into a state where one CPU is mostly
idle.

Qualitatively, this feels a bit different. With the higher clock
frequency it seemed like the CPU would easily get "stuck" in this
state where it's mostly idle, and it would stay there for a long
time. With the low wakeup frequency, I'm seeing it toggle between the
busy and mostly-idle states more quickly.

I tried a similar test using usleep() and gettimeofday() rather than
/dev/rtc:

   while (1) {
      usleep(300000);

      gettimeofday(&t1, NULL);
      do {
         gettimeofday(&t2, NULL);
      } while (t2.tv_usec - t1.tv_usec +
               (t2.tv_sec - t1.tv_sec) * 1000000 < 100000);
   }

With this test program, I haven't yet seen a CPU imbalance that lasts
longer than a fraction of a second.

--Micah

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/