linux-kernel - Re: A long overdue fork-bomb defense ?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <B9B9DCEE-5AAE-4EEA-B89A-403F36A26064@osndok.com>
Date:	Sat, 9 Apr 2011 18:20:42 -0500
From:	Robert Hailey <lkml@...dok.com>
To:	Valdis.Kletnieks@...edu
Cc:	linux-kernel@...r.kernel.org
Subject: Re: A long overdue fork-bomb defense ?

On 2011/04/09 (Apr), at 5:44 PM, Valdis.Kletnieks@...edu wrote:

> On Sat, 09 Apr 2011 17:25:48 CDT, Robert Hailey said:
>
>> Is there a better way to handle the integer overflows?
> ...
> Might want to think about what situations will cause an integer  
> overflow
> here.  What has to happen before an overflow is a concern?

It's counting forks per-process, so it is theoretically bound to  
overflow eventually.

If we use an unsigned 32-bit counter and take 'cron' as an example  
(say it forks 20 times / minute), it would take about 400 years to  
overflow. If on the other hand, we have a naive transaction processor  
using inetd (which if i'm not mistaken forks at every inbound  
connection) at 60,000/second (!) it will wrap about every 48 days.  
Using the div-2 method the next will come sooner, so maybe every  
month....

I think the cost of lock is well amortized even at 32-bits.

>> Thankfully, this logic would only be triggered when the process table
>> is full. At that point I doubt anyone would miss the compute time of
>> even the most painful lock :)
>
> If you get to the point where it's full, you've already lost.
> ...
> Depends why a fork bomb was allowed to be fatal in the first place...

Can you elaborate as to why "you've already lost" if the process table  
is full?

I am particularly interested in any additional information as to why  
fork bombs are fatal, and what the most evil/elusive fork bomb would  
be. At present I think it's about this:

bomb (rate, memory) {
	while(true) {
		if (root) {
			daemonize();
			new_session();
			new_process_group();
			new_executable_name();
			if (some_chance) change_uid;
		}
		leak_some(memory);
		bomb(rate*2, memory);
		delay(rate);
	}
}
bomb(time_on_target, memory_ceiling_tuned);

>
>> Presuming for a moment that it works, I think the worst case is
>> actually a single (perhaps compromised) process spawning child fork
>> bombs. For that matter it could be a bash shell with the user setting
>> them off. In that case it might *never* cause enough forking it to  
>> get
>> itself automatically killed, but the system would still be  
>> [somewhat?]
>
> bash$ :(){ :|:&};:
>
> Try it and see. ;)

Not yet, but maybe soon... ;-)

>
>> responsive through the attack b/c it no longer denies a legitimate
>> fork, i.e. logging in & using a shell work, even while the process
>> table is *FULL* of active fork bombs.
>
> I suspect recent patches that allow an entire process tree to be  
> sent SIGSTOP
> will be a more productive approach.

Surely you're not relying on a user to catch the fork bomb, so how is  
it detected?

BTW, already starting to get off discussion of original idea & psuedo- 
code.

--
Robert Hailey

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/