linux-kernel - Re: [RFC] audit: avoid soft lockup in audit_log

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130828204900.4dc02876@redhat.com>
Date:	Wed, 28 Aug 2013 20:49:00 -0400
From:	Luiz Capitulino <lcapitulino@...hat.com>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	linux-kernel@...r.kernel.org, oleg@...hat.com, eparis@...hat.com,
	rgb@...hat.com
Subject: Re: [RFC] audit: avoid soft lockup in audit_log_start()

On Wed, 28 Aug 2013 16:08:13 -0700
Andrew Morton <akpm@...ux-foundation.org> wrote:

> On Wed, 28 Aug 2013 18:54:36 -0400 Luiz Capitulino <lcapitulino@...hat.com> wrote:
> 
> > > Are you really sure that kauditd is stuck in schedule() and doesn't
> > > come out?
> > 
> > No, that's a guess. Inferred from:
> > 
> > 1. I tried calling wake_up_interruptible(&kauditd_wait); right
> >    before wait_for_auditd(). Nothing changes
> > 
> > 2. I added this debug printks:
> > 
> >   diff --git a/kernel/audit.c b/kernel/audit.c
> >   index 91e53d0..27448ad 100644
> >   --- a/kernel/audit.c
> >   +++ b/kernel/audit.c
> >   @@ -458,11 +458,14 @@ static int kauditd_thread(void *dummy)
> >    		set_current_state(TASK_INTERRUPTIBLE);
> >    		add_wait_queue(&kauditd_wait, &wait);
> >    
> >   +		pr_emerg_ratelimited("*** sleeping\n");
> >   +
> >    		if (!skb_queue_len(&audit_skb_queue)) {
> >    			try_to_freeze();
> >    			schedule();
> >    		}
> >    
> >   +		pr_emerg_ratelimited("*** waking up\n");
> >    		__set_current_state(TASK_RUNNING);
> >    		remove_wait_queue(&kauditd_wait, &wait);
> >    	}
> > 
> >  I get several pairs of sleeping/waking up strings right before the
> >  system begins to shut down. Then it stops (even though we do
> >  have SKBs queued)
> 
> Well.  I assume "*** sleeping" the last thing kauditd prints?  If the
> last print is "waking up" then obviously kauditd is stuck somewhere
> else, which makes more sense.  A sysrq-T (or echo t >
> /proc/sysrq-trigger) should provide clues.

Yes, "*** sleeping" is the last one.

> I'd remove the "_ratelimited" - you don't want your debug messages to be
> ratelimited (ie: omitted).

I'll try that.

Another interesting info is that the process that triggers the hang
is killall5 and by the time the hang happens auditd is not running
anymore.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/