linux-kernel - Re: [tip:perfcounters/core] perf_counter: Simplify and fix task migration counting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 19 Jun 2009 13:59:39 +0200
From:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
To:	mingo@...hat.com, hpa@...or.com, paulus@...ba.org, acme@...hat.com,
	linux-kernel@...r.kernel.org, efault@....de, mtosatti@...hat.com,
	tglx@...utronix.de, cjashfor@...ux.vnet.ibm.com, mingo@...e.hu
Cc:	linux-tip-commits@...r.kernel.org
Subject: Re: [tip:perfcounters/core] perf_counter: Simplify and fix task
 migration counting

On Fri, 2009-06-19 at 11:52 +0000, tip-bot for Peter Zijlstra wrote:
> Commit-ID:  e5289d4a181fb6c0b7a7607649af2ffdc491335c
> Gitweb:     http://git.kernel.org/tip/e5289d4a181fb6c0b7a7607649af2ffdc491335c
> Author:     Peter Zijlstra <a.p.zijlstra@...llo.nl>
> AuthorDate: Fri, 19 Jun 2009 13:22:51 +0200
> Committer:  Ingo Molnar <mingo@...e.hu>
> CommitDate: Fri, 19 Jun 2009 13:43:12 +0200
> 
> perf_counter: Simplify and fix task migration counting
> 
> The task migrations counter was causing rare and hard to decypher
> memory corruptions under load. After a day of debugging and bisection
> we found that the problem was introduced with:
> 
>   3f731ca: perf_counter: Fix cpu migration counter
> 
> Turning them off fixes the crashes. Incidentally, the whole
> perf_counter_task_migration() logic can be done simpler as well,
> by injecting a proper sw-counter event.
> 
> This cleanup also fixed the crashes. The precise failure mode is
> not completely clear yet, but we are clearly not unhappy about
> having a fix ;-)


I actually do know what happens:

static struct perf_counter_context *
perf_lock_task_context(struct task_struct *task, unsigned long *flags)
{
	struct perf_counter_context *ctx;

	rcu_read_lock();
 retry:
	ctx = rcu_dereference(task->perf_counter_ctxp);
	if (ctx) {

		spin_lock_irqsave(&ctx->lock, *flags);
		if (ctx != rcu_dereference(task->perf_counter_ctxp)) {
			spin_unlock_irqrestore(&ctx->lock, *flags);
			goto retry;
		}
	}
	rcu_read_unlock();
	return ctx;
}


static struct perf_counter_context *perf_pin_task_context(struct task_struct *task)
{
	struct perf_counter_context *ctx;
	unsigned long flags;

	ctx = perf_lock_task_context(task, &flags);
	if (ctx) {
		++ctx->pin_count;
		get_ctx(ctx);
		spin_unlock_irqrestore(&ctx->lock, flags);
	}
	return ctx;
}

Is buggy because perf_lock_task_context() can return a dead context.

the RCU read lock in perf_lock_task_context() only guarantees the memory
won't get freed, it doesn't guarantee the object is valid (in our case
refcount > 0).

Therefore we can return a locked object that can get freed the moment we
release the rcu read lock.

perf_pin_task_context() then increases the refcount and does an unlock
on freed memory.

That increased refcount will cause a double free, in case it started out
with 0.



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/