linux-kernel - Re: [Bug] Reproducible data corruption on i5-3340M: Please revert 53a59fc67!

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130814174039.GA24033@dhcp22.suse.cz>
Date:	Wed, 14 Aug 2013 19:40:39 +0200
From:	Michal Hocko <mhocko@...e.cz>
To:	Ben Tebulin <tebulin@...glemail.com>
Cc:	mgorman@...e.de, hannes@...xchg.org, bsingharora@...il.com,
	kamezawa.hiroyu@...fujitsu.com, linux-mm@...ck.org,
	Rik van Riel <riel@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [Bug] Reproducible data corruption on i5-3340M: Please revert
 53a59fc67!

[Let's CC some more people]

On Wed 14-08-13 18:36:53, Ben Tebulin wrote:
> Hello Michal, Johannes, Balbir, Kamezawa and Mailing lists!

Hi,

> Since v3.7.2 on two independent machines a very specific Git repository
> fails in 9/10 cases on git-fsck due to an SHA1/memory failures. This
> only occurs on a very specific repository and can be reproduced stably
> on two independent laptops. Git mailing list ran out of ideas and for me 
> this looks like some very exotic kernel issue.
> 
> After a _very long session of rebooting and bisecting_ the Linux kernel
> (fortunately I had a SSD and ccache!) I was able to pinpoint the cause
> to the following patch:
> 
> *"mm: limit mmu_gather batching to fix soft lockups on !CONFIG_PREEMPT"*
>   787f7301074ccd07a3e82236ca41eefd245f4e07 linux stable    [1]
>   53a59fc67f97374758e63a9c785891ec62324c81 upstream commit [2]

Thanks for bisecting this up!

I will look into this but I find it really strange. The patch only
limits the number of batched pages to be freed. This might happen even
without the patch, albeit less likely, when a new batch cannot be
allocated.
That being said, I do not see anything obviously wrong with the patch
itself. Maybe we are not flushing those pages properly in some corner
case which doesn't trigger normally. I will have to look at it but I
really think this just exhibits a subtle bug in batch pages freeing.

I have no objection to revert the patch for now until we find out what
is really going on.

> More details are available in my previous discussion on the Git mailing:
> 
>    http://thread.gmane.org/gmane.comp.version-control.git/231872
> 
> Never had any hardware/stability issues _at all_ with these machines. 
> Only one repo out of 112 is affected. It's a git-svn clone and even 
> recreated copies out of svn do trigger the same failure.
> 
> I was able to bisect this error to this very specific commit. 
> Furthermore: Reverting this commit in 3.9.11 still solves the error. 
> 
> I assume this is a regression of the Linux kernel (not Git) and would 
> kindly ask you to revert the afore mentioned commits.
> 
> Thanks!
> - Ben
> 
> 
> I'm not subscribed - please CC me.
> 
> [1] https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=787f7301074ccd07a3e82236ca41eefd245f4e07
> [2] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=53a59fc67f97374758e63a9c785891ec62324c81
> 

-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/