linux-kernel - Re: 2.6.21-git10/11: files getting truncated on xfs? or maybe an nlink problem?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20070509215538.GK11115@waste.org>
Date:	Wed, 9 May 2007 16:55:38 -0500
From:	Matt Mackall <mpm@...enic.com>
To:	Jeremy Fitzhardinge <jeremy@...p.org>
Cc:	David Chinner <dgc@....com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: 2.6.21-git10/11: files getting truncated on xfs? or maybe an nlink problem?

On Wed, May 09, 2007 at 02:09:50PM -0700, Jeremy Fitzhardinge wrote:
> I've had a couple of instances of a linux-2.6 mercurial repo getting
> corrupted in some odd way this morning.  It looks like files are being
> truncated; not to size 0, but losing something off the end.
> 
> This is on an xfs filesystem.  I haven't had any crashes/oops, and I
> don't think its the normal files getting filled with 0 problem.  I saw
> this before the most recent set of xfs updates, but it happened again
> afterwards too.
> 
> Mercurial uses a strictly append-only model for updating its repo files,
> but it looks like maybe an append operation didn't stick.

(Unless you're using the mq extension, which regularly truncates
files. But you're definitely the first person to run into this sort of
thing in any case.)

> I'm repulling a fresh copy of the repo; I'll be able to compare
> before/after.  Update: yep, definitely truncated:
> 
> $ ls -l .hg-new/store/data/_documentation/pi-futex.txt.i .hg-broken/store/data/_documentation/pi-futex.txt.i
> 4 -rw-rw-r-- 1 jeremy jeremy 3309 May  9 09:43 .hg-broken/store/data/_documentation/pi-futex.txt.i
> 4 -rw-rw-r-- 1 jeremy jeremy 3797 May  9 13:38 .hg-new/store/data/_documentation/pi-futex.txt.i
> 
> also
>   3476 -rw-rw-r--  1 jeremy jeremy   3558208 May  9 13:55 00manifest.i
>   3476 -rw-rw-r--  1 jeremy jeremy   3555200 May  9 09:41 00manifest.i~
>
> where 00manifest.i~ is the broken one. The files are identical up to the
> truncation point.
> 
> The repo passed "hg verify" just after I pulled it, so this corruption
> came about after a while.
> 
> Hm, the other possibility is that nlinks is being misreported.

I think if the files are identical up to the truncation point, we can
rule out nlink concerns. If for some reason Mercurial's COW logic got
fooled, the result would be a bit of a jumble at the end. And you'd be
unlikely to hit any sort of race as a single user on a laptop.

Can you use hg debugindex to determine if the truncation point
corresponds with a whole delta or whether it's in the middle of a
delta? 

For noninterleaved revlogs like your manifest above, the .i file is an
index build of a collection of 64-byte entries. It's pretty hard to
imagine how you'd lose 8 bytes since all I/O is done in multiples of
64-bytes. Oh, I'm misreading that - they differ by 3008 bytes, which
is 47 * 64. 

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/