Date:	Wed, 27 Oct 2010 04:05:16 +0200
From:	"Amir G." <amir73il@...rs.sourceforge.net>
To:	Greg Freemyer <greg.freemyer@...il.com>
Cc:	Ext4 Developers List <linux-ext4@...r.kernel.org>,
	next3-devel@...ts.sourceforge.net
Subject: Re: [RFC] Ext4 snapshots design challenges

On Wed, Oct 27, 2010 at 2:13 AM, Greg Freemyer <greg.freemyer@...il.com> wrote:
> On Mon, Oct 25, 2010 at 12:05 PM, Amir G.
> <amir73il@...rs.sourceforge.net> wrote:
>> On Mon, Oct 25, 2010 at 5:24 PM, Greg Freemyer <greg.freemyer@...il.com> wrote:
>>> Amir,
>>>
>>> I recently saw an announcement for X-Ways Forensics
>>> (http://www.x-ways.net/) that they now support next3 as a filesystem
>>> to analyze.  See Oct. 10 msg under topic "Announcements: X-Ways
>>> Forensics 15.8" at http://www.winhex.net/  (I think that is a public
>>> posting board.)
>>>
>>> I was surprised to see that, but assuming it was indeed your project
>>> they added support for, I congratulate you on the above.
>>>
>>
>> Thanks! I guess :-)
>> I am pretty clueless with regards to the big players in the storage market.
>> I do not know X-Ways, but it looks like they are a big player.
>
>
> X-Ways is a computer forensic tool.  It is used to find evidence on
> computers.  (You might want to check my sig below.)  X-Ways is one of
> the 3 biggest forensic suite vendors and their forensic app sells for
> about $1K.  (My company has 3 licenses.)
>
> A perfect situation for analysis of a next3 based filesystem would be
> if a contract had been fraudulently updated after it was signed and
> X-Ways was able to pull up older versions of the contract and prove
> the fraud.
>
> The fact that they took the time to recover documents out of a next3
> filesystem implies they thought next3 was deployed widely enough to be
> worth the effort.
>
> I know they also add features for specific large customers, so it
> could simply be that a large client of their's asked them to add next3
> support for some internal reason.
>

That's very interesting. I sure hope that next3 (or better yet ext4 snapshots)
will be widely deployed, but I am guessing that X-Ways are trying to
stay in sync
with latest libext2, so when Ted accepted the on-disk format changes to libext2
a few months ago, they must have updated their library as well.

>>> I'm curious what level of support they offer.  In particular, they
>>> only offer limited support for NTFS shadow copies, so I'm curious if
>>> the next3 support is similarly limited.
>>>
>>> Or since next3 is GPL they may have been able to do a more
>>> comprehensive job with it than with ntfs shadow copies.
>>>
>>> Any info you have would be appreciated.
>>> Greg
>>>
>>
>> As you can figure out, I was not involved or notified about this move.
>> Judging from their release notes, I would say that the added support is
>> mostly adding some information tags and verifying the correctness of the
>> exclude bitmap:
>>
>> * Support for the Linux file system next3. The exclude bitmap inode
>> will be evaluated,
>>  and snapshot files are marked with (SF) in the Attribute column.
>>  Specialist license or higher required.
>
> But the ability to pull out snapshot files in an orderly fashion is
> the core functionality they could add from their perspective.  So
> while you may think this is basic, it means they took the time to
> decode your filesystem structure and pull out snapshot files.  Since
> they don't actually use any of the GPL code (or at least I hope they
> don't, that means they had to develop the fs analyser just for next3.
> Not something I suspect can be done with limited effort.
>

The changes that next3 made to on-disk format of ext3 are minor:
http://sourceforge.net/apps/mediawiki/next3/index.php?title=On-disk_format
(and have already been pushed to mainline)

So if you have a code that decodes ext3 structures, be it GPL or not,
the effort required to decode next3 is very limited and it looks to me like
they have only invested that limited effort so far.

However, if any of you forensic developers out there hears me,
you should know that extracting a full snapshot image, or a snapshot
files report,
should be a trivial task if you have all the snapshot file structures decoded.

I was planning to implement something like e2image -r /dev/sda1@1,
but I am probably not going to get around to that in the near future.

> They do the same for NTFS shadow volumes, but even now the
> functionality is not complete enough they call it supported.
>
>> You shouldn't be too surprised to learn that the only file system
>> integrity test that
>> I have added in my e2fsprogs patches is verifying the correctness of
>> the exclude bitmap ;-)
>>
>> Thanks for the info and sorry if your post was rejected from next3-devel.
>> I fixed the permissions for out of list posts.
>
> No problem
>
>> Amir.
>>
>
> Greg
> --
> Greg Freemyer
> Head of EDD Tape Extraction and Processing team
> Litigation Triage Solutions Specialist
> http://www.linkedin.com/in/gregfreemyer
> CNN/TruTV Aired Forensic Imaging Demo -
>    http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retrieved/
>
> The Norcross Group
> The Intersection of Evidence & Technology
> http://www.norcrossgroup.com
>
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists