Message-Id: <201408252344.31771@pali>
Date:	Mon, 25 Aug 2014 23:44:31 +0200
From:	Pali Rohár <pali.rohar@...il.com>
To:	Austin S Hemmelgarn <ahferroin7@...il.com>
Cc:	Jan Kara <jack@...e.cz>, linux-kernel@...r.kernel.org
Subject: Re: Linux UDF support

Hi,

On Monday 25 August 2014 16:05:27 Austin S Hemmelgarn wrote:
> On 2014-08-25 09:24, Pali Rohár wrote:
> > Hi,
> > 
> > On Monday 25 August 2014 14:45:13 Austin S Hemmelgarn wrote:
> >> On 2014-08-24 08:46, Pali Rohár wrote:
> >>> Hi,
> >>> 
> >>> I would like to know what is state of linux UDF driver. It
> >>> is experimental or is now suitable for storing data?
> >> 
> >> I know that read support works for every version I have
> >> tested, but I've only tested it reading data from DVD's and
> >> Blu-Ray discs, so I don't know how well it works for other
> >> purposes.
> > 
> > Ok. I'm thinking about using UDF on HDD and usb flash disks
> > (not on optical medium). And here I need write support too.
> > 
> >>> According to wikipedia [1] UDF has open specification
> >>> format and can be used also for HDDs (not only optical
> >>> discs).
> >>> 
> >>> In OS support table is written that all major and other
> >>> minor OSs support UDF FS (without needs for additional
> >>> programs).
> >>> 
> >>> So it looks like UDF is good candidate for multi OS
> >>> filesystem. Are there any disadvantages for using UDF on
> >>> e.g USB flash disk? (when I want read/write support on
> >>> Linux, Windows 7 and Mac OS X)
> >> 
> >> If you are going to go that way, make sure to use the
> >> Spared Build, as otherwise you will run in to the same
> >> media wear-out issues that NTFS and FAT have.  Also, keep
> >> in mind that pre-Vista Windows and pre-10.4 OSX don't have
> >> very good support for the newer formats.
> > 
> > What is Spared Build? And how to use it?
> 
> Sparse Build is one of the extensions that was added in 1.50. 
> It provides a table indicating sectors that have worn out and
> therefore should be left unused.  The general idea is that
> rewritable media is almost always limited in the number of
> writes it can handle to a given location, and when you exceed
> that number, you get data corruption at that location.  This
> is usually most noticeable on flash media, but it happens on
> {CD,DVD,BD}-RW discs and hard drives as well.  FAT has no
> provisions for handling this, and NTFS and ext4 just return
> the corrupted data.  UDF however, has sufficient error
> correction ability (because it was designed with optical
> media in mind) that it can usually detect the corruption,
> recover the corrupted data, and then if you are using the
> Sparse Build format, mark that block as unusable, and write
> the new data out elsewhere.
> 

Looks good. But I think that when I reformat disk I will lost all 
these data about detected corruption... Or will duplicate mkudffs 
calls preserve these information about HW errors?

So if I format block device (hdd) with mkudffs and rev 0x0201 it 
is automatically enabled? Or is something more needed?

> > Problem with NTFS is that linux driver has write support
> > marked as experimental. FAT has problems with big files and
> > for exFAT there is no driver in linux kernel yet...
> 
> For NTFS, you should be using NTFS-3G, not the kernel driver,
> and I personally wouldn't trust the OS X driver for NTFS for
> anything beyond read-only usage (I only barely trust NTFS-3G
> as it is, and have about as much trust in the Linux kernel
> driver for it as I would in somebody trying to convince me
> that arsenic is good for you).  And for exFAT there is FUSE
> module.  I haven't tested either for much other than
> read-only scenarios, but I can confirm that they are great
> for data recovery.
> 

I'm not sure if using fuse for anything other then experiments is 
reasonable. And speed of fuse (ntfs-3g or exfat) is also 
horrible. And now I think that only usage is read-only data 
recovery...

So still would like to hear what is state of udf read/write 
support in kernel (for usb/flash/hdd disks).

> >>> Because lot of manuals say that FAT32 (or NTFS) is only
> >>> one solution for using USB flash disk on more OS.
> >>> 
> >>> On wikipedia there is one note about linux: Write support
> >>> is only up to UDF version 2.01. Is this restriction still
> >>> valid?
> >> 
> >> I do know that we support reading UDF 2.60 (I've used linux
> >> to read Blu-Ray discs), but I have no idea about write
> >> support for versions above 2.01.
> >> 
> >>> What will happen if I try to mount FS with UDF version
> >>> 2.60 in R/W mode on linux? It will fallback to R/O mode?
> >>> Or newly written files will be in previous (2.01)
> >>> versions?
> >>> 
> >>> And last question: Is there some fsck tool for UDF? Or at
> >>> least tool which print if FS is in inconsistent state?
> >> 
> >> Most Linux distributions have a package called udftools,
> >> the upstream URL given by portage is
> >> http://sf.net/projects/linux-udf/
> > 
> > That project does not have udf fsck tool :-(
> 
> IIRC, there are a lot of issues that UDF will recover from
> gracefully. It has ridiculously good error correction
> abilities, and the newer versions even have the option of
> duplicate copies of filesystem metadata to provide even more
> resilience.
> 

But this depends on driver implementation, rigth? Is linux kernel 
driver able to recovery and fix (on the fly) errors at mount time 
(like ubifs or ext4)? Somebody said that ubifs does not need fsck 
because kernel driver doing similar job of recovery at mount 
time... It is same for udf implementation in linux?

> >>> [1] - http://en.wikipedia.org/wiki/Universal_Disk_Format
> > 
> > Ok, I will wait for response from maintainer Jan, he
> > probably would know more...

-- 
Pali Rohár
pali.rohar@...il.com

Download attachment "signature.asc " of type "application/pgp-signature" (199 bytes)

Powered by blists - more mailing lists