linux-kernel - Re: UBI fastmap updates

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1343929200.25013.197.camel@sauron.fi.intel.com>
Date:	Thu, 02 Aug 2012 20:40:00 +0300
From:	Artem Bityutskiy <artem.bityutskiy@...ux.intel.com>
To:	Tim Bird <tim.bird@...sony.com>
Cc:	Richard Weinberger <richard@....at>,
	"linux-mtd@...ts.infradead.org" <linux-mtd@...ts.infradead.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"adrian.hunter@...el.com" <adrian.hunter@...el.com>,
	"Heinz.Egger@...utronix.de" <Heinz.Egger@...utronix.de>,
	"thomas.wucher@...utronix.de" <thomas.wucher@...utronix.de>,
	"shmulik.ladkani@...il.com" <shmulik.ladkani@...il.com>,
	"tglx@...utronix.de" <tglx@...utronix.de>,
	"Marius.Mazarel@...l.ro" <Marius.Mazarel@...l.ro>,
	"nyoushchenko@...sta.com" <nyoushchenko@...sta.com>
Subject: Re: UBI fastmap updates

Hi Tim,

On Thu, 2012-08-02 at 10:03 -0700, Tim Bird wrote:
> I'm don't understand what "UBI liability" is.  Can you please clarify?
> What breaks if the PEBs get consumed?

let me try. Let's forget about bad blocks and assume we are talking
about NOR flash. For simplicity.

Let's also first forget about fastmap so far and talk about the current
design.

Suppose you have 100 PEBs on your flash. Suppose UBI reserves 10 for its
internal needs (volume table, etc). 90 PEBs are available to the user.

User now can create one or many volumes, but the overall size of the
volumes cannot be larger than 90 LEBs.

This means that UBI guarantees that you can always fill all volumes with
data there will always be enough PEBs to map to. This is UBI liability.

UBI will not allow you to create a volume of 100 LEBs because in this
case it will not be able to guarantee that all LEBs will be writable.

I have invented this "liability" term on the spot actually. It basically
means what UBI already "promised", what it reserved an put aside.

Now let's add fastmap support to the picture.

Suppose fastmap took another 10 PEBs and now we have 80 PEBs for the
user.

The user can create a volume of 80 LEBs in size. And UBI has to
guarantee that the user can at any point of time fill all of them with
data.

This means that fastmap in no circumstances can grab any more than 10
PEBs, because they are all reserved, UBI liability is 80 PEBs.

On other words, fastmap has to know how much PEBs it needs at the UBI
initialization time, and reserve them. The _maximum_ value.

The same way other UBI sub-systems do. E.g., the volume table code
reserves 2 PEBs, because this is the maximum it needs at any point of
time. The WL subsystem reserves 1 PEB.

Of course this is not about reserving any specific PEB, this is just
accounting - we have a couple of variable for reserved PEBs count.

So let's return to the error messages I spotted. They say that fastmap
needs a PEB but cannot find one. The flash is nandsim and has no
badblocks. Why fastmap did not find one? Because it did not reserve
enough. And UBI tests create volumes of maximum possible size and fill
it with data, so all available PEBs are mapped and thus, used.

What this means that the following situation is possible: the UBI volume
is not fully filled yet and not all LEBs are mapped, so there are
available PEBs, and fastmap successfully grows and reduces the amount of
available PEBs. And when the user writes more data, he gets -ENOSPC.

And this is basically the problem I indicated.

To make my description complete, let's add NAND to the picture. We
simply reserve 2% (by default, it is configurable) of PEBs for bad
blocks handling. This is because vendors typically say that this is the
maximum amount for the flash life-time.

If NAND wears-out a lot, and we run out of reserved PEBs, we switch to
R/O mode, because we cannot anymore keep our "promise", the liability.

You can look at it this way. If you create an UBI volume, and mount it
with UBIFS, you usually expect that all the free file-system space is
available to you.

You probably will be disappointed if you write your file and get -ENOSPC
because fastmap does grew and consumed a PEB which which was promised to
your volume.

-- 
Best Regards,
Artem Bityutskiy

Download attachment "signature.asc" of type "application/pgp-signature" (837 bytes)