linux-kernel - RE: [PATCH 0/5] cramfs refresh for embedded usage

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 16 Aug 2017 10:29:07 -0400 (EDT)
From:   Nicolas Pitre <nicolas.pitre@...aro.org>
To:     Chris Brandt <Chris.Brandt@...esas.com>
cc:     Alexander Viro <viro@...iv.linux.org.uk>,
        "linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
        "linux-embedded@...r.kernel.org" <linux-embedded@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: RE: [PATCH 0/5] cramfs refresh for embedded usage

On Wed, 16 Aug 2017, Chris Brandt wrote:

> On Wednesday, August 16, 2017, Nicolas Pitre wrote:
> > > Yes, now I can boot with my rootfs being a XIP cramfs.
> > >
> > > However, like you said, libc is not XIP.
> > 
> > I think I have it working now. Probably learned more about the memory
> > management internals than I ever wanted to know. Please try the patch
> > below on top of all the previous ones. If it works for you as well then
> > I'll rebase and repost the whole thing.
> > 
> > diff --git a/fs/cramfs/inode.c b/fs/cramfs/inode.c
> > index 4c7f01fcd2..0b651f985c 100644
> > --- a/fs/cramfs/inode.c
> > +++ b/fs/cramfs/inode.c
> 
> 
> Yes, that worked. Very nice!

Good.

> Just FYI,
> I'm running an xipImage with all the RZ/A1 upstream drivers enabled and 
> only using about 4.5MB of total system RAM.
> That's pretty good. Of course for a real application, you would trim off
> the drivers and subsystems you don't plan on using, thus lowering your
> RAM usage.

On my MMU-less test target I'm going under the 1MB mark now.

> > +/*
> > + * It is possible for cramfs_physmem_mmap() to partially populate the mapping
> > + * causing page faults in the unmapped area. When that happens, we need to
> > + * split the vma so that the unmapped area gets its own vma that can be backed
> > + * with actual memory pages and loaded normally. This is necessary because
> > + * remap_pfn_range() overwrites vma->vm_pgoff with the pfn and filemap_fault()
> > + * no longer works with it. Furthermore this makes /proc/x/maps right.
> > + * Q: is there a way to do split vma at mmap() time?
> > + */
> 
> So if I understand correctly, the issue is that sometimes you only have 
> a partial PAGE worth that you need to map. Correct?

Yes, or the page is stored in its compressed form in the filesystem, or 
it is misaligned, or any combination of those.

> For the AXFS file system, XIP page mapping was done on a per page 
> decision, not per file. So the mkfs.axfs tool would only mark a page as XIP if
> the entire section would fit in a complete PAGE. If for example you had
> a partial page at the end of a multi page code segment, it would put 
> that partial page in a separate portion of the AXFS image and be marked as
> 'copy to RAM' instead of being marked as 'map as XIP'.
> So in the AXFS case, it was a combination of the creation tool and file 
> system driver features to fix the partial page issue.
> Not sure if any of this info is relevant, but I thought I would mention 
> anyway.

Same applies here.  The XIP decision is no longer a per file thing. This 
is why mkcramfs puts loadable and read-only ELF segments into 
uncompressed and aligned blocks while still packing the remaining of the 
file. The partial page issue can be "fixed" within mkcramfs if 
considered worth it. To incure the page alignment overhead only once, 
all the uncompressed blocks can be located together away from their file 
block tables, etc.  The extended format implemented in this seris allows 
for all this layout flexibility the fs creation tool may exploit.

The current restriction in the fs driver at the moment is that XIP 
blocks must be contiguous in the filesystem. That is a hard requirement 
in the non-mmu case anyway.

Given that I also applied the device table patch to mkcramfs (that 
allows for the creation of device nodes and arbitrary 
user/group/permission without being root) it would be possible to extend 
this mechanism to implement other XIP patterns such as for 
uncompressible media files for example.

> Thank you for your efforts on adding XIP to cramfs!

Thank you for testing.

Nicolas