Message-ID: <f9958544-e9e9-d0f0-ceff-30e73d119bad@linux.alibaba.com>
Date:   Wed, 24 May 2023 19:06:46 +0800
From:   Gao Xiang <hsiangkao@...ux.alibaba.com>
To:     Giuseppe Scrivano <gscrivan@...hat.com>
Cc:     Alexander Larsson <alexl@...hat.com>,
        Mike Snitzer <snitzer@...nel.org>,
        Du Rui <durui@...ux.alibaba.com>, dm-devel@...hat.com,
        linux-kernel@...r.kernel.org, Alasdair Kergon <agk@...hat.com>,
        Aleksa Sarai <cyphar@...har.com>
Subject: Re: dm overlaybd: targets mapping OverlayBD image



On 2023/5/24 03:48, Giuseppe Scrivano wrote:
> Gao Xiang <hsiangkao@...ux.alibaba.com> writes:
> 
>> Hi Giuseppe,
>>
>> On 2023/5/24 01:11, Giuseppe Scrivano wrote:
>>> Gao Xiang <hsiangkao@...ux.alibaba.com> writes:
>>>
>>
>> ...
>>
>>>> Agreed, I hope you guys could actually sit down and evaluate a proper
>>>> solution on the next OCI v2, currently I know there are:
>>>>
>>>>    - Composefs
>>>>    - (e)stargz   https://github.com/containerd/stargz-snapshotter
>>>>    - Nydus       https://github.com/containerd/nydus-snapshotter
>>>>    - OverlayBD   https://github.com/containerd/accelerated-container-image
>>>>    - SOCI        https://github.com/awslabs/soci-snapshotter
>>>>    - Tarfs
>>>>    - (maybe even more..)
>>>>
>>>> Honestly, I do think OSTree/Composefs is the best approach for now for
>>>> deduplication and page cache sharing (due to kernel limitation of page
>>>> cache sharing and overlayfs copyup limitation).  I'm too tired of
>>>> container image stuffs honestly.  Too much unnecessary manpower waste.
>>> for a file-based storage model, I am not sure a new format would
>>> really
>>> buy us much or it can be significantly different.
>>> Without a proper support from the kernel, a new format would still
>>> need
>>> to create the layout overlay expects, so it won't be much different than
>>> what we have now.
>>
>> I've seen lot efforts on this, for example,
>> https://docs.google.com/presentation/d/1lBKVrYzm9JEYuw-gIEsrcePSK0jL1Boe/edit#slide=id.p22
>>
>> Merging the writable layer and read-only layers with overlayfs is
>> feasible. I mean, at least for composefs model on backing XFS/btrfs, we
>> could merge these layers with overlayfs so that I guess reflink could
>> be done to avoid full copyup as well?  I do think that's a net win.
>>
>>> The current OCI format, with some tweaks like (e)stargz or
>>> zstd:chunked,
>>> already make its content addressable and a client can retrieve only the
>>> subset of the files that are needed.  At the same time we maintain the
>>> simplicity of a tarball and it won't break existing clients.
>>
>> (e)stargz or zstd:chunked still needs to be converted by the publisher
>> and not all exist OCI images are stored in this way.  But apart from
>> detailed comparsion, disk mapping image approaches seems really a
>> drawback at least on my side.
> 
> these images can be treated as if all their files are missing and the
> checksum is calculated on the receiver side.  They will still be stored
> locally indexed by their checksum.  We lose the possibility to pull only
> the missing files but we maintain the other advantages at runtime.  In
> this way moving to a new format can be done incrementally without
> breaking what we have now.

Yeah, that is on-demand loading stuffs (another story) but my
opinion was that I could see a win of composefs model is that
you could use EROFS + overlayfs + XFS/btrfs to do partial copyup
by using clone_file_range() to copy up within the same fs (since
all layers including the writable layer are actually landed
in the same fs so overlayfs will just clone_file_range()).

In principle, we could do some hack to do clone_file_range()
across different fses which are actually backed by the same
fs for other approaches, but that approach cannot be not
easily landed upstream TBH.

Thanks,
Gao Xiang

Powered by blists - more mailing lists