Message-ID: <9e13765c-5ea2-6841-0c4b-6fad21134201@xs4all.nl>
Date:   Wed, 6 Sep 2023 11:43:25 +0200
From:   Hans Verkuil <hverkuil@...all.nl>
To:     Tomasz Figa <tfiga@...omium.org>
Cc:     Robin Murphy <robin.murphy@....com>, Anle Pan <anle.pan@....com>,
        Christoph Hellwig <hch@....de>, m.szyprowski@...sung.com,
        mchehab@...nel.org, linux-media@...r.kernel.org,
        linux-kernel@...r.kernel.org, hui.fang@....com
Subject: Re: [PATCH] media: videobuf2-dma-sg: limit the sg segment size

On 06/09/2023 11:26, Tomasz Figa wrote:
> On Wed, Sep 6, 2023 at 5:52 PM Hans Verkuil <hverkuil@...all.nl> wrote:
>>
>> Hi all,
>>
>> On 29/08/2023 13:14, Robin Murphy wrote:
>>> On 2023-08-29 11:03, Tomasz Figa wrote:
>>>> Hi Anle,
>>>>
>>>> On Mon, Aug 28, 2023 at 8:57 AM Anle Pan <anle.pan@....com> wrote:
>>>>>
>>>>> When allocating from pages, the size of the sg segment is unlimited and
>>>>> the default is UINT_MAX. This will cause the DMA stream mapping failed
>>>>> later with a "swiotlb buffer full" error.
>>>>
>>>> Thanks for the patch. Good catch.
>>>>
>>>>> The default maximum mapping
>>>>> size is 128 slots x 2K = 256K, determined by "IO_TLB_SEGSIZE".
>>>>> To fix the issue, limit the sg segment size according to
>>>>> "dma_max_mapping_size" to match the mapping limit.
>>>>>
>>>>> Signed-off-by: Anle Pan <anle.pan@....com>
>>>>> ---
>>>>>   drivers/media/common/videobuf2/videobuf2-dma-sg.c | 9 +++++++--
>>>>>   1 file changed, 7 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
>>>>> index fa69158a65b1..b608a7c5f240 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
>>>>> @@ -105,6 +105,7 @@ static void *vb2_dma_sg_alloc(struct vb2_buffer *vb, struct device *dev,
>>>>>          struct sg_table *sgt;
>>>>>          int ret;
>>>>>          int num_pages;
>>>>> +       size_t max_segment = 0;
>>>>>
>>>>>          if (WARN_ON(!dev) || WARN_ON(!size))
>>>>>                  return ERR_PTR(-EINVAL);
>>>>> @@ -134,8 +135,12 @@ static void *vb2_dma_sg_alloc(struct vb2_buffer *vb, struct device *dev,
>>>>>          if (ret)
>>>>>                  goto fail_pages_alloc;
>>>>>
>>>>> -       ret = sg_alloc_table_from_pages(buf->dma_sgt, buf->pages,
>>>>> -                       buf->num_pages, 0, size, GFP_KERNEL);
>>>>> +       if (dev)
>>>
>>> dev can't be NULL, see the context above.
>>>
>>>>> +               max_segment = dma_max_mapping_size(dev);
>>>>> +       if (max_segment == 0)
>>>>> +               max_segment = UINT_MAX;
>>>>> +       ret = sg_alloc_table_from_pages_segment(buf->dma_sgt, buf->pages,
>>>>> +               buf->num_pages, 0, size, max_segment, GFP_KERNEL);
>>>>
>>>> One thing that I'm not sure about here is that we use
>>>> sg_alloc_table_from_pages_segment(), but we actually don't pass the
>>>> max segment size (as returned by dma_get_max_seg_size()) to it.
>>>> I'm also not exactly sure what's the difference between "max mapping
>>>> size" and "max seg size".
>>>> +Robin Murphy +Christoph Hellwig I think we could benefit from your
>>>> expertise here.
>>>
>>> dma_get_max_seg_size() represents a capability of the device itself, namely the largest contiguous range it can be programmed to access in a single DMA descriptor/register/whatever. Conversely,
>>> dma_max_mapping_size() is a capablity of the DMA API implementation, and represents the largest contiguous mapping it is guaranteed to be able to handle (each segment in the case of dma_map_sg(), or
>>> the whole thing for dma_map_page()). Most likely the thing you want here is min_not_zero(max_seg_size, max_mapping_size).
>>>
>>>> Generally looking at videobuf2-dma-sg, I feel like we would benefit
>>>> from some kind of dma_alloc_table_from_pages() that simply takes the
>>>> struct dev pointer and does everything necessary.
>>>
>>> Possibly; this code already looks lifted from drm_prime_pages_to_sg(), and if it's needed here then presumably vb2_dma_sg_get_userptr() also needs it, at the very least.
>>
>> I also see sg_alloc_table_from_pages being called in vb2_dc_get_userptr in videobuf2-dma-contig.c,
>> presumably that needs the same fix?
>>
>> From what I gather from this thread there are no improved helpers on the immediate
>> horizon, so this issue has to be fixed in videobuf2 for now.
> 
> Agreed. (Although I suspect the real issue that NXP is having isn't
> really this and this is just a side effect.)
> 
>>
>> So this requires a v2 that fixes this also in vb2_dma_sg_get_userptr and vb2_dc_get_userptr,
>> correct? If so, then it would be nice if Anle can post a v2 with those changes.
> 
> If we need to fix 3 different callers, could we at least add an
> internal vb2 helper for this (e.g. vb2_asg_alloc_table_from_pages())?

Definitely.

> 
>>
>> Note that when I grepped for sg_alloc_table_from_pages_segment users, I noticed that
>> in most cases dma_max_mapping_size is used, but in one case it uses dma_get_max_seg_size
>> (drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c). I have no idea what the difference is
>> between the two.
> 
> FWIW, Robin explained that earlier in the thread. To be fully correct,
> we need to consider both...

Ah, I missed that explanation. Yes, it looks like both need to be considered.

> 
>>
>> One small change that would make sg_alloc_table_from_pages_segment() a bit easier
>> to work with is if it would replace a max_segment value of 0 with UINT_MAX. Then
>> you can just stick in dma_max_mapping_size(dev) as the argument.
>>
>> Alternatively, if we can be certain that dma_max_mapping_size(dev) never returns 0,
>> then that 'if (max_segment == 0)' part can just be dropped.
> 
> That's also the reason I wanted a helper. Not sure if it's so easy to
> change the calling convention now.

There are not all that many users of sg_alloc_table_from_pages(_segment),
dma_get_max_seg_size and dma_max_mapping_size, so it doesn't look too difficult
for a sufficiently motivated person (not me!) to make changes here.

Regards,

	Hans

> 
>>
>> I also wonder if any of the other (non-media) users of sg_alloc_table_from_pages
>> would need to use sg_alloc_table_from_pages_segment as well. Is this really
>> media specific? Or is media just the most likely subsystem to hit this issue due
>> to the large amounts of memory it uses?
> 
> I don't think it's media-specific, but also as I said earlier, it
> should be really rare to hit it - swiotlb isn't normally expected for
> reasonably capable systems and buffer allocation being done correctly.
> 
> Best regards,
> Tomasz

Powered by blists - more mailing lists