Message-ID: <AANLkTim9EOUVuSq6Z3q6-j-x2w+JDD8F-_xH614uDFi8@mail.gmail.com>
Date:	Mon, 6 Sep 2010 11:24:51 -0600
From:	Cam Macdonell <cam@...ualberta.ca>
To:	Avi Kivity <avi@...hat.com>
Cc:	"qemu-devel@...gnu.org Developers" <qemu-devel@...gnu.org>,
	linux-kernel <linux-kernel@...r.kernel.org>
Subject: Re: [Qemu-devel] Guest cannot handle a PCI BAR > 1GB

On Mon, Sep 6, 2010 at 10:37 AM, Cam Macdonell <cam@...ualberta.ca> wrote:
>
>
> On Sun, Sep 5, 2010 at 10:50 AM, Avi Kivity <avi@...hat.com> wrote:
>>
>>  On 09/04/2010 01:22 AM, Cam Macdonell wrote:
>>>
>>> Hi,
>>>
>>> I'm trying to test 2 GB (and eventually larger) BARs with ivshmem and
>>> I get an error in the guest that it is able to find a mem resource for
>>> a BAR larger than 1GB.  I'm using 64-bit BARs.
>>>
>>> when running with 6GB of RAM and a 1GB BAR for ivshmem, it finds a
>>> resource (and searches beyond 32-bit values to find it).  Here is a
>>> log from printfs added to the loop that searches the resources from
>>> find_resource() in kernel/resource.c:363.
>>>
>>
>> This is a kernel question, not a qemu issue.  Copying lkml.
>>
>>> trying 'tmp.start' 1000 to
>>>         'tmp.end' fff
>>> trying 'tmp.start' 9f400 to
>>>         'tmp.end' 9f3ff
>>> trying 'tmp.start' a0000 to
>>>         'tmp.end' effff
>>> trying 'tmp.start' 100000 to
>>>         'tmp.end' fffff
>>> trying 'tmp.start' dfffd000 to
>>>         'tmp.end' dfffcfff
>>> trying 'tmp.start' e0000000 to
>>>         'tmp.end' efffffff
>>> trying 'tmp.start' f2000000 to
>>>         'tmp.end' f1ffffff
>>> trying 'tmp.start' f2001000 to
>>>         'tmp.end' f200ffff
>>> trying 'tmp.start' f2020000 to
>>>         'tmp.end' f201ffff
>>> trying 'tmp.start' f2021000 to
>>>         'tmp.end' f202ffff
>>> trying 'tmp.start' f2040000 to
>>>         'tmp.end' f203ffff
>>> trying 'tmp.start' f2040100 to
>>>         'tmp.end' febfffff
>>> trying 'tmp.start' fec00400 to
>>>         'tmp.end' fffbffff
>>> trying 'tmp.start' 100000000 to
>>>         'tmp.end' ffffffff
>>> trying 'tmp.start' 1a0000000 to
>>>         'tmp.end' ffffffffffffffff
>>> pci 0000:00:04.0: BAR 2: assigned [mem 0x1c0000000-0x1ffffffff 64bit]
>>> pci 0000:00:04.0: BAR 2: set to [mem 0x1c0000000-0x1ffffffff 64bit]
>>> (PCI address [0x1c0000000-0x1ffffffff]
>>>
>>> and you can see the BAR is successfully assigned.
>>>
>>> However, with a 2GB BAR (below), the search fails, but it also never
>>> searches beyong 32-bits.  Again, all that's changed is the size of the
>>> ivshmem region.
>>>
>>> trying 'tmp.start' 1000 to
>>>         'tmp.end' fff
>>> trying 'tmp.start' 9f400 to
>>>         'tmp.end' 9f3ff
>>> trying 'tmp.start' a0000 to
>>>         'tmp.end' effff
>>> trying 'tmp.start' 100000 to
>>>         'tmp.end' fffff
>>> trying 'tmp.start' dfffd000 to
>>>         'tmp.end' dfffcfff
>>> pci 0000:00:04.0: BAR 2: can't assign mem (size 0x80000000)
>>>
>>> Is there a limit to PCI BAR sizes or resources?  Any pointers or
>>> further debugging tips are greatly appreciated.
>>>
>>
>> What kernel version are you looking at?
>
> latest kvm git, 2.6.36-rc2+
>
>>
>> Please add printks to the loop so we can see this->start and this->end.  It smells like a truncation issue.
>
> Success with a 1GB BAR
> this->start 1000, this->end 9f3ff
> this->start 9f400, this->end 9ffff
> this->start f0000, this->end fffff
> this->start 100000, this->end dfffcfff
> this->start dfffd000, this->end dfffffff
> this->start f0000000, this->end f1ffffff
> this->start f2000000, this->end f2000fff
> this->start f2010000, this->end f201ffff
> this->start f2020000, this->end f2020fff
> this->start f2030000, this->end f203ffff
> this->start f2040000, this->end f20400ff
> this->start fec00000, this->end fec003ff
> this->start fffc0000, this->end ffffffff
> this->start 100000000, this->end 11fffffff
> tmp.start 120000000, tmp.end ffffffffffffffff
> pci 0000:00:04.0: BAR 2: assigned [mem 0x140000000-0x17fffffff 64bit]
> and when it fails with a 2GB BAR, the following is printed
> this->start 1000, this->end 9f3ff
> this->start 9f400, this->end 9ffff
> this->start f0000, this->end fffff
> this->start 100000, this->end dfffcfff
> this->start dfffd000, this->end dfffffff
> pci 0000:00:04.0: BAR 2: can't assign mem (size 0x80000000)
> I added a few more debug statements and found that in the failure case, the function returns that it found a region (the last one printed before the error).  I've added printfs for the two tests in the if that determine when a region is found:
>        if (tmp.start < tmp.end && tmp.end - tmp.start >= size - 1) {
>             new->start = tmp.start;
>             new->end = tmp.start + size - 1;
>             printk(KERN_INFO "returning 0\n");
>             return 0;
>         }
> this->start 1000, this->end 9f3ff
> tmp.start 80000000, tmp.end fff
>     true: ffffffff80000fff >= 7fffffff
> this->start 9f400, this->end 9ffff
> tmp.start 80000000, tmp.end 9f3ff
>     true: ffffffff8009f3ff >= 7fffffff
> this->start f0000, this->end fffff
> tmp.start 80000000, tmp.end effff
>     true: ffffffff800effff >= 7fffffff
> this->start 100000, this->end dfffcfff
> tmp.start 80000000, tmp.end fffff
>     true: ffffffff800fffff >= 7fffffff
> this->start dfffd000, this->end dfffffff
> tmp.start 100000, tmp.end dfffcfff
>     true: 100000 < dfffcfff
>     true: dfefcfff >= 7fffffff
> returning 0
> pci 0000:00:04.0: BAR 2: can't assign mem (size 0x80000000)

Further to this, it seems tmp.start is getting set to zero by the ALIGN macro

2GB BAR:
this->start dfffd000, this->end dfffffff
tmp.start dfffd000
tmp.start 0
tmp.start 100000, tmp.end dfffcfff

1GB BAR:
this->start dfffd000, this->end dfffffff
tmp.start dfffd000
tmp.start 100000000
tmp.start 100000000, tmp.end dfffcfff

I'll dig into the ALIGN macro.

>
>
>
>
>>
>> --
>> error compiling committee.c: too many arguments to function
>>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists