Message-ID: <5523EFA8.1090105@huawei.com>
Date:	Tue, 7 Apr 2015 22:54:32 +0800
From:	leizhen <thunder.leizhen@...wei.com>
To:	Marc Zyngier <marc.zyngier@....com>
CC:	Thomas Gleixner <tglx@...utronix.de>,
	Jason Cooper <jason@...edaemon.net>,
	linux-kernel <linux-kernel@...r.kernel.org>,
	Zefan Li <lizefan@...wei.com>,
	"huxinwei@...wei.com" <huxinwei@...wei.com>,
	Tianhong Ding <dingtianhong@...wei.com>,
	Kefeng Wang <wangkefeng.wang@...wei.com>,
	Yun Wu <wuyun.wu@...wei.com>
Subject: Re: [PATCH 1/3] irqchip/gicv3-its: Adjust the implementation of its_alloc_tables

On 2015/4/7 21:02, Marc Zyngier wrote:
> On 07/04/15 13:32, leizhen wrote:
>> On 2015/4/7 17:33, Marc Zyngier wrote:
>>> On Tue, 7 Apr 2015 08:47:32 +0100
>>> Zhen Lei <thunder.leizhen@...wei.com> wrote:
>>>
>>>> For the old version, there maybe some faults:
>>>> 1. For each Interrupt Collection table, 4K size is enough. But now the
>>>>    default is 64K(if "Page Size" field is not read-only).
>>>
>>> Why is that a problem?
>>
>> psz is initialized to SZ_64K, and it can only be changed in below branch:
>> 		if ((val ^ tmp) & GITS_BASER_PAGE_SIZE_MASK) {
>> 			/*
>> 			 * Page size didn't stick. Let's try a smaller
>> 			 * size and retry. If we reach 4K, then
>> 			 * something is horribly wrong...
>> 			 */
>> 			switch (psz) {
>> 			case SZ_16K:
>> 				psz = SZ_4K;
>> 				goto retry_baser;
>> 			case SZ_64K:
>> 				psz = SZ_16K;
>> 				goto retry_baser;
>> 			}
>> 		}
>>
>> The condition enter this branch is that "Page Size" field is read-only. If
>> all GITS_BASERn."Page Size" field are writeable, psz is always SZ_64K. That means,
>> all "Virtual CPUs" and "Interrupt Collections" tables will allocte 64K memory.
> 
> At that stage, I don't think it matters (if you want to save memory,
> have a look at how we allocate the device table...). At boot time,
> getting 64k is cheap, and it doesn't really shows up on a multi-GB machine.

No, No. Base on "Page Size" field descripton, each table is at least need 4K(4K,16K,64K:
we have no other choice).

Er, The maximum entry-size is 32(2^5), 4K page memory can at least support 128(minus one)
cores. So, 4K is enough now. Yes, if there too many cores(physical or virtual) exist, we
should allocate memory like device table. We simply allocate one page now, becuause we also
allocate memory for "Virtual CPUs" table.

BTW. Do you need to allocate memory for "Virtual CPUs" table? or leave it to Virtualization?
We really don't known how many "Virtual CPUs" will be, it's not controlled by host OS.

If we have no need to consider "Virtual CPUs" table, we can allocate memory like device table.
And the code will become robust.

> 
>>>
>>>> 2. Suppose two read-only "Page Size" table exist, and software found 4K
>>>>    first, then 16K. For the first time, the value of local variable "psz"
>>>>    will drop to 4K, but we have not reinitialize it again in the loop.
>>>>    So, further process 16K read-only "Page Size" will report error.
>>>>
>>>> In this patch, detect all the read-only fields first. So that, no need
>>>> to try over and over again.
>>>
>>> Frankly, if such HW has been actually designed, the implementor needs
>>> to be £$%^&%$"£$% pretty heavily, because this doesn't make much
>>> sense, from a HW or SW point of view: Either you support
>>> multiple page sizes, or you don't -  you don't do it partially.
>>>
>>> Now, do you know of any HW in the wild that is that crazy? Because I'm
>>
>> I really don't know. I just found this by code review. If SW doesn't like
>> HW to be implemented like that, we should comment it clearly. Otherwise,
>> the reader maybe confused, like me.
>>
>> As you see. The code is not hard to write, and not hard to understand. If we can
>> easily tolerate all of the possible, why we not do it?
> 
> Because carrying code paths that don't get exercised is a maintenance
> burden and a source of bugs. If you want to add a comment that describes
> precisely the configurations we support, feel free to summit one.

Oh, I give up add the comment. My English is poor, it's horrible to describe a lot
of words. And as you mentioned, it may not make sense.

> 
> But I don't want to add more complexity to support configurations that
> don't really make sense from a HW perspective until someone actually (1)
> build such a configuration and (2) tries to get it supported by mainline.
> 
> I understand where you're coming from, and I appreciate your effort to
> make the code more feature-complete. But there is a fine line between
> between supporting the configurations that actually make sense, and
> those that nobody would ever produce.
> 
> So NAK for the "multiple page size" part of the patch. The first point

OK.

> you raise is more interesting, and I wouldn't mind a patch addressing that.
> 
> Thanks,
> 
> 	M.
> 
>>> not eager to have that kind of approach just because someone *could*
>>> implement something silly. From previous discussions with Abel, I
>>> gathered that the Huawei ITS is hardwired to 16k, which lead to commit
>>> 790b57aed156d.
>>>
>>> Thanks,
>>>
>>> 	M.
>>>
>>>> Signed-off-by: Zhen Lei <thunder.leizhen@...wei.com>
>>>> ---
>>>>  drivers/irqchip/irq-gic-v3-its.c | 105 +++++++++++++++++++++++----------------
>>>>  1 file changed, 62 insertions(+), 43 deletions(-)
>>>>
>>>> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
>>>> index 9687f8a..2577f06 100644
>>>> --- a/drivers/irqchip/irq-gic-v3-its.c
>>>> +++ b/drivers/irqchip/irq-gic-v3-its.c
>>>> @@ -800,20 +800,21 @@ static int its_alloc_tables(struct its_node *its)
>>>>  {
>>>>  	int err;
>>>>  	int i;
>>>> -	int psz = SZ_64K;
>>>> -	u64 shr = GITS_BASER_InnerShareable;
>>>> -	u64 cache = GITS_BASER_WaWb;
>>>> +	int psz;
>>>> +	u64 shr;
>>>> +	u64 cache;
>>>>
>>>>  	for (i = 0; i < GITS_BASER_NR_REGS; i++) {
>>>>  		u64 val = readq_relaxed(its->base + GITS_BASER + i * 8);
>>>>  		u64 type = GITS_BASER_TYPE(val);
>>>>  		u64 entry_size = GITS_BASER_ENTRY_SIZE(val);
>>>> -		int order = get_order(psz);
>>>> +		int order;
>>>>  		int alloc_size;
>>>>  		u64 tmp;
>>>>  		void *base;
>>>>
>>>> -		if (type == GITS_BASER_TYPE_NONE)
>>>> +		switch (type) {
>>>> +		case GITS_BASER_TYPE_NONE:
>>>>  			continue;
>>>>
>>>>  		/*
>>>> @@ -824,16 +825,65 @@ static int its_alloc_tables(struct its_node *its)
>>>>  		 *
>>>>  		 * For other tables, only allocate a single page.
>>>>  		 */
>>>> -		if (type == GITS_BASER_TYPE_DEVICE) {
>>>> +		case GITS_BASER_TYPE_DEVICE: {
>>>>  			u64 typer = readq_relaxed(its->base + GITS_TYPER);
>>>>  			u32 ids = GITS_TYPER_DEVBITS(typer);
>>>>
>>>> -			order = get_order((1UL << ids) * entry_size);
>>>> -			if (order >= MAX_ORDER) {
>>>> -				order = MAX_ORDER - 1;
>>>> -				pr_warn("%s: Device Table too large, reduce its page order to %u\n",
>>>> -					its->msi_chip.of_node->full_name, order);
>>>> +			alloc_size = (1UL << ids) * entry_size;
>>>> +			break;
>>>>  			}
>>>> +
>>>> +		default:
>>>> +			alloc_size = PAGE_SIZE;
>>>> +			break;
>>>> +		}
>>>> +
>>>> +		psz = PAGE_SIZE;
>>>> +		shr = GITS_BASER_InnerShareable;
>>>> +		cache = GITS_BASER_WaWb;
>>>> +
>>>> +		/* Test the fields that maybe read-only */
>>>> +		tmp = shr | (~val & GITS_BASER_PAGE_SIZE_MASK);
>>>> +		writeq_relaxed(tmp, its->base + GITS_BASER + i * 8);
>>>> +		tmp = readq_relaxed(its->base + GITS_BASER + i * 8);
>>>> +
>>>> +		/*
>>>> +		 * Shareability didn't stick. Just use
>>>> +		 * whatever the read reported, which is likely
>>>> +		 * to be the only thing this redistributor
>>>> +		 * supports. If that's zero, make it
>>>> +		 * non-cacheable as well.
>>>> +		 */
>>>> +		shr = tmp & GITS_BASER_SHAREABILITY_MASK;
>>>> +		if (!shr)
>>>> +			cache = GITS_BASER_nC;
>>>> +
>>>> +		/*
>>>> +		 * "Page Size" field is read-only. Should ensure the table
>>>> +		 * start address and size align to it.
>>>> +		 */
>>>> +		if (!((val ^ tmp) & GITS_BASER_PAGE_SIZE_MASK))
>>>> +			switch (val & GITS_BASER_PAGE_SIZE_MASK) {
>>>> +			case GITS_BASER_PAGE_SIZE_4K:
>>>> +				psz = SZ_4K;
>>>> +				break;
>>>> +
>>>> +			case GITS_BASER_PAGE_SIZE_16K:
>>>> +				psz = SZ_16K;
>>>> +				break;
>>>> +
>>>> +			default:
>>>> +				psz = SZ_64K;
>>>> +				break;
>>>> +			}
>>>> +
>>>> +		alloc_size = ALIGN(alloc_size, psz);
>>>> +
>>>> +		order = get_order(alloc_size);
>>>> +		if (order >= MAX_ORDER) {
>>>> +			order = MAX_ORDER - 1;
>>>> +			pr_warn("%s: Device Table too large, reduce its page order to %u\n",
>>>> +				its->msi_chip.of_node->full_name, order);
>>>>  		}
>>>>
>>>>  		alloc_size = (1 << order) * PAGE_SIZE;
>>>> @@ -845,7 +895,6 @@ static int its_alloc_tables(struct its_node *its)
>>>>
>>>>  		its->tables[i] = base;
>>>>
>>>> -retry_baser:
>>>>  		val = (virt_to_phys(base) 				 |
>>>>  		       (type << GITS_BASER_TYPE_SHIFT)			 |
>>>>  		       ((entry_size - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>>>> @@ -870,40 +919,10 @@ retry_baser:
>>>>  		writeq_relaxed(val, its->base + GITS_BASER + i * 8);
>>>>  		tmp = readq_relaxed(its->base + GITS_BASER + i * 8);
>>>>
>>>> -		if ((val ^ tmp) & GITS_BASER_SHAREABILITY_MASK) {
>>>> -			/*
>>>> -			 * Shareability didn't stick. Just use
>>>> -			 * whatever the read reported, which is likely
>>>> -			 * to be the only thing this redistributor
>>>> -			 * supports. If that's zero, make it
>>>> -			 * non-cacheable as well.
>>>> -			 */
>>>> -			shr = tmp & GITS_BASER_SHAREABILITY_MASK;
>>>> -			if (!shr)
>>>> -				cache = GITS_BASER_nC;
>>>> -			goto retry_baser;
>>>> -		}
>>>> -
>>>> -		if ((val ^ tmp) & GITS_BASER_PAGE_SIZE_MASK) {
>>>> -			/*
>>>> -			 * Page size didn't stick. Let's try a smaller
>>>> -			 * size and retry. If we reach 4K, then
>>>> -			 * something is horribly wrong...
>>>> -			 */
>>>> -			switch (psz) {
>>>> -			case SZ_16K:
>>>> -				psz = SZ_4K;
>>>> -				goto retry_baser;
>>>> -			case SZ_64K:
>>>> -				psz = SZ_16K;
>>>> -				goto retry_baser;
>>>> -			}
>>>> -		}
>>>> -
>>>>  		if (val != tmp) {
>>>>  			pr_err("ITS: %s: GITS_BASER%d doesn't stick: %lx %lx\n",
>>>>  			       its->msi_chip.of_node->full_name, i,
>>>> -			       (unsigned long) val, (unsigned long) tmp);
>>>> +			       (unsigned long)val, (unsigned long)tmp);
>>>>  			err = -ENXIO;
>>>>  			goto out_free;
>>>>  		}
>>>> --
>>>> 1.8.0
>>>>
>>>>
>>>
>>>
>>>
>>
>>
> 
> 


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists