netdev - Re: Low performance Intel 10GE NIC (3.2.10) on 2.6.38 Kernel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4D9DE465.1080008@intel.com>
Date:	Thu, 07 Apr 2011 09:20:53 -0700
From:	Alexander Duyck <alexander.h.duyck@...el.com>
To:	Eric Dumazet <eric.dumazet@...il.com>
CC:	Wei Gu <wei.gu@...csson.com>, netdev <netdev@...r.kernel.org>,
	"Kirsher, Jeffrey T" <jeffrey.t.kirsher@...el.com>
Subject: Re: Low performance Intel 10GE NIC (3.2.10) on 2.6.38 Kernel

On 4/7/2011 9:03 AM, Eric Dumazet wrote:
> Le jeudi 07 avril 2011 à 08:58 -0700, Alexander Duyck a écrit :
>> On 4/7/2011 4:46 AM, Eric Dumazet wrote:
>>> Le jeudi 07 avril 2011 à 19:15 +0800, Wei Gu a écrit :
>>>> Hi,
>>>> I compile the ixgbe driver into the kernel and run the test again and also change the copy to clone in the fw hook
>>>> This is the perf report while I was forwarding 150Kpps with
>>>> The attached file include the basic info about my test system. Please let me know if I did some thing wrong.
>>>>
>>>> +     71.91%          swapper  [kernel.kallsyms]            [k] poll_idle
>>>> +     10.43%          swapper  [kernel.kallsyms]            [k] intel_idle
>>>> -      8.00%     ksoftirqd/24  [kernel.kallsyms]            [k] _raw_spin_unlock_irqrestore
>>>> \u2592   - _raw_spin_unlock_irqrestore
>>>> \u2592      - 42.25% alloc_iova
>>>> \u2592           intel_alloc_iova
>>>> \u2592           __intel_map_single
>>>> \u2592           intel_map_page
>>
>> I'm almost certain this is the issue here.  I am pretty sure the
>> intel_map_page call indicates that you are running with the Intel IOMMU
>> enabled.  As Eric suggested you can either rebuild your kernel with
>> "CONFIG_DMAR=N", or pass the kernel the parameter "intel_iommu=off" in
>> order to disable it so that it will instead just use SWIOTLB.
>
> What's the purpose of intel_iommu ?
>
> Could this be speedup if ixgbe uses a perqueue iommu context instead of
> a per device, so that we dont hit a single spinlock ?

The intel_iommu is meant to be a security feature.  Primarily it is used 
in virtualzation where it allows KVM or Xen to direct assign a device 
without having to worry about the guest getting access to the hosts 
physical memory by submitting invalid DMA requests.

If virtualzation isn't in use I would recommend turning it off as it can 
have a negative impact especially on small packet performance due to the 
extra locking overhead that is required for DMA map and unmap calls.

Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html