my_cpu_ptr(xx) = per_cpu_ptr(xx, smp_processor_id). The problem with per_cpu_ptr(x, smp_processor_id) is that it requires an array lookup to find the offset for the cpu. Processors typically have the offset for the current cpu area in some kind of (arch dependent) efficiently accessible register or memory location. We can use that instead of doing the array lookup to speed up the determination of the addressof the percpu variable. This is particularly significant because these lookups occur in performance critical paths of the core kernel. This optimization is a prerequiste to the introduction of per processor atomic operations for the core code. Atomic per processor operations implicitly do the offset calculation to the current per cpu area in a single instruction. All the locations touched by this patchset are potential candidates for atomic per cpu operations. my_cpu_ptr comes in two flavors. The preemption context matters since we are referring the the currently executing processor. In many cases we must insure that the processor does not change while a code segment is executed. __my_cpu_ptr -> Do not check for preemption context my_cpu_ptr -> Check preemption context Cc: David Howells Cc: Tejun Heo Cc: Ingo Molnar Cc: Rusty Russell Cc: Eric Dumazet Signed-off-by: Christoph Lameter --- include/linux/percpu.h | 11 +++++++++++ 1 file changed, 11 insertions(+) Index: linux-2.6/include/linux/percpu.h =================================================================== --- linux-2.6.orig/include/linux/percpu.h 2009-05-27 11:32:36.000000000 -0500 +++ linux-2.6/include/linux/percpu.h 2009-05-27 11:33:32.000000000 -0500 @@ -78,6 +78,9 @@ extern ssize_t __init pcpu_embed_first_c */ #define per_cpu_ptr(ptr, cpu) SHIFT_PERCPU_PTR((ptr), per_cpu_offset((cpu))) +#define my_cpu_ptr(ptr) SHIFT_PERCPU_PTR(ptr, my_cpu_offset) +#define __my_cpu_ptr(ptr) SHIFT_PERCPU_PTR(ptr, __my_cpu_offset) + extern void *__alloc_reserved_percpu(size_t size, size_t align); #else /* CONFIG_HAVE_DYNAMIC_PER_CPU_AREA */ @@ -94,6 +97,12 @@ struct percpu_data { (__typeof__(ptr))__p->ptrs[(cpu)]; \ }) +#define my_cpu_ptr(ptr) \ + per_cpu_ptr(ptr, smp_processor_id()) + +#define __my_cpu_ptr(ptr) \ + per_cpu_ptr(ptr, raw_smp_processor_id()) + #endif /* CONFIG_HAVE_DYNAMIC_PER_CPU_AREA */ extern void *__alloc_percpu(size_t size, size_t align); @@ -102,6 +111,8 @@ extern void free_percpu(void *__pdata); #else /* CONFIG_SMP */ #define per_cpu_ptr(ptr, cpu) ({ (void)(cpu); (ptr); }) +#define my_cpu_ptr(ptr) per_cpu_ptr(ptr, 0) +#define __my_cpu_ptr(ptr) my_cpu_ptr(ptr) static inline void *__alloc_percpu(size_t size, size_t align) { -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/