Message-ID: <20130424044848.GI2672@localhost.localdomain>
Date:	Wed, 24 Apr 2013 12:48:48 +0800
From:	Han Pingtian <hanpt@...ux.vnet.ibm.com>
To:	linux-kernel@...r.kernel.org
Cc:	mhocko@...e.cz, rientjes@...gle.com
Subject: Re: OOM-killer and strange RSS value in 3.9-rc7

On Tue, Apr 23, 2013 at 03:15:58PM +0200, Michal Hocko wrote:
> On Tue 23-04-13 12:22:34, Han Pingtian wrote:
> > On Mon, Apr 22, 2013 at 01:40:52PM +0200, Michal Hocko wrote:
> > > >  CONFIG_PPC_BOOK3S_64=y
> > > >  # CONFIG_PPC_BOOK3E_64 is not set
> > > > -CONFIG_GENERIC_CPU=y
> > > > +# CONFIG_GENERIC_CPU is not set
> > > >  # CONFIG_CELL_CPU is not set
> > > >  # CONFIG_POWER4_CPU is not set
> > > >  # CONFIG_POWER5_CPU is not set
> > > >  # CONFIG_POWER6_CPU is not set
> > > > -# CONFIG_POWER7_CPU is not set
> > > > +CONFIG_POWER7_CPU=y
> > > 
> > > Wow, so the two configs are for different architectures? Not very much
> > > helpful. Could you stick with a single machine and do just small updates
> > > to the config to the point where the problem is no longer present,
> > > please?
> > > 
> > No, both configs are for power system. I changed it to
> > 'CONFIG_POWER7_CPU' by myself in the good one. I think this difference
> > doesn't matter here. 
> 
> Ohh, good, I thought it was a x86 thingy. Sorry about the confusion. No
> other arch does that...
> 
> > > [...]
> > > > -CONFIG_SLUB_DEBUG=y
> > > > -# CONFIG_COMPAT_BRK is not set
> > > > -# CONFIG_SLAB is not set
> > > > -CONFIG_SLUB=y
> > > > +CONFIG_COMPAT_BRK=y
> > > > +CONFIG_SLAB=y
> > > 
> > > I would start with the bad config and SLUB changed to SLAB in the first
> > > step, though.
> > > 
> > Yep, after I changed bad config to use SLAB, the problem disppears after
> > using the new kernel. Looks like something wrong in SLUB?
> 
> It seems so or maybe there is a different issue which is just made
> visible by SLUB or it is an arch specific problem.
> What is the workload that triggers this behavior? Can you reduce it to a
> minimum test case?
> 
I can trigger this behavior just with "make -j 64" to rebuild kernel
from source code. In fact, just after a fresh reboot, I haven't run
anything, the memory is almost all used or leaked:

# uname -a
Linux riblp3.upt.austin.ibm.com 3.9.0-rc7 #7 SMP Tue Apr 23 22:49:09 CDT 2013 ppc64 ppc64 ppc64 GNU/Linux
# free -m
             total       used       free     shared    buffers     cached
Mem:          3788       3697         90          0          1        143
-/+ buffers/cache:       3552        235
Swap:         4031          5       4026
# awk '{print $1":", $2*$4/1024/1024, $3*$4/1024/1024}' /proc/slabinfo |sort -k 3 -n
#: 0 0
bip-128: 0 0
bsg_cmd: 0 0
configfs_dir_cache: 0 0
dma-kmalloc-1024: 0 0
dma-kmalloc-128: 0 0
dma-kmalloc-131072: 0 0
dma-kmalloc-16: 0 0
dma-kmalloc-16384: 0 0
dma-kmalloc-192: 0 0
dma-kmalloc-2048: 0 0
dma-kmalloc-256: 0 0
dma-kmalloc-32: 0 0
dma-kmalloc-32768: 0 0
dma-kmalloc-4096: 0 0
dma-kmalloc-512: 0 0
dma-kmalloc-64: 0 0
dma-kmalloc-65536: 0 0
dma-kmalloc-8: 0 0
dma-kmalloc-8192: 0 0
dma-kmalloc-96: 0 0
dm_rq_target_io: 0 0
dm_uevent: 0 0
dquot: 0 0
ext4_allocation_context: 0 0
ext4_extent_status: 0 0
ext4_free_data: 0 0
ext4_inode_cache: 0 0
ext4_io_end: 0 0
ext4_io_page: 0 0
ext4_xattr: 0 0
isofs_inode_cache: 0 0
jbd2_journal_head: 0 0
jbd2_revoke_record_s: 0 0
kcopyd_job: 0 0
kmalloc-131072: 0 0
net_namespace: 0 0
nfsd4_delegations: 0 0
nfsd4_openowners: 0 0
nfs_direct_cache: 0 0
pid_namespace: 0 0
rtas_flash_cache: 0 0
scsi_tgt_cmd: 0 0
slabinfo: 0 0
tw_sock_TCPv6: 0 0
UDP-Lite: 0 0
UDPLITEv6: 0 0
xfs_dquot: 0 0
mqueue_inode_cache: 0.0623779 0.0623779
tw_sock_TCP: 0.0625 0.0625
hugetlbfs_inode_cache: 0.124146 0.124146
taskstats: 0.186745 0.186745
fscache_cookie_jar: 0.187454 0.187454
ip_fib_trie: 0.187454 0.187454
bip-256: 0.249756 0.249756
shared_policy_node: 0.249939 0.249939
nfs_inode_cache: 0.25 0.25
rpc_inode_cache: 0.311279 0.311279
UDPv6: 0.553711 0.553711
kmem_cache_node: 0.5625 0.5625
dio: 0.622559 0.622559
TCPv6: 0.622559 0.622559
blkdev_queue: 0.679443 0.679443
bdev_cache: 0.748535 0.748535
xfs_efd_item: 0.994873 0.994873
fsnotify_event_holder: 1.18721 1.18721
nf_conntrack_c00000000102d400: 1.18721 1.18721
kmem_cache: 1.23047 1.23047
TCP: 1.96875 1.96875
UDP: 2.12085 2.12085
xfs_btree_cur: 2.24945 2.24945
buffer_head: 2.4369 2.4369
posix_timers_cache: 2.55812 2.55812
xfs_ili: 3.36841 3.36841
blkdev_ioc: 3.37418 3.37418
bip-16: 3.6731 3.6731
xfs_da_state: 3.99121 3.99121
xfs_trans: 3.99902 3.99902
shmem_inode_cache: 4.24429 4.24429
blkdev_requests: 4.67949 4.67949
ext4_system_zone: 4.68636 4.68636
radix_tree_node: 4.79664 4.79664
sigqueue: 4.93027 4.93027
fsnotify_event: 4.93629 4.93629
kmalloc-8: 4.9375 4.9375
files_cache: 4.98047 4.98047
RAW: 4.99023 4.99023
pgtable-2^6: 5 5
sock_inode_cache: 5.10498 5.10498
anon_vma: 5.25 5.25
numa_policy: 5.3112 5.3112
kmalloc-32: 5.00131 5.625
xfs_inode: 5.86353 5.86353
signal_cache: 5.90625 5.90625
kmalloc-64: 5.72687 6.375
sysfs_dir_cache: 6.74835 6.74835
vm_area_struct: 7.05905 7.05905
dentry: 9.11609 9.11609
inode_cache: 10.6849 10.6849
sighand_cache: 11.2061 11.2061
proc_inode_cache: 11.3722 11.3722
idr_layer_cache: 11.9883 11.9883
kmalloc-4096: 10.7812 12
kmalloc-256: 13.1768 16.0625
kmalloc-16: 9.19449 16.25
kmalloc-1024: 6.83789 22.0625
kmalloc-192: 9.17725 28.8468
kmalloc-128: 9.43066 31.125
kmalloc-65536: 40 40
pgtable-2^12: 88.25 91.5
kmalloc-8192: 28.3672 117.5
kmalloc-32768: 20.4375 131.5
task_struct: 25.5788 230.002
kmalloc-2048: 16.8555 299.688
kmalloc-96: 7.84653 300.956
kmalloc-16384: 54.125 504.25
kmalloc-512: 17.2104 1170.56
# cat /proc/meminfo
MemTotal:        3879104 kB
MemFree:           90624 kB
Buffers:            1920 kB
Cached:           173888 kB
SwapCached:         5184 kB
Active:           240064 kB
Inactive:         152256 kB
Active(anon):     153088 kB
Inactive(anon):    68608 kB
Active(file):      86976 kB
Inactive(file):    83648 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       4128704 kB
SwapFree:        4123520 kB
Dirty:                64 kB
Writeback:             0 kB
AnonPages:        213056 kB
Mapped:            71808 kB
Shmem:              3200 kB
Slab:            3275968 kB
SReclaimable:      52864 kB
SUnreclaim:      3223104 kB
KernelStack:       14352 kB
PageTables:        14464 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     6068224 kB
Committed_AS:    2497728 kB
VmallocTotal:   8589934592 kB
VmallocUsed:       17984 kB
VmallocChunk:   8589641856 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:      16384 kB

> I guess that you are still testing with vanilla 3.9-rc? kernel without
> any additional patches or 3rd party modules, right?
> 
Right. I'm testing with vanilla 3.9-rc7 kernel without any additional
patches and 3rd party modules.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists