lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAFCpxET8=jJ4QOO_96xBf4rC2tBeRwaZifEDBAcz1oFLDtui6A@mail.gmail.com> Date: Tue, 13 Oct 2015 15:17:28 -0500 From: Viacheslav Fedorov <viacheslavf@...il.com> To: Hugh Dickins <hughd@...gle.com> Cc: linux-kernel@...r.kernel.org Subject: Re: Fwd: Memory leakage when using read_swap_cache_async() ? Wow! Thank you so much! I literally spent a week trying to figure out the reason for the behavior. This solves the issue I was having at this point. Thanks again! Slava On Mon, Oct 12, 2015 at 7:40 PM, Hugh Dickins <hughd@...gle.com> wrote: > On Mon, 12 Oct 2015, Viacheslav Fedorov wrote: > >> Hi ladies and gentlemen, >> >> Please help - I am stuck. >> Looks like memory pages go unaccounted for, somewhere in the system, >> when I am prefetching pages from swap file. Using kernel 3.8.0 (Ubuntu >> distribution). >> >> So what I do is I have two simple applications that allocate 1GB >> memory each and then proceed to write arbitrary data to the allocated >> memory, with page granularity. >> The system has a total of 1.3GB RAM to simulate memory pressure etc >> and encourage swapping. >> >> To test the basic functionality of my approach, I included a simple >> "next-page" prefetch code into the mm/memory.c, just after the call to >> do_swap_page (http://lxr.free-electrons.com/source/mm/memory.c?v=3.8#L3637) >> What it should do is prefetch 10 pages following the one that has just >> been swapped in on-demand. I don't really care which ones, what >> matters is want to try something different than reading sequentially >> from the swap file (read-ahead). >> See the code below. >> >> The problem is, if I run the test apps for the sufficient amount of >> time, the "used" memory increases to the point where the applications >> are killed by an OOM-killer. >> After that, the available system memory is about half of what it >> should have been (e.g. 600MB/1.3GB). Same thing happens to memory >> usage if I stop the applications before the OOM-killer is triggered, >> I am also attaching the /proc/meminfo contents for comparison on a >> freshly-booted system vs. one where the two apps have been running for >> 200 seconds before quitting, with and without the prefetch code in >> kernel. >> Note how Active and Inactive show a lot of memory being "used" for something. >> The printouts were made when the system is completely idle with >> nothing running etc. >> >> >> I also tried prefetching from within the swapin_readahead() function >> (http://lxr.free-electrons.com/source/mm/swap_state.c?v=3.8#L373) with >> the same result - memory gets "used up". >> >> >> I would greatly appreciate any help! >> Pretty much ran out of ideas as to what could be causing this. >> Thanks a lot, >> Slava >> >> ============== code fragment used for prefetching ============ >> >> (( at the beginning of handle_pte_fault() function... >> int rett, ii; >> pte_t *ptee; >> swp_entry_t sentry; >> struct page *page; >> )) >> >> rett = do_swap_page(mm, vma, address, >> pte, pmd, flags, entry); >> >> //////////////////////////////////////////////////////////////// >> /// test for memory leaks//////////////////// >> for(ii=1;ii<10;ii++){ >> ptee = pte_offset_map(pmd, address + (ii*2)<<12); >> if(pte_present(*ptee) || pte_none(*ptee) || >> pte_file(*ptee)) return rett; // no such page in page table >> sentry = pte_to_swp_entry(*ptee); >> if (non_swap_entry(sentry)){ >> // printk(KERN_ALERT " non swap entry\n"); >> return rett; >> } >> delayacct_set_flag(DELAYACCT_PF_SWAPIN); >> page = lookup_swap_cache(sentry); > > That gets a reference to page. > >> if (!page) { >> page = read_swap_cache_async(sentry, >> GFP_HIGHUSER_MOVABLE, vma, address); >> if(page) page_cache_release(page); >> } >> lru_add_drain(); >> // if(page) unlock_page(page); > > But you don't appear to release that reference. > >> delayacct_clear_flag(DELAYACCT_PF_SWAPIN); >> } >> // printk(KERN_ALERT "swapped pages\n"); >> //////////////////////////////////////////// >> ////////////////////////////////////////// >> >> return rett; >> >> ===================== end code fragment ===================== -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists