linux-kernel - Re: Fwd: Memory leakage when using read_swap_cache

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Tue, 13 Oct 2015 15:17:28 -0500
From:	Viacheslav Fedorov <viacheslavf@...il.com>
To:	Hugh Dickins <hughd@...gle.com>
Cc:	linux-kernel@...r.kernel.org
Subject: Re: Fwd: Memory leakage when using read_swap_cache_async() ?

Wow! Thank you so much!
I literally spent a week trying to figure out the reason for the behavior.

This solves the issue I was having at this point.

Thanks again!
Slava

On Mon, Oct 12, 2015 at 7:40 PM, Hugh Dickins <hughd@...gle.com> wrote:
> On Mon, 12 Oct 2015, Viacheslav Fedorov wrote:
>
>> Hi ladies and gentlemen,
>>
>> Please help - I am stuck.
>> Looks like memory pages go unaccounted for, somewhere in the system,
>> when I am prefetching pages from swap file. Using kernel 3.8.0 (Ubuntu
>> distribution).
>>
>> So what I do is I have two simple applications that allocate 1GB
>> memory each and then proceed to write arbitrary data to the allocated
>> memory, with page granularity.
>> The system has a total of 1.3GB RAM to simulate memory pressure etc
>> and encourage swapping.
>>
>> To test the basic functionality of my approach, I included a simple
>> "next-page" prefetch code into the mm/memory.c, just after the call to
>> do_swap_page (http://lxr.free-electrons.com/source/mm/memory.c?v=3.8#L3637)
>> What it should do is prefetch 10 pages following the one that has just
>> been swapped in on-demand. I don't really care which ones, what
>> matters is want to try something different than reading sequentially
>> from the swap file (read-ahead).
>> See the code below.
>>
>> The problem is, if I run the test apps for the sufficient amount of
>> time, the "used" memory increases to the point where the applications
>> are killed by an OOM-killer.
>> After that, the available system memory is about half of what it
>> should have been (e.g. 600MB/1.3GB). Same thing happens to memory
>> usage if I stop the applications before the OOM-killer is triggered,
>> I am also attaching the /proc/meminfo contents for comparison on a
>> freshly-booted system vs. one where the two apps have been running for
>> 200 seconds before quitting, with and without the prefetch code in
>> kernel.
>> Note how Active and Inactive show a lot of memory being "used" for something.
>> The printouts were made when the system is completely idle with
>> nothing running etc.
>>
>>
>> I also tried prefetching from within the swapin_readahead() function
>> (http://lxr.free-electrons.com/source/mm/swap_state.c?v=3.8#L373) with
>> the same result - memory gets "used up".
>>
>>
>> I would greatly appreciate any help!
>> Pretty much ran out of ideas as to what could be causing this.
>> Thanks a lot,
>> Slava
>>
>> ============== code fragment used for prefetching ============
>>
>> (( at the beginning of handle_pte_fault() function...
>>         int rett, ii;
>>         pte_t *ptee;
>>         swp_entry_t sentry;
>>         struct page *page;
>> ))
>>
>>                 rett = do_swap_page(mm, vma, address,
>>                                         pte, pmd, flags, entry);
>>
>> ////////////////////////////////////////////////////////////////
>>                 /// test for memory leaks////////////////////
>>                    for(ii=1;ii<10;ii++){
>>                         ptee = pte_offset_map(pmd, address + (ii*2)<<12);
>>                         if(pte_present(*ptee) || pte_none(*ptee) ||
>> pte_file(*ptee)) return rett; // no such page in page table
>>                         sentry = pte_to_swp_entry(*ptee);
>>                         if (non_swap_entry(sentry)){
>>                         //      printk(KERN_ALERT " non swap entry\n");
>>                                 return rett;
>>                         }
>>                         delayacct_set_flag(DELAYACCT_PF_SWAPIN);
>>                         page = lookup_swap_cache(sentry);
>
> That gets a reference to page.
>
>>                         if (!page) {
>>                                 page = read_swap_cache_async(sentry,
>> GFP_HIGHUSER_MOVABLE, vma, address);
>>                                 if(page) page_cache_release(page);
>>                         }
>>                         lru_add_drain();
>> //                      if(page) unlock_page(page);
>
> But you don't appear to release that reference.
>
>>                         delayacct_clear_flag(DELAYACCT_PF_SWAPIN);
>>                     }
>>    //                     printk(KERN_ALERT "swapped pages\n");
>>                 ////////////////////////////////////////////
>> //////////////////////////////////////////
>>
>> return rett;
>>
>> ===================== end code fragment =====================
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/