linux-kernel - Fwd: Memory leakage when using read_swap_cache

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAFCpxESP3qPY_exsR_ZXRdTZrHCuBR6O6zOrYG9-hCc3cDvt2w@mail.gmail.com>
Date:	Mon, 12 Oct 2015 18:43:37 -0500
From:	Viacheslav Fedorov <viacheslavf@...il.com>
To:	linux-kernel@...r.kernel.org
Subject: Fwd: Memory leakage when using read_swap_cache_async() ?

Hi ladies and gentlemen,

Please help - I am stuck.
Looks like memory pages go unaccounted for, somewhere in the system,
when I am prefetching pages from swap file. Using kernel 3.8.0 (Ubuntu
distribution).

So what I do is I have two simple applications that allocate 1GB
memory each and then proceed to write arbitrary data to the allocated
memory, with page granularity.
The system has a total of 1.3GB RAM to simulate memory pressure etc
and encourage swapping.

To test the basic functionality of my approach, I included a simple
"next-page" prefetch code into the mm/memory.c, just after the call to
do_swap_page (http://lxr.free-electrons.com/source/mm/memory.c?v=3.8#L3637)
What it should do is prefetch 10 pages following the one that has just
been swapped in on-demand. I don't really care which ones, what
matters is want to try something different than reading sequentially
from the swap file (read-ahead).
See the code below.

The problem is, if I run the test apps for the sufficient amount of
time, the "used" memory increases to the point where the applications
are killed by an OOM-killer.
After that, the available system memory is about half of what it
should have been (e.g. 600MB/1.3GB). Same thing happens to memory
usage if I stop the applications before the OOM-killer is triggered,
I am also attaching the /proc/meminfo contents for comparison on a
freshly-booted system vs. one where the two apps have been running for
200 seconds before quitting, with and without the prefetch code in
kernel.
Note how Active and Inactive show a lot of memory being "used" for something.
The printouts were made when the system is completely idle with
nothing running etc.

I also tried prefetching from within the swapin_readahead() function
(http://lxr.free-electrons.com/source/mm/swap_state.c?v=3.8#L373) with
the same result - memory gets "used up".

I would greatly appreciate any help!
Pretty much ran out of ideas as to what could be causing this.
Thanks a lot,
Slava

============== code fragment used for prefetching ============

(( at the beginning of handle_pte_fault() function...
        int rett, ii;
        pte_t *ptee;
        swp_entry_t sentry;
        struct page *page;
))

                rett = do_swap_page(mm, vma, address,
                                        pte, pmd, flags, entry);

////////////////////////////////////////////////////////////////
                /// test for memory leaks////////////////////
                   for(ii=1;ii<10;ii++){
                        ptee = pte_offset_map(pmd, address + (ii*2)<<12);
                        if(pte_present(*ptee) || pte_none(*ptee) ||
pte_file(*ptee)) return rett; // no such page in page table
                        sentry = pte_to_swp_entry(*ptee);
                        if (non_swap_entry(sentry)){
                        //      printk(KERN_ALERT " non swap entry\n");
                                return rett;
                        }
                        delayacct_set_flag(DELAYACCT_PF_SWAPIN);
                        page = lookup_swap_cache(sentry);
                        if (!page) {
                                page = read_swap_cache_async(sentry,
GFP_HIGHUSER_MOVABLE, vma, address);
                                if(page) page_cache_release(page);
                        }
                        lru_add_drain();
//                      if(page) unlock_page(page);
                        delayacct_clear_flag(DELAYACCT_PF_SWAPIN);
                    }
   //                     printk(KERN_ALERT "swapped pages\n");
                ////////////////////////////////////////////
//////////////////////////////////////////

return rett;

===================== end code fragment =====================

View attachment "meminfo_comparisons.txt" of type "text/plain" (3712 bytes)

View attachment "mbench.c" of type "text/x-csrc" (251 bytes)