linux-kernel - vmalloc search size larger than the size it is actually needed

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date: Tue, 2 Jan 2024 10:37:15 -0800
From: Yuxiao Zhang <yuxiaozhang@...gle.com>
To: akpm@...ux-foundation.org, hch@...radead.org, urezki@...il.com, 
	lstoakes@...il.com
Cc: linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: vmalloc search size larger than the size it is actually needed

Hi vmalloc experts,

Recently I found an issue when unbind/bind a driver and hit vmalloc
failure. The symptom is that the initial memory allocation can succeed
during the start up. But if later I unbind (so it frees the memory)
and bind it again (allocating the same size), it will fail. further
investigation shows that it is because that the search size is
slightly larger than the real size it is needed:

/* Adjust the search size for alignment overhead. */
        length = adjust_search_size ? size + align - 1 : size;

Assuming that the device wants to vmalloc 256M mem, What happens here
is that during the bootup, the driver bind happens quite early so that
there is plenty of free chunks in vmalloc spaces, and the largest is
far greater than 256M, so the check never failed. After the boot up,
the vmalloc is filled with other stuff, and the largetst remaining
chunk is smaller than 256M. Then I unbind the device, the used 256M
chunk is returned to the free vm tree/list. When I bind it again,
ideally it should take the previous area, but the search is checking
more than 256M (256M + align - 1) so the vmalloc failed, although the
original chunk can still fit.

Is this expected? Maybe we can add another find_vmap_lowest_match in
__alloc_vmap_area with adjust_search_size==false, if the original find
failed. Does this make sense?

Thanks,
-Yuxiao