linux-kernel - [PATCH v4 03/40] lib/sbitmap; optimize __sbitmap_get_word() by using find_and_set

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Thu, 20 Jun 2024 10:56:26 -0700
From: Yury Norov <yury.norov@...il.com>
To: linux-kernel@...r.kernel.org,
	Jens Axboe <axboe@...nel.dk>,
	linux-block@...r.kernel.org
Cc: Yury Norov <yury.norov@...il.com>,
	Alexey Klimov <alexey.klimov@...aro.org>,
	Bart Van Assche <bvanassche@....org>,
	Jan Kara <jack@...e.cz>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Matthew Wilcox <willy@...radead.org>,
	Mirsad Todorovac <mirsad.todorovac@....unizg.hr>,
	Rasmus Villemoes <linux@...musvillemoes.dk>,
	Sergey Shtylyov <s.shtylyov@....ru>
Subject: [PATCH v4 03/40] lib/sbitmap; optimize __sbitmap_get_word() by using find_and_set_bit()

__sbitmap_get_word() opencodes either find_and_set_bit_wrap(), or
find_and_set_next_bit(), depending on wrap parameter. Simplify it
by using atomic find_bit() API.

While here, simplify sbitmap_find_bit_in_word(), which calls it.

CC: Jens Axboe <axboe@...nel.dk>
Signed-off-by: Yury Norov <yury.norov@...il.com>
Reviewed-by: Jan Kara <jack@...e.cz>
---
 lib/sbitmap.c | 47 ++++++++++-------------------------------------
 1 file changed, 10 insertions(+), 37 deletions(-)

diff --git a/lib/sbitmap.c b/lib/sbitmap.c
index 1e453f825c05..3881996217c9 100644
--- a/lib/sbitmap.c
+++ b/lib/sbitmap.c
@@ -4,6 +4,7 @@
  * Copyright (C) 2013-2014 Jens Axboe
  */
 
+#include <linux/find_atomic.h>
 #include <linux/sched.h>
 #include <linux/random.h>
 #include <linux/sbitmap.h>
@@ -133,38 +134,13 @@ void sbitmap_resize(struct sbitmap *sb, unsigned int depth)
 }
 EXPORT_SYMBOL_GPL(sbitmap_resize);
 
-static int __sbitmap_get_word(unsigned long *word, unsigned long depth,
+static inline int __sbitmap_get_word(unsigned long *word, unsigned long depth,
 			      unsigned int hint, bool wrap)
 {
-	int nr;
-
-	/* don't wrap if starting from 0 */
-	wrap = wrap && hint;
-
-	while (1) {
-		nr = find_next_zero_bit(word, depth, hint);
-		if (unlikely(nr >= depth)) {
-			/*
-			 * We started with an offset, and we didn't reset the
-			 * offset to 0 in a failure case, so start from 0 to
-			 * exhaust the map.
-			 */
-			if (hint && wrap) {
-				hint = 0;
-				continue;
-			}
-			return -1;
-		}
+	if (wrap)
+		return find_and_set_bit_wrap_lock(word, depth, hint);
 
-		if (!test_and_set_bit_lock(nr, word))
-			break;
-
-		hint = nr + 1;
-		if (hint >= depth - 1)
-			hint = 0;
-	}
-
-	return nr;
+	return find_and_set_next_bit_lock(word, depth, hint);
 }
 
 static int sbitmap_find_bit_in_word(struct sbitmap_word *map,
@@ -175,15 +151,12 @@ static int sbitmap_find_bit_in_word(struct sbitmap_word *map,
 	int nr;
 
 	do {
-		nr = __sbitmap_get_word(&map->word, depth,
-					alloc_hint, wrap);
-		if (nr != -1)
-			break;
-		if (!sbitmap_deferred_clear(map))
-			break;
-	} while (1);
+		nr = __sbitmap_get_word(&map->word, depth, alloc_hint, wrap);
+		if (nr < depth)
+			return nr;
+	} while (sbitmap_deferred_clear(map));
 
-	return nr;
+	return -1;
 }
 
 static int sbitmap_find_bit(struct sbitmap *sb,
-- 
2.43.0