lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAA0+VHz9+LpS9jGbTEE_OE9kvgdXDqj3qMQQV5kTTJ12WDxMPg@mail.gmail.com> Date: Wed, 13 Mar 2013 20:04:57 +0400 From: Andrey Sidorov <qrxd43@...orola.com> To: linux-ext4@...r.kernel.org Subject: mke2fs with bigalloc is too slow Hi, It takes 29 seconds to format 1TB partition as ext4 with 256k cluster size on MIPS. e2fsprogs version is 1.42.7. The most time consumer is ext2fs_convert_subcluster_bitmap which folds 30M into 500K walking through block bitmap bit by bit. Afair, it takes less time on x86 since there are asm bit ops for x86, but mips uses generic ones. Of course this folding can be optimized so that it tests series of bits by byte, short or int and does less shifts and other address arithmetic in between, but I don't expect that to improve time dramatically. First thing I did is allocated per-cluster bitmap instead of per-block bitmap in ext2fs_initialize so that conversion doesn't occur. That dropped mke2fs time from 29s to 2.5s. e2fsck -f found this fresh fs as a good one and mounting/writing/reading/unmounting also went good. Of course groups are allocated at different offsets and about 60M of usable space are lost if compared to 'slow' formatting. That's fine, we can live with that. Are there any long-term consequences that I've missed? Are there any reasons for using block bitmap instead of cluster bitmap except for meta-data space efficiency? As a long term solution I see following options: 1) Continue using block bitmap as base but allocate mirror cluster bitmap and replay all base block bitmap allocations to it as soon as they happen. This will increase memory usage by block/cluster ratio, e.g. 1.6% in case of 4k blocks / 256k clusters. 2) Use cluster bitmap as base and use small sub-cluster bitmaps for tracking blocks inside allocated clusters (some sort of search tree). This will reduce allocated memory significantly in case of high cluster/block ratio . 3) Variation of (2). Allocate cluster bitmap as base. Allocate block bitmap via mmap as a mirror, but don't commit all the memory by memset. This is simpler than (2) since there is no search structure and at the same time memory is not over-allocated. Please let me know your thoughts. Thanks! Regards, Andrey. -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists