| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <20251020020749.5522-1-linkinjeon@kernel.org>
Date: Mon, 20 Oct 2025 11:07:38 +0900
From: Namjae Jeon <linkinjeon@...nel.org>
To: viro@...iv.linux.org.uk,
brauner@...nel.org,
hch@...radead.org,
hch@....de,
tytso@....edu,
willy@...radead.org,
jack@...e.cz,
djwong@...nel.org,
josef@...icpanda.com,
sandeen@...deen.net,
rgoldwyn@...e.com,
xiang@...nel.org,
dsterba@...e.com,
pali@...nel.org,
ebiggers@...nel.org,
neil@...wn.name,
amir73il@...il.com
Cc: linux-fsdevel@...r.kernel.org,
linux-kernel@...r.kernel.org,
iamjoonsoo.kim@....com,
cheol.lee@....com,
jay.sim@....com,
gunho.lee@....com,
Namjae Jeon <linkinjeon@...nel.org>
Subject: [PATCH 00/11] ntfsplus: ntfs filesystem remake
Introduction
============
The NTFS filesystem[1] still remains the default filesystem for Windows
and The well-maintained NTFS driver in the Linux kernel enhances
interoperability with Windows devices, making it easier for Linux users
to work with NTFS-formatted drives. Currently, ntfs support in Linux was
the long-neglected NTFS Classic (read-only), which has been removed from
the Linux kernel, leaving the poorly maintained ntfs3. ntfs3 still has
many problems and is poorly maintained, so users and distributions are
still using the old legacy ntfs-3g.
What is ntfsplus?
=================
The remade ntfs called ntfsplus is an implementation that supports write
and the essential requirements(iomap, no buffer-head, utilities, xfstests
test result) based on read-only classic NTFS.
The old read-only ntfs code is much cleaner, with extensive comments,
offers readability that makes understanding NTFS easier. This is why
ntfsplus was developed on old read-only NTFS base.
The target is to provide current trends(iomap, no buffer head, folio),
enhanced performance, stable maintenance, utility support including fsck.
Key Features
============
- Write support:
Implement write support on classic read-only NTFS. Additionally,
integrate delayed allocation to enhance write performance through
multi-cluster allocation and minimized fragmentation of cluster bitmap.
- Switch to using iomap:
Use iomap for buffered IO writes, reads, direct IO, file extent mapping,
readpages, writepages operations.
- Stop using the buffer head:
The use of buffer head in old ntfs and switched to use folio instead.
As a result, CONFIG_BUFFER_HEAD option enable is removed in Kconfig also.
- Public utilities include fsck[2]:
While ntfs-3g includes ntfsprogs as a component, it notably lacks
the fsck implementation. So we have launched a new ntfs utilitiies
project called ntfsprogs-plus by forking from ntfs-3g after removing
unnecessary ntfs fuse implementation. fsck.ntfs can be used for ntfs
testing with xfstests as well as for recovering corrupted NTFS device.
- Performance Enhancements:
- ntfsplus vs. ntfs3:
* Performance was benchmarked using iozone with various chunk size.
- In single-thread(1T) write tests, ntfsplus show approximately
3~5% better performance.
- In multi-thread(4T) write tests, ntfsplus show approximately
35~110% better performance.
- Read throughput is identical for both ntfs implementations.
1GB file size:4096 size:16384 size:65536
MB/sec ntfsplus | ntfs3 ntfsplus | ntfs3 ntfsplus | ntfs3
─────────────────────────────────────────────────────────────────
read 399 | 399 426 | 424 429 | 430
─────────────────────────────────────────────────────────────────
write(1T) 291 | 276 325 | 305 333 | 317
write(4T) 105 | 50 113 | 78 114 | 99.6
* File list browsing performance. (about 12~14% faster)
files:100000 files:200000 files:400000
Sec ntfsplus | ntfs3 ntfsplus | ntfs3 ntfsplus | ntfs3
─────────────────────────────────────────────────────────────────
ls -lR 7.07 | 8.10 14.03 | 16.35 28.27 | 32.86
* mount time.
parti_size:1TB parti_size:2TB parti_size:4TB
Sec ntfsplus | ntfs3 ntfsplus | ntfs3 ntfsplus | ntfs3
─────────────────────────────────────────────────────────────────
mount 0.38 | 2.03 0.39 | 2.25 0.70 | 4.51
The following are the reasons why ntfsplus performance is higher
compared to ntfs3:
- Use iomap aops.
- Delayed allocation support.
- Optimize zero out for newly allocated clusters.
- Optimize runlist merge overhead with small chunck size.
- pre-load mft(inode) blocks and index(dentry) blocks to improve
readdir + stat performance.
- Load lcn bitmap on background.
- Stability improvement:
a. Pass more xfstests tests:
ntfsplus passed 287 tests, significantly higher than ntfs3's 218.
ntfsplus implement fallocate, idmapped mount and permission, etc,
resulting in a significantly high number of xfstests passing compared
to ntfs3.
b. Bonnie++ issue[3]:
The Bonnie++ benchmark fails on ntfs3 with a "Directory not empty"
error during file deletion. ntfs3 currently iterates directory
entries by reading index blocks one by one. When entries are deleted
concurrently, index block merging or entry relocation can cause
readdir() to skip some entries, leaving files undeleted in
workloads(bonnie++) that mix unlink and directory scans.
ntfsplus implement leaf chain traversal in readdir to avoid entry skip
on deletion.
- Journaling support:
ntfs3 does not provide full journaling support. It only implement journal
replay[4], which in our testing did not function correctly. My next task
after upstreaming will be to add full journal support to ntfsplus.
The feature comparison summary
==============================
Feature ntfsplus ntfs3
=================================== ======== ===========
Write support Yes Yes
iomap support Yes No
No buffer head Yes No
Public utilities(mkfs, fsck, etc.) Yes No
xfstests passed 287 218
Idmapped mount Yes No
Delayed allocation Yes No
Bonnie++ Pass Fail
Journaling Planned Inoperative
=================================== ======== ===========
References
==========
[1] https://en.wikipedia.org/wiki/NTFS
[2] https://github.com/ntfsprogs-plus/ntfsprogs-plus
[3] https://lore.kernel.org/ntfs3/CAOZgwEd7NDkGEpdF6UQTcbYuupDavaHBoj4WwTy3Qe4Bqm6V0g@mail.gmail.com/
[4] https://marc.info/?l=linux-fsdevel&m=161738417018673&q=mbox
Namjae Jeon (11):
ntfsplus: in-memory, on-disk structures and headers
ntfsplus: add super block operations
ntfsplus: add inode operations
ntfsplus: add directory operations
ntfsplus: add file operations
ntfsplus: add iomap and address space operations
ntfsplus: add attrib operatrions
ntfsplus: add runlist handling and cluster allocator
ntfsplus: add reparse and ea operations
ntfsplus: add misc operations
ntfsplus: add Kconfig and Makefile
fs/Kconfig | 1 +
fs/Makefile | 1 +
fs/ntfsplus/Kconfig | 45 +
fs/ntfsplus/Makefile | 18 +
fs/ntfsplus/aops.c | 631 +++++
fs/ntfsplus/aops.h | 92 +
fs/ntfsplus/attrib.c | 5373 +++++++++++++++++++++++++++++++++++++
fs/ntfsplus/attrib.h | 159 ++
fs/ntfsplus/attrlist.c | 276 ++
fs/ntfsplus/attrlist.h | 21 +
fs/ntfsplus/bitmap.c | 193 ++
fs/ntfsplus/bitmap.h | 90 +
fs/ntfsplus/collate.c | 173 ++
fs/ntfsplus/collate.h | 37 +
fs/ntfsplus/compress.c | 1565 +++++++++++
fs/ntfsplus/dir.c | 1226 +++++++++
fs/ntfsplus/dir.h | 33 +
fs/ntfsplus/ea.c | 712 +++++
fs/ntfsplus/ea.h | 25 +
fs/ntfsplus/file.c | 1056 ++++++++
fs/ntfsplus/index.c | 2114 +++++++++++++++
fs/ntfsplus/index.h | 127 +
fs/ntfsplus/inode.c | 3705 +++++++++++++++++++++++++
fs/ntfsplus/inode.h | 354 +++
fs/ntfsplus/layout.h | 2288 ++++++++++++++++
fs/ntfsplus/lcnalloc.c | 993 +++++++
fs/ntfsplus/lcnalloc.h | 127 +
fs/ntfsplus/logfile.c | 773 ++++++
fs/ntfsplus/logfile.h | 316 +++
fs/ntfsplus/mft.c | 2630 ++++++++++++++++++
fs/ntfsplus/mft.h | 93 +
fs/ntfsplus/misc.c | 221 ++
fs/ntfsplus/misc.h | 218 ++
fs/ntfsplus/mst.c | 195 ++
fs/ntfsplus/namei.c | 1606 +++++++++++
fs/ntfsplus/ntfs.h | 172 ++
fs/ntfsplus/ntfs_iomap.c | 704 +++++
fs/ntfsplus/ntfs_iomap.h | 22 +
fs/ntfsplus/reparse.c | 550 ++++
fs/ntfsplus/reparse.h | 15 +
fs/ntfsplus/runlist.c | 1995 ++++++++++++++
fs/ntfsplus/runlist.h | 91 +
fs/ntfsplus/super.c | 2716 +++++++++++++++++++
fs/ntfsplus/unistr.c | 471 ++++
fs/ntfsplus/upcase.c | 73 +
fs/ntfsplus/volume.h | 241 ++
include/uapi/linux/ntfs.h | 23 +
47 files changed, 34560 insertions(+)
create mode 100644 fs/ntfsplus/Kconfig
create mode 100644 fs/ntfsplus/Makefile
create mode 100644 fs/ntfsplus/aops.c
create mode 100644 fs/ntfsplus/aops.h
create mode 100644 fs/ntfsplus/attrib.c
create mode 100644 fs/ntfsplus/attrib.h
create mode 100644 fs/ntfsplus/attrlist.c
create mode 100644 fs/ntfsplus/attrlist.h
create mode 100644 fs/ntfsplus/bitmap.c
create mode 100644 fs/ntfsplus/bitmap.h
create mode 100644 fs/ntfsplus/collate.c
create mode 100644 fs/ntfsplus/collate.h
create mode 100644 fs/ntfsplus/compress.c
create mode 100644 fs/ntfsplus/dir.c
create mode 100644 fs/ntfsplus/dir.h
create mode 100644 fs/ntfsplus/ea.c
create mode 100644 fs/ntfsplus/ea.h
create mode 100644 fs/ntfsplus/file.c
create mode 100644 fs/ntfsplus/index.c
create mode 100644 fs/ntfsplus/index.h
create mode 100644 fs/ntfsplus/inode.c
create mode 100644 fs/ntfsplus/inode.h
create mode 100644 fs/ntfsplus/layout.h
create mode 100644 fs/ntfsplus/lcnalloc.c
create mode 100644 fs/ntfsplus/lcnalloc.h
create mode 100644 fs/ntfsplus/logfile.c
create mode 100644 fs/ntfsplus/logfile.h
create mode 100644 fs/ntfsplus/mft.c
create mode 100644 fs/ntfsplus/mft.h
create mode 100644 fs/ntfsplus/misc.c
create mode 100644 fs/ntfsplus/misc.h
create mode 100644 fs/ntfsplus/mst.c
create mode 100644 fs/ntfsplus/namei.c
create mode 100644 fs/ntfsplus/ntfs.h
create mode 100644 fs/ntfsplus/ntfs_iomap.c
create mode 100644 fs/ntfsplus/ntfs_iomap.h
create mode 100644 fs/ntfsplus/reparse.c
create mode 100644 fs/ntfsplus/reparse.h
create mode 100644 fs/ntfsplus/runlist.c
create mode 100644 fs/ntfsplus/runlist.h
create mode 100644 fs/ntfsplus/super.c
create mode 100644 fs/ntfsplus/unistr.c
create mode 100644 fs/ntfsplus/upcase.c
create mode 100644 fs/ntfsplus/volume.h
create mode 100644 include/uapi/linux/ntfs.h
--
2.34.1
Powered by blists - more mailing lists