| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Mon, 22 Oct 2018 17:23:36 -0700
From: Liu Bo <obuil.liubo@...il.com>
To: David Sterba <dsterba@...e.com>
Cc: torvalds@...ux-foundation.org, Chris Mason <clm@...com>,
linux-btrfs@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [GIT PULL] Btrfs updates for 4.20, part 1
On Mon, Oct 22, 2018 at 10:24 AM David Sterba <dsterba@...e.com> wrote:
>
> Hi,
>
> this is the first batch with fixes and some nice performance improvements.
>
> Preliminary results show eg. more files/sec in fsmark, better perf on
> multi-threaded workloads (filebench, dbench), fewer context switches and
> overall better memory allocation characteristics (multiple benchmarks).
>
> Apart from general performance, there's an improvement for qgroups +
> balance workload that's been troubling our users.
>
> Note for stable: there are 20+ patches tagged for stable, out of 90. Not
> all of them apply cleanly on all stable versions but the conflicts are
> mostly due to simple cleanups and resolving should be obvious. The fixes
> are otherwise independent.
>
> No merge conflicts expected. Please pull, thanks.
>
>
> Performance improvements:
>
> * blocking mode of path is gone, means that only the spinning mode is used;
I'd like to do a few corrections here, the transition from the
spinning mode to blocking mode is removed, we still need blocking mode
of path for sleeping context.
thanks,
liubo
> the blocking resulted in more unnecessary wakeups and updates to the path
> locks, the effects are measurable and improve latency and scaleability
>
> * qgroups: first batch of changes that should speedup balancing with qgroups
> on, skip quota accounting on unchanged subtrees, overall gain is about 30+%
> in runtime
>
> * use rb-tree with cached first node for several structures, small improvement
> to avoid pointer chasing
>
> Fixes:
>
> * trim
> * fix: some blockgroups could have been missed if their logical address was
> past the total filesystem size (ie. after a lot of balancing)
> * better error reporting, after processing blockgroups and whole device
> * fix: continue trimming block groups after an error is encountered
> * check for trim support of the device earlier and avoid some unnecessary work
> * less interaction with transaction commit that improves latency on slower
> storage (eg. image files over NFS)
>
> * fsync
> * fix warning when replaying log after fsync of a O_TMPFILE
> * fix wrong dentries after fsync of file that got its parent replaced
>
> * qgroups: fix rescan that might misc some dirty groups
>
> * don't clean dirty pages during buffered writes, this could lead to lost
> updates in some corner cases
>
> * some block groups could have been delayed in creation, if the allocation
> triggered another one
>
> * error handling improvements
>
> Cleanups:
>
> * removed unused struct members and variables
> * function return type cleanups
> * delayed refs code refactoring
>
> * protect against deadlock that could be caused by crafted image that tries to
> allocate from a tree that's locked already
>
> ----------------------------------------------------------------
> The following changes since commit 35a7f35ad1b150ddf59a41dcac7b2fa32982be0e:
>
> Linux 4.19-rc8 (2018-10-15 07:20:24 +0200)
>
> are available in the Git repository at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.20-part1-tag
>
> for you to fetch changes up to d9352794dad9f28535439d85a815978878c141ab:
>
> btrfs: switch return_bigger to bool in find_ref_head (2018-10-15 17:23:41 +0200)
>
> ----------------------------------------------------------------
> Anand Jain (2):
> btrfs: add assertions where number of devices could go below 0
> btrfs: add helper to obtain number of devices with ongoing dev-replace
>
> Chris Mason (1):
> Btrfs: don't clean dirty pages during buffered writes
>
> Colin Ian King (2):
> btrfs: remove unused pointer inode in relink_file_extents
> btrfs: remove unused pointer 'tree' in btrfs_submit_compressed_read
>
> David Sterba (12):
> btrfs: tests: add separate stub for find_lock_delalloc_range
> btrfs: tests: move testing members of struct btrfs_root to the end
> btrfs: tests: group declarations of self-test helpers
> btrfs: tests: polish ifdefs around testing helper
> btrfs: use common helper instead of open coding a bit test
> btrfs: remove btrfs_dev_replace::read_locks
> btrfs: open code btrfs_dev_replace_clear_lock_blocking
> btrfs: open code btrfs_dev_replace_stats_inc
> btrfs: open code btrfs_after_dev_replace_commit
> btrfs: dev-replace: avoid useless lock on error handling path
> btrfs: dev-replace: move replace members out of fs_info
> btrfs: dev-replace: remove pointless assert in write unlock
>
> Filipe Manana (2):
> Btrfs: fix warning when replaying log after fsync of a tmpfile
> Btrfs: fix wrong dentries after fsync of file that got its parent replaced
>
> Jeff Mahoney (5):
> btrfs: fix error handling in free_log_tree
> btrfs: fix error handling in btrfs_dev_replace_start
> btrfs: iterate all devices during trim, instead of fs_devices::alloc_list
> btrfs: don't attempt to trim devices that don't support it
> btrfs: keep trim from interfering with transaction commits
>
> Josef Bacik (7):
> btrfs: wait on caching when putting the bg cache
> btrfs: release metadata before running delayed refs
> btrfs: protect space cache inode alloc with GFP_NOFS
> btrfs: reset max_extent_size on clear in a bitmap
> btrfs: make sure we create all new block groups
> btrfs: assert on non-empty delayed iputs
> btrfs: drop min_size from evict_refill_and_join
>
> Liu Bo (19):
> Btrfs: do not unnecessarily pass write_lock_level when processing leaf
> Btrfs: remove always true if branch in btrfs_get_extent
> Btrfs: use next_state in find_first_extent_bit
> btrfs: free path at an earlier point in btrfs_get_extent
> Btrfs: remove confusing tracepoint in btrfs_add_reserved_bytes
> Btrfs: fix alignment in declaration and prototype of btrfs_get_extent
> Btrfs: set leave_spinning in btrfs_get_extent
> Btrfs: use args in the correct order for kcalloc in btrfsic_read_block
> Btrfs: unify error handling of btrfs_lookup_dir_item
> Btrfs: remove unnecessary level check in balance_level
> Btrfs: assert page dirty bit on extent buffer pages
> Btrfs: skip set_page_dirty if eb pages are already dirty
> Btrfs: remove wait_ordered_range in btrfs_evict_inode
> Btrfs: delayed-refs: use rb_first_cached for href_root
> Btrfs: delayed-refs: use rb_first_cached for ref_tree
> Btrfs: delayed-inode: use rb_first_cached for ins_root and del_root
> Btrfs: extent_map: use rb_first_cached
> Btrfs: preftree: use rb_first_cached
> Btrfs: kill btrfs_clear_path_blocking
>
> Lu Fengqi (10):
> btrfs: simplify the send_in_progress check in btrfs_delete_subvolume
> btrfs: switch update_size to bool in btrfs_block_rsv_migrate and btrfs_rsv_add_bytes
> btrfs: Remove root parameter from btrfs_insert_dir_item
> btrfs: remove a useless return statement in btrfs_block_rsv_add
> btrfs: qgroup: move the qgroup->members check out from (!qgroup)'s else branch
> btrfs: delayed-ref: pass delayed_refs directly to btrfs_select_ref_head
> btrfs: delayed-ref: pass delayed_refs directly to btrfs_delayed_ref_lock
> btrfs: remove fs_info from btrfs_check_space_for_delayed_refs
> btrfs: remove fs_info from btrfs_should_throttle_delayed_refs
> btrfs: switch return_bigger to bool in find_ref_head
>
> Misono Tomohiro (2):
> btrfs: Remove 'objectid' member from struct btrfs_root
> btrfs: remove redundant variable from btrfs_cross_ref_exist
>
> Nikolay Borisov (8):
> btrfs: Make btrfs_find_device_by_path return struct btrfs_device
> btrfs: Make btrfs_find_device_missing_or_by_path return directly a device
> btrfs: Make btrfs_find_device_by_devspec return btrfs_device directly
> btrfs: Remove logically dead code from btrfs_orphan_cleanup
> btrfs: handle error of get_old_root
> btrfs: Factor out ref head locking code in __btrfs_run_delayed_refs
> btrfs: Factor out loop processing all refs of a head
> btrfs: refactor __btrfs_run_delayed_refs loop
>
> Omar Sandoval (2):
> Btrfs: clean up scrub is_dev_replace parameter
> Btrfs: get rid of btrfs_symlink_aops
>
> Qu Wenruo (16):
> btrfs: qgroup: Dirty all qgroups before rescan
> btrfs: Handle owner mismatch gracefully when walking up tree
> btrfs: locking: Add extra check in btrfs_init_new_buffer() to avoid deadlock
> btrfs: Enhance btrfs_trim_fs function to handle error better
> btrfs: Ensure btrfs_trim_fs can trim the whole filesystem
> btrfs: relocation: Add basic extent backref related comments for build_backref_tree
> btrfs: qgroup: Introduce trace event to analyse the number of dirty extents accounted
> btrfs: qgroup: Introduce function to trace two swaped extents
> btrfs: qgroup: Introduce function to find all new tree blocks of reloc tree
> btrfs: qgroup: Use generation-aware subtree swap to mark dirty extents
> btrfs: qgroup: Don't trace subtree if we're dropping reloc tree
> btrfs: qgroup: Only trace data extents in leaves if we're relocating data block group
> btrfs: tree-checker: Check level for leaves and nodes
> btrfs: qgroup: Avoid calling qgroup functions if qgroup is not enabled
> btrfs: relocation: Cleanup while loop using rbtree_postorder_for_each_entry_safe
> btrfs: relocation: Remove redundant tree level check
>
> Su Yue (1):
> btrfs: defrag: use btrfs_mod_outstanding_extents in cluster_pages_for_defrag
>
> zhong jiang (4):
> btrfs: remove unneeded NULL checks before kfree
> btrfs: change btrfs_free_reserved_bytes to return void
> btrfs: change btrfs_pin_log_trans to return void
> btrfs: change remove_extent_mapping to return void
>
> fs/btrfs/backref.c | 39 ++--
> fs/btrfs/btrfs_inode.h | 8 +-
> fs/btrfs/check-integrity.c | 6 +-
> fs/btrfs/compression.c | 2 -
> fs/btrfs/ctree.c | 68 +-----
> fs/btrfs/ctree.h | 56 ++---
> fs/btrfs/delayed-inode.c | 41 ++--
> fs/btrfs/delayed-inode.h | 4 +-
> fs/btrfs/delayed-ref.c | 69 +++---
> fs/btrfs/delayed-ref.h | 10 +-
> fs/btrfs/dev-replace.c | 64 ++----
> fs/btrfs/dev-replace.h | 8 -
> fs/btrfs/dir-item.c | 8 +-
> fs/btrfs/disk-io.c | 24 +-
> fs/btrfs/export.c | 4 +-
> fs/btrfs/extent-tree.c | 424 +++++++++++++++++++++--------------
> fs/btrfs/extent_io.c | 33 ++-
> fs/btrfs/extent_io.h | 4 +-
> fs/btrfs/extent_map.c | 32 +--
> fs/btrfs/extent_map.h | 4 +-
> fs/btrfs/file.c | 33 ++-
> fs/btrfs/free-space-cache.c | 16 +-
> fs/btrfs/inode.c | 120 ++++------
> fs/btrfs/ioctl.c | 18 +-
> fs/btrfs/qgroup.c | 455 ++++++++++++++++++++++++++++++++++++--
> fs/btrfs/qgroup.h | 8 +
> fs/btrfs/ref-verify.c | 8 +-
> fs/btrfs/relocation.c | 74 +++----
> fs/btrfs/scrub.c | 34 ++-
> fs/btrfs/send.c | 24 +-
> fs/btrfs/super.c | 6 +-
> fs/btrfs/tests/extent-io-tests.c | 10 +-
> fs/btrfs/tests/extent-map-tests.c | 4 +-
> fs/btrfs/transaction.c | 31 +--
> fs/btrfs/tree-checker.c | 14 ++
> fs/btrfs/tree-log.c | 86 +++++--
> fs/btrfs/tree-log.h | 2 +-
> fs/btrfs/volumes.c | 117 +++++-----
> fs/btrfs/volumes.h | 9 +-
> include/trace/events/btrfs.h | 36 ++-
> 40 files changed, 1268 insertions(+), 745 deletions(-)
Powered by blists - more mailing lists