Message-Id: <cover.1574439340.git.dsterba@suse.com>
Date:   Fri, 22 Nov 2019 17:23:44 +0100
From:   David Sterba <dsterba@...e.com>
To:     torvalds@...ux-foundation.org
Cc:     David Sterba <dsterba@...e.com>, linux-btrfs@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: [GIT PULL] Btrfs updates for 5.5

Hi,

there are some new features, followed by cleanups. Please pull, thanks.

User visible changes:

- new block group profiles: RAID1 with 3- and 4- copies
  - RAID1 in btrfs has always 2 copies, now add support for 3 and 4
  - this is an incompat feature (named RAID1C34)
  - recommended use of RAID1C3 is replacement of RAID6 profile on metadata,
    this brings a more reliable resiliency against 2 device loss/damage

- support for new checksums
  - per-filesystem, set at mkfs time
  - fast hash (crc32c successor): xxhash, 64bit digest
  - strong hashes (both 256bit): sha256 (slower, FIPS), blake2b (faster)
  - the blake2b module goes via the crypto tree, btrfs.ko has a soft
    dependency

- speed up lseek, don't take inode locks unnecessarily, this can speed up
  parallel SEEK_CUR/SEEK_SET/SEEK_END by 80%

- send:
  - allow clone operations within the same file
  - limit maximum number of sent clone references to avoid slow backref walking

- error message improvements: device scan prints process name and PID

Core changes:

- cleanups
  - remove unique workqueue helpers, used to provide a way to avoid deadlocks
    in the workqueue code, now done in a simpler way
  - remove lots of indirect function calls in compression code
  - extent IO tree code moved out of extent_io.c
  - cleanup backup superblock handling at mount time
  - transaction life cycle documentation and cleanups
  - locking code cleanups, annotations and documentation
  - add more cold, const, pure function attributes
  - removal of unused or redundant struct members or variables

- new tree-checker sanity tests
  - try to detect missing INODE_ITEM, cross-reference checks of DIR_ITEM,
    DIR_INDEX, INODE_REF, and XATTR_* items

- remove own bio scheduling code (used to avoid checksum submissions being
  stuck behind other IO), replaced by cgroup controller-based code to allow
  better control and avoid priority inversions in cases where the custom and
  cgroup scheduling disagreed

Fixes:

- avoid getting stuck during cyclic writebacks

- fix trimming of ranges crossing block group boundaries

- fix rename exchange on subvolumes, all involved subvolumes need to be
  recorded in the transaction

----------------------------------------------------------------
The following changes since commit af42d3466bdc8f39806b26f593604fdc54140bcb:

  Linux 5.4-rc8 (2019-11-17 14:47:30 -0800)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.5-tag

for you to fetch changes up to fa17ed069c61286b26382e23b57a62930657b9c1:

  btrfs: drop bdev argument from submit_extent_page (2019-11-18 23:43:58 +0100)

----------------------------------------------------------------
Anand Jain (3):
      btrfs: balance: use term redundancy instead of integrity in message
      btrfs: print process name and pid that calls device scanning
      btrfs: use bool argument in free_root_pointers()

Chengguang Xu (3):
      btrfs: props: remove unnecessary hash_init()
      btrfs: use enum for extent type defines
      btrfs: use better definition of number of compression type

Chris Mason (5):
      Btrfs: stop using btrfs_schedule_bio()
      Btrfs: delete the entire async bio submission framework
      Btrfs: only associate the locked page with one async_chunk struct
      Btrfs: use REQ_CGROUP_PUNT for worker thread submitted bios
      Btrfs: extent_write_locked_range() should attach inode->i_wb

Dan Carpenter (1):
      btrfs: clean up locking name in scrub_enumerate_chunks()

David Sterba (54):
      btrfs: make locking assertion helpers static inline
      btrfs: make btrfs_assert_tree_locked static inline
      btrfs: move btrfs_set_path_blocking to other locking functions
      btrfs: move btrfs_unlock_up_safe to other locking functions
      btrfs: add 64bit safe helper for power of two checks
      btrfs: use has_single_bit_set for clarity
      btrfs: drop unused parameter is_new from btrfs_iget
      btrfs: add __cold attribute to more functions
      btrfs: add const function attribute
      btrfs: add __pure attribute to functions
      btrfs: opencode extent_buffer_get
      btrfs: export compression and decompression callbacks
      btrfs: switch compression callbacks to direct calls
      btrfs: compression: attach workspace manager to the ops
      btrfs: compression: let workspace manager init take only the type
      btrfs: compression: inline init_workspace_manager
      btrfs: compression: let workspace manager cleanup take only the type
      btrfs: compression: inline cleanup_workspace_manager
      btrfs: compression: export alloc/free/get/put callbacks of all algos
      btrfs: compression: inline get_workspace
      btrfs: compression: inline put_workspace
      btrfs: compression: pass type to btrfs_get_workspace
      btrfs: compression: inline alloc_workspace
      btrfs: compression: pass type to btrfs_put_workspace
      btrfs: compression: inline free_workspace
      btrfs: compression: remove ops pointer from workspace_manager
      btrfs: tracepoints: drop typecasts from printk
      btrfs: tracepoints: constify all pointers
      btrfs: assert extent_map bdevs and lookup_map and split
      btrfs: get bdev from latest_dev for dio bh_result
      btrfs: sysfs: export supported checksums
      btrfs: add member for a specific checksum driver
      btrfs: add blake2b to checksumming algorithms
      btrfs: move block_group_item::used to block group
      btrfs: move block_group_item::flags to block group
      btrfs: remove embedded block_group_cache::item
      btrfs: rename block_group_item on-stack accessors to follow naming
      btrfs: rename extent buffer block group item accessors
      btrfs: add dedicated members for start and length of a block group
      btrfs: sink write_flags to __extent_writepage_io
      btrfs: sink write flags to cow_file_range_async
      btrfs: add support for 3-copy replication (raid1c3)
      btrfs: add support for 4-copy replication (raid1c4)
      btrfs: add incompat for raid1 with 3, 4 copies
      btrfs: drop incompat bit for raid1c34 after last block group is gone
      btrfs: merge blocking_writers branches in btrfs_tree_read_lock
      btrfs: set blocking_writers directly, no increment or decrement
      btrfs: access eb::blocking_writers according to ACCESS_ONCE policies
      btrfs: document extent buffer locking
      btrfs: rename btrfs_block_group_cache
      btrfs: get bdev directly from fs_devices in submit_extent_page
      btrfs: drop bio_set_dev where not needed
      btrfs: remove extent_map::bdev
      btrfs: drop bdev argument from submit_extent_page

Filipe Manana (8):
      Btrfs: make btrfs_wait_extents() static
      Btrfs: fix negative subv_writers counter and data space leak after buffered write
      Btrfs: fix metadata space leak on fixup worker failure to set range as delalloc
      Btrfs: remove wait queue from space_info structure
      Btrfs: remove unnecessary delalloc mutex for inodes
      Btrfs: send, allow clone operations within the same file
      Btrfs: send, skip backreference walking for extents with many references
      Btrfs: fix block group remaining RO forever after error during device replace

Goldwyn Rodrigues (1):
      btrfs: simplify inode locking for RWF_NOWAIT

Johannes Thumshirn (10):
      btrfs: raid56: reduce indentation in lock_stripe_add
      btrfs: remove pointless local variable in lock_stripe_add()
      btrfs: reduce indentation in btrfs_may_alloc_data_chunk
      btrfs: remove pointless indentation in btrfs_read_sys_array()
      btrfs: add xxhash64 to checksumming algorithms
      btrfs: add sha256 to checksumming algorithm
      btrfs: sysfs: show used checksum driver per filesystem
      btrfs: remove cached space_info in btrfs_statfs()
      btrfs: change btrfs_fs_devices::seeding to bool
      btrfs: change btrfs_fs_devices::rotating to bool

Josef Bacik (9):
      btrfs: separate out the extent leak code
      btrfs: separate out the extent io init function
      btrfs: move extent_io_tree defs to their own header
      btrfs: export find_delalloc_range
      btrfs: move the failrec tree stuff into extent-io-tree.h
      btrfs: use refcount_inc_not_zero in kill_all_nodes
      btrfs: check page->mapping when loading free space cache
      btrfs: use btrfs_block_group_cache_done in update_block_group
      btrfs: record all roots for rename exchange on a subvol

Marcos Paulo de Souza (2):
      btrfs: block-group: Rework documentation of check_system_chunk function
      btrfs: ioctl: Try to use btrfs_fs_info instead of *file

Nikolay Borisov (17):
      btrfs: Add assert to catch nested transaction commit
      btrfs: Don't opencode btrfs_find_name_in_backref in backref_in_log
      btrfs: Properly handle backref_in_log retval
      btrfs: Open-code name_in_log_ref in replay_one_name
      btrfs: User assert to document transaction requirement
      btrfs: Rename btrfs_join_transaction_nolock
      btrfs: Speed up btrfs_file_llseek
      btrfs: Simplify btrfs_file_llseek
      btrfs: Return offset from find_desired_extent
      btrfs: Cleanup and simplify find_newest_super_backup
      btrfs: Remove newest_gen argument from find_oldest_super_backup
      btrfs: Add read_backup_root
      btrfs: Factor out tree roots initialization during mount
      btrfs: Don't use objectid_mutex during mount
      btrfs: Remove unused next_root_backup function
      btrfs: Rename find_oldest_super_backup to init_backup_root_slot
      btrfs: Streamline btrfs_fs_info::backup_root_index semantics

Omar Sandoval (7):
      btrfs: get rid of unnecessary memset() of work item
      btrfs: don't prematurely free work in run_ordered_work()
      btrfs: don't prematurely free work in end_workqueue_fn()
      btrfs: don't prematurely free work in reada_start_machine_worker()
      btrfs: don't prematurely free work in scrub_missing_raid56_worker()
      btrfs: get rid of unique workqueue helper functions
      btrfs: get rid of pointless wtag variable in async-thread.c

Qu Wenruo (16):
      btrfs: tree-checker: Try to detect missing INODE_ITEM
      btrfs: tree-checker: Add check for INODE_REF
      btrfs: ctree: Reduce one indent level for btrfs_search_slot()
      btrfs: ctree: Reduce one indent level for btrfs_search_old_slot()
      btrfs: ctree: Remove stray comment of setting up path lock
      btrfs: transaction: describe transaction states and transitions
      btrfs: transaction: Cleanup unused TRANS_STATE_BLOCKED
      btrfs: tree-checker: Refactor prev_key check for ino into a function
      btrfs: Enhance error output for write time tree checker
      btrfs: Remove btrfs_bio::flags member
      btrfs: tree-checker: Check item size before reading file extent type
      btrfs: volumes: Use more straightforward way to calculate map length
      btrfs: Ensure we trim ranges across block group boundary
      btrfs: block-group: Refactor btrfs_read_block_groups()
      btrfs: block-group: Reuse the item key from caller of read_one_block_group()
      btrfs: scrub: Don't check free space before marking a block group RO

Tejun Heo (1):
      btrfs: Avoid getting stuck during cyclic writebacks

 fs/btrfs/Kconfig                       |   2 +
 fs/btrfs/async-thread.c                | 113 +++----
 fs/btrfs/async-thread.h                |  37 +--
 fs/btrfs/block-group.c                 | 589 +++++++++++++++++----------------
 fs/btrfs/block-group.h                 |  51 +--
 fs/btrfs/btrfs_inode.h                 |   3 -
 fs/btrfs/compression.c                 | 269 +++++++++++----
 fs/btrfs/compression.h                 |  46 +--
 fs/btrfs/ctree.c                       | 287 +++++++---------
 fs/btrfs/ctree.h                       |  51 +--
 fs/btrfs/delalloc-space.c              |  21 +-
 fs/btrfs/delayed-inode.c               |  18 +-
 fs/btrfs/dev-replace.c                 |   2 +-
 fs/btrfs/dev-replace.h                 |   2 +-
 fs/btrfs/disk-io.c                     | 365 ++++++++++----------
 fs/btrfs/disk-io.h                     |   4 +-
 fs/btrfs/export.c                      |   4 +-
 fs/btrfs/extent-io-tree.h              | 248 ++++++++++++++
 fs/btrfs/extent-tree.c                 | 146 ++++----
 fs/btrfs/extent_io.c                   | 120 ++++---
 fs/btrfs/extent_io.h                   | 231 +------------
 fs/btrfs/extent_map.c                  |   6 +-
 fs/btrfs/extent_map.h                  |  11 +-
 fs/btrfs/file-item.c                   |   1 -
 fs/btrfs/file.c                        |  74 ++---
 fs/btrfs/free-space-cache.c            | 118 +++----
 fs/btrfs/free-space-cache.h            |  39 +--
 fs/btrfs/free-space-tree.c             | 133 ++++----
 fs/btrfs/free-space-tree.h             |  18 +-
 fs/btrfs/inode.c                       | 170 ++++++----
 fs/btrfs/ioctl.c                       |  49 ++-
 fs/btrfs/locking.c                     | 309 ++++++++++++++---
 fs/btrfs/locking.h                     |  13 +-
 fs/btrfs/lzo.c                         |  53 +--
 fs/btrfs/misc.h                        |  11 +
 fs/btrfs/ordered-data.c                |   7 +-
 fs/btrfs/ordered-data.h                |   2 +-
 fs/btrfs/print-tree.c                  |   6 +-
 fs/btrfs/props.c                       |   6 +-
 fs/btrfs/qgroup.c                      |  11 +-
 fs/btrfs/qgroup.h                      |   2 +-
 fs/btrfs/raid56.c                      | 101 +++---
 fs/btrfs/reada.c                       |  19 +-
 fs/btrfs/relocation.c                  |  43 ++-
 fs/btrfs/scrub.c                       | 100 +++---
 fs/btrfs/send.c                        |  45 ++-
 fs/btrfs/space-info.c                  |   8 +-
 fs/btrfs/space-info.h                  |   3 +-
 fs/btrfs/super.c                       |  26 +-
 fs/btrfs/sysfs.c                       |  47 ++-
 fs/btrfs/sysfs.h                       |   2 +-
 fs/btrfs/tests/btrfs-tests.c           |  11 +-
 fs/btrfs/tests/btrfs-tests.h           |   4 +-
 fs/btrfs/tests/free-space-tests.c      |  15 +-
 fs/btrfs/tests/free-space-tree-tests.c | 101 +++---
 fs/btrfs/transaction.c                 |  98 +++++-
 fs/btrfs/transaction.h                 |   5 +-
 fs/btrfs/tree-checker.c                | 211 +++++++++---
 fs/btrfs/tree-log.c                    | 136 ++++----
 fs/btrfs/volumes.c                     | 494 ++++++++-------------------
 fs/btrfs/volumes.h                     |  24 +-
 fs/btrfs/zlib.c                        |  52 +--
 fs/btrfs/zstd.c                        |  47 +--
 include/trace/events/btrfs.h           | 131 ++++----
 include/uapi/linux/btrfs.h             |   5 +-
 include/uapi/linux/btrfs_tree.h        |  23 +-
 66 files changed, 2780 insertions(+), 2619 deletions(-)
 create mode 100644 fs/btrfs/extent-io-tree.h

Powered by blists - more mailing lists