linux-kernel - [PATCH 0/12 v3] Writeback improvements

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 27 Sep 2017 14:13:47 -0600
From:   Jens Axboe <axboe@...nel.dk>
To:     linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org
Cc:     hannes@...xchg.org, jack@...e.cz, torvalds@...ux-foundation.org
Subject: [PATCH 0/12 v3] Writeback improvements

We've had some issues with writeback in presence of memory reclaim
at Facebook, and this patch set attempts to fix it up. The real
functional change for that issue is patch 10. The rest are cleanups,
as well as the removal of doing non-range cyclic writeback. The users
of that was sync_inodes_sb() and wakeup_flusher_threads(), both of
which writeback all of the dirty pages.

The basic idea is that we have callers that call
wakeup_flusher_threads() with nr_pages == 0. This means 'writeback
everything'. For memory reclaim situations, we can end up queuing
a TON of these kinds of writeback units. This can cause softlockups
and further memory issues, since we allocate huge amounts of
struct wb_writeback_work to handle this writeback. Handle this
situation more gracefully.


 drivers/md/bitmap.c                      |    2 
 drivers/staging/lustre/lustre/llite/rw.c |   25 ++-----
 fs/afs/write.c                           |   25 +------
 fs/btrfs/extent_io.c                     |   31 ++-------
 fs/buffer.c                              |   60 +++---------------
 fs/ceph/addr.c                           |   26 ++-----
 fs/cifs/file.c                           |   20 +-----
 fs/ext4/inode.c                          |   26 +++----
 fs/f2fs/data.c                           |   26 ++-----
 fs/fs-writeback.c                        |  103 +++++++++++++++++++------------
 fs/gfs2/aops.c                           |   27 ++------
 fs/ntfs/aops.c                           |    2 
 fs/ntfs/mft.c                            |    2 
 fs/sync.c                                |    2 
 include/linux/backing-dev-defs.h         |    1 
 include/linux/backing-dev.h              |    2 
 include/linux/buffer_head.h              |    2 
 include/linux/writeback.h                |    5 -
 include/trace/events/btrfs.h             |    2 
 include/trace/events/ext4.h              |    2 
 include/trace/events/f2fs.h              |    2 
 include/trace/events/writeback.h         |    4 -
 mm/page-writeback.c                      |   44 ++-----------
 mm/vmscan.c                              |    2 
 24 files changed, 159 insertions(+), 284 deletions(-)


Changes since v2:

- Removal of non-range_cyclic writeback.
- Cleanup of the buffer.c failure handling code, utilize
  __GFP_NOFAIL instead of rolling our own.
- Reinstate cyclic writeback for laptop mode, it's now the only
  option available.
- Rebased on top of master, and series shuffled around.

Changes since v1:

- Rename WB_zero_pages to WB_start_all (Amir).
- Remove a test_bit() for a condition where we always expect the bit
  to be set.
- Remove 'nr_pages' from the wakeup flusher threads helpers, since
  everybody now passes in zero. Enables further cleanups in later
  patches too (Jan).
- Fix a case where I forgot to clear WB_start_all if 'work' allocation
  failed.
- Get rid of cond_resched() in the wb_do_writeback() loop.

-- 
Jens Axboe