Message-ID: <20231109154004.3317227-1-dhowells@redhat.com>
Date:   Thu,  9 Nov 2023 15:39:23 +0000
From:   David Howells <dhowells@...hat.com>
To:     Marc Dionne <marc.dionne@...istor.com>
Cc:     David Howells <dhowells@...hat.com>, linux-afs@...ts.infradead.org,
        linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: [PATCH 00/41] afs: Fix probe handling, server rotation and RO volume callback handling

Hi Marc,

Here are a set of patches to make some substantial fixes to the afs
filesystem including:

 (1) Fix fileserver probe handling so that the next round of probes doesn't
     break ongoing server/address rotation by clearing all the probe result
     tracking.  Instead, detach the probe results into a separate struct
     and allocate a new one each time we start probing and update the
     pointer to it.  Probes are also sent in order of address preference to
     try and improve the chance that the preferred one will complete first.

 (2) Fix server rotation so that it uses configurable address preferences
     across on the probes that have completed so far than ranking them by
     RTT as the latter doesn't necessarily give the best route.  The
     preference list can be altered by echoing commands into
     /proc/net/afs/addr_prefs.

 (3) Fix the handling of Read-Only (and Backup) volume callbacks as there
     is one per volume, not one per file, so if someone performs a command
     that, say, offlines the volume but doesn't change it, when it comes
     back online we don't spam the server with a status fetch for every
     vnode we're using.  Instead, check the Creation timestamp in the
     VolSync record when prompted by a callback break.

 (4) Handle volume regression (ie. a RW volume being restored from a
     backup) by scrubbing all cache data for that volume.  This is detected
     from the VolSync creation timestamp.

 (5) Adjust abort handling and abort -> error mapping to match better with
     what other AFS clients do.

 (6) Handle file locking locally on RO volumes rather than trying to get
     them on the server.  Exclusive locks aren't really handled in a RO
     volume.

 (7) Set RO volumes to be RO superblocks.

 (8) Fix offline and busy volume state handling as they only apply to
     individual server instances and not entire volumes and the rotation
     algorithm should go and look at other servers if available.  Also make
     it sleep briefly before each retry if all the volume instances are
     unavailable.

In addition there are a number of small fixes in rxrpc and afs included
here so that those problems don't affect testing.

The patches can be found here:

	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=afs-fixes

Thanks,
David

---
%(shortlog)s
%(diffstat)s

David Howells (41):
  rxrpc: Fix RTT determination to use PING ACKs as a source
  rxrpc: Fix two connection reaping bugs
  rxrpc: Fix some minor issues with bundle tracing
  afs: Fix afs_server_list to be cleaned up with RCU
  afs: Make error on cell lookup failure consistent with OpenAFS
  afs: Remove whitespace before most ')' from the trace header
  afs: Automatically generate trace tag enums
  afs: Add comments on abort handling
  afs: Turn the afs_addr_list address array into an array of structs
  rxrpc, afs: Allow afs to pin rxrpc_peer objects
  afs: Don't skip server addresses for which we didn't get an RTT
    reading
  afs: Rename addr_list::failed to probe_failed
  afs: Handle the VIO abort explicitly
  afs: Use op->nr_iterations=-1 to indicate to begin fileserver
    iteration
  afs: Return ENOENT if no cell DNS record can be found
  afs: Wrap most op->error accesses with inline funcs
  afs: Don't put afs_call in afs_wait_for_call_to_complete()
  afs: Simplify error handling
  afs: Add a tracepoint for struct afs_addr_list
  afs: Rename some fields
  afs: Use peer + service_id as call address
  afs: Fold the afs_addr_cursor struct in
  rxrpc: Create a procfile to display outstanding clien conn bundles
  afs: Add some more info to /proc/net/afs/servers
  afs: Remove the unimplemented afs_cmp_addr_list()
  afs: Provide a way to configure address priorities
  afs: Mark address lists with configured priorities
  afs: Dispatch fileserver probes in priority order
  afs: Dispatch vlserver probes in priority order
  afs: Keep a record of the current fileserver endpoint state
  afs: Combine the endpoint state bools into a bitmask
  afs: Fix file locking on R/O volumes to operate in local mode
  afs: Mark a superblock for an R/O or Backup volume as SB_RDONLY
  afs: Make it possible to find the volumes that are using a server
  afs: Defer volume record destruction to a workqueue
  afs: Move the vnode/volume validity checking code into its own file
  afs: Apply server breaks to mmap'd files in the call processor
  afs: Parse the VolSync record in the reply of a number of RPC ops
  afs: Overhaul invalidation handling to better support RO volumes
  afs: Fix fileserver rotation
  afs: Fix offline and busy handling

 fs/afs/Makefile              |   2 +
 fs/afs/addr_list.c           | 224 +++++------
 fs/afs/addr_prefs.c          | 531 +++++++++++++++++++++++++
 fs/afs/afs.h                 |   4 +
 fs/afs/callback.c            | 138 ++++---
 fs/afs/cell.c                |   5 +-
 fs/afs/cmservice.c           |   5 +-
 fs/afs/dir.c                 |  55 +--
 fs/afs/dir_silly.c           |   2 +-
 fs/afs/dynroot.c             |   4 +-
 fs/afs/file.c                |  20 +-
 fs/afs/fs_operation.c        |  86 ++--
 fs/afs/fs_probe.c            | 323 ++++++++-------
 fs/afs/fsclient.c            |  56 ++-
 fs/afs/inode.c               | 198 +---------
 fs/afs/internal.h            | 355 +++++++++++------
 fs/afs/main.c                |   1 +
 fs/afs/misc.c                |  10 +-
 fs/afs/proc.c                | 102 ++++-
 fs/afs/rotate.c              | 495 ++++++++++++++++-------
 fs/afs/rxrpc.c               | 107 ++---
 fs/afs/server.c              | 130 +++---
 fs/afs/server_list.c         | 126 +++++-
 fs/afs/super.c               |  11 +-
 fs/afs/validation.c          | 376 ++++++++++++++++++
 fs/afs/vl_alias.c            |  69 +---
 fs/afs/vl_list.c             |  29 +-
 fs/afs/vl_probe.c            |  60 ++-
 fs/afs/vl_rotate.c           | 223 +++++++----
 fs/afs/vlclient.c            | 124 ++++--
 fs/afs/volume.c              |  66 +++-
 fs/afs/write.c               |   6 +-
 fs/afs/yfsclient.c           |   6 +-
 include/net/af_rxrpc.h       |  15 +-
 include/trace/events/afs.h   | 742 +++++++++++++++++++----------------
 include/trace/events/rxrpc.h |   3 +
 net/rxrpc/af_rxrpc.c         |  62 ++-
 net/rxrpc/ar-internal.h      |   6 +-
 net/rxrpc/call_object.c      |  17 +-
 net/rxrpc/conn_client.c      |  17 +-
 net/rxrpc/conn_object.c      |   2 +-
 net/rxrpc/input.c            |   4 +
 net/rxrpc/local_object.c     |   2 +-
 net/rxrpc/net_ns.c           |   4 +
 net/rxrpc/peer_object.c      |  56 ++-
 net/rxrpc/proc.c             |  76 ++++
 net/rxrpc/sendmsg.c          |  11 +-
 47 files changed, 3304 insertions(+), 1662 deletions(-)
 create mode 100644 fs/afs/addr_prefs.c
 create mode 100644 fs/afs/validation.c

Powered by blists - more mailing lists