lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Mon, 22 Apr 2019 10:27:57 -0300
From:   Mauro Carvalho Chehab <mchehab+samsung@...nel.org>
To:     Linux Doc Mailing List <linux-doc@...r.kernel.org>
Cc:     Mauro Carvalho Chehab <mchehab+samsung@...nel.org>,
        Mauro Carvalho Chehab <mchehab@...radead.org>,
        linux-kernel@...r.kernel.org, Jonathan Corbet <corbet@....net>
Subject: [PATCH v2 68/79] docs: md: convert to ReST

Rename the md documentation files to ReST, add an
index for them and adjust in order to produce a nice html
output via the Sphinx build system.

At its new index.rst, let's add a :orphan: while this is not linked to
the main index.rst file, in order to avoid build warnings.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@...nel.org>
---
 Documentation/md/index.rst                    |  12 ++
 .../md/{md-cluster.txt => md-cluster.rst}     | 188 ++++++++++++------
 .../md/{raid5-cache.txt => raid5-cache.rst}   |  28 +--
 .../md/{raid5-ppl.txt => raid5-ppl.rst}       |   2 +
 4 files changed, 153 insertions(+), 77 deletions(-)
 create mode 100644 Documentation/md/index.rst
 rename Documentation/md/{md-cluster.txt => md-cluster.rst} (68%)
 rename Documentation/md/{raid5-cache.txt => raid5-cache.rst} (92%)
 rename Documentation/md/{raid5-ppl.txt => raid5-ppl.rst} (98%)

diff --git a/Documentation/md/index.rst b/Documentation/md/index.rst
new file mode 100644
index 000000000000..c4db34ed327d
--- /dev/null
+++ b/Documentation/md/index.rst
@@ -0,0 +1,12 @@
+:orphan:
+
+====
+RAID
+====
+
+.. toctree::
+   :maxdepth: 1
+
+   md-cluster
+   raid5-cache
+   raid5-ppl
diff --git a/Documentation/md/md-cluster.txt b/Documentation/md/md-cluster.rst
similarity index 68%
rename from Documentation/md/md-cluster.txt
rename to Documentation/md/md-cluster.rst
index e1055f105cf5..96eb52cec7eb 100644
--- a/Documentation/md/md-cluster.txt
+++ b/Documentation/md/md-cluster.rst
@@ -1,19 +1,24 @@
+==========
+MD Cluster
+==========
+
 The cluster MD is a shared-device RAID for a cluster, it supports
 two levels: raid1 and raid10 (limited support).
 
 
 1. On-disk format
+=================
 
 Separate write-intent-bitmaps are used for each cluster node.
 The bitmaps record all writes that may have been started on that node,
-and may not yet have finished. The on-disk layout is:
+and may not yet have finished. The on-disk layout is::
 
-0                    4k                     8k                    12k
--------------------------------------------------------------------
-| idle                | md super            | bm super [0] + bits |
-| bm bits[0, contd]   | bm super[1] + bits  | bm bits[1, contd]   |
-| bm super[2] + bits  | bm bits [2, contd]  | bm super[3] + bits  |
-| bm bits [3, contd]  |                     |                     |
+  0                    4k                     8k                    12k
+  -------------------------------------------------------------------
+  | idle                | md super            | bm super [0] + bits |
+  | bm bits[0, contd]   | bm super[1] + bits  | bm bits[1, contd]   |
+  | bm super[2] + bits  | bm bits [2, contd]  | bm super[3] + bits  |
+  | bm bits [3, contd]  |                     |                     |
 
 During "normal" functioning we assume the filesystem ensures that only
 one node writes to any given block at a time, so a write request will
@@ -28,10 +33,12 @@ node) is writing.
 
 
 2. DLM Locks for management
+===========================
 
 There are three groups of locks for managing the device:
 
 2.1 Bitmap lock resource (bm_lockres)
+-------------------------------------
 
  The bm_lockres protects individual node bitmaps. They are named in
  the form bitmap000 for node 1, bitmap001 for node 2 and so on. When a
@@ -48,6 +55,7 @@ There are three groups of locks for managing the device:
  joins the cluster.
 
 2.2 Message passing locks
+-------------------------
 
  Each node has to communicate with other nodes when starting or ending
  resync, and for metadata superblock updates.  This communication is
@@ -55,116 +63,155 @@ There are three groups of locks for managing the device:
  with the Lock Value Block (LVB) of one of the "message" lock.
 
 2.3 new-device management
+-------------------------
 
  A single lock: "no-new-dev" is used to co-ordinate the addition of
  new devices - this must be synchronized across the array.
  Normally all nodes hold a concurrent-read lock on this device.
 
 3. Communication
+================
 
  Messages can be broadcast to all nodes, and the sender waits for all
  other nodes to acknowledge the message before proceeding.  Only one
  message can be processed at a time.
 
 3.1 Message Types
+-----------------
 
  There are six types of messages which are passed:
 
- 3.1.1 METADATA_UPDATED: informs other nodes that the metadata has
+3.1.1 METADATA_UPDATED
+^^^^^^^^^^^^^^^^^^^^^^
+
+   informs other nodes that the metadata has
    been updated, and the node must re-read the md superblock. This is
    performed synchronously. It is primarily used to signal device
    failure.
 
- 3.1.2 RESYNCING: informs other nodes that a resync is initiated or
+3.1.2 RESYNCING
+^^^^^^^^^^^^^^^
+   informs other nodes that a resync is initiated or
    ended so that each node may suspend or resume the region.  Each
    RESYNCING message identifies a range of the devices that the
    sending node is about to resync. This overrides any previous
    notification from that node: only one ranged can be resynced at a
    time per-node.
 
- 3.1.3 NEWDISK: informs other nodes that a device is being added to
+3.1.3 NEWDISK
+^^^^^^^^^^^^^
+
+   informs other nodes that a device is being added to
    the array. Message contains an identifier for that device.  See
    below for further details.
 
- 3.1.4 REMOVE: A failed or spare device is being removed from the
+3.1.4 REMOVE
+^^^^^^^^^^^^
+
+   A failed or spare device is being removed from the
    array. The slot-number of the device is included in the message.
 
- 3.1.5 RE_ADD: A failed device is being re-activated - the assumption
+ 3.1.5 RE_ADD:
+
+   A failed device is being re-activated - the assumption
    is that it has been determined to be working again.
 
- 3.1.6 BITMAP_NEEDS_SYNC: if a node is stopped locally but the bitmap
+ 3.1.6 BITMAP_NEEDS_SYNC:
+
+   If a node is stopped locally but the bitmap
    isn't clean, then another node is informed to take the ownership of
    resync.
 
 3.2 Communication mechanism
+---------------------------
 
  The DLM LVB is used to communicate within nodes of the cluster. There
  are three resources used for the purpose:
 
-  3.2.1 token: The resource which protects the entire communication
+3.2.1 token
+^^^^^^^^^^^
+   The resource which protects the entire communication
    system. The node having the token resource is allowed to
    communicate.
 
-  3.2.2 message: The lock resource which carries the data to
-   communicate.
+3.2.2 message
+^^^^^^^^^^^^^
+   The lock resource which carries the data to communicate.
 
-  3.2.3 ack: The resource, acquiring which means the message has been
+3.2.3 ack
+^^^^^^^^^
+
+   The resource, acquiring which means the message has been
    acknowledged by all nodes in the cluster. The BAST of the resource
    is used to inform the receiving node that a node wants to
    communicate.
 
 The algorithm is:
 
- 1. receive status - all nodes have concurrent-reader lock on "ack".
+ 1. receive status - all nodes have concurrent-reader lock on "ack"::
 
-   sender                         receiver                 receiver
-   "ack":CR                       "ack":CR                 "ack":CR
+	sender                         receiver                 receiver
+	"ack":CR                       "ack":CR                 "ack":CR
 
- 2. sender get EX on "token"
-    sender get EX on "message"
-    sender                        receiver                 receiver
-    "token":EX                    "ack":CR                 "ack":CR
-    "message":EX
-    "ack":CR
+ 2. sender get EX on "token",
+    sender get EX on "message"::
+
+	sender                        receiver                 receiver
+	"token":EX                    "ack":CR                 "ack":CR
+	"message":EX
+	"ack":CR
 
     Sender checks that it still needs to send a message. Messages
     received or other events that happened while waiting for the
     "token" may have made this message inappropriate or redundant.
 
- 3. sender writes LVB.
+ 3. sender writes LVB
+
     sender down-convert "message" from EX to CW
+
     sender try to get EX of "ack"
-    [ wait until all receivers have *processed* the "message" ]
-
-                                     [ triggered by bast of "ack" ]
-                                     receiver get CR on "message"
-                                     receiver read LVB
-                                     receiver processes the message
-                                     [ wait finish ]
-                                     receiver releases "ack"
-                                     receiver tries to get PR on "message"
-
-   sender                         receiver                  receiver
-   "token":EX                     "message":CR              "message":CR
-   "message":CW
-   "ack":EX
+
+    ::
+
+      [ wait until all receivers have *processed* the "message" ]
+
+                                       [ triggered by bast of "ack" ]
+                                       receiver get CR on "message"
+                                       receiver read LVB
+                                       receiver processes the message
+                                       [ wait finish ]
+                                       receiver releases "ack"
+                                       receiver tries to get PR on "message"
+
+     sender                         receiver                  receiver
+     "token":EX                     "message":CR              "message":CR
+     "message":CW
+     "ack":EX
 
  4. triggered by grant of EX on "ack" (indicating all receivers
     have processed message)
+
     sender down-converts "ack" from EX to CR
+
     sender releases "message"
+
     sender releases "token"
-                               receiver upconvert to PR on "message"
-                               receiver get CR of "ack"
-                               receiver release "message"
 
-   sender                      receiver                   receiver
-   "ack":CR                    "ack":CR                   "ack":CR
+    ::
+
+                                 receiver upconvert to PR on "message"
+                                 receiver get CR of "ack"
+                                 receiver release "message"
+
+     sender                      receiver                   receiver
+     "ack":CR                    "ack":CR                   "ack":CR
 
 
 4. Handling Failures
+====================
 
 4.1 Node Failure
+----------------
 
  When a node fails, the DLM informs the cluster with the slot
  number. The node starts a cluster recovery thread. The cluster
@@ -177,11 +224,11 @@ The algorithm is:
 	- cleans the bitmap of the failed node
 	- releases bitmap<number> lock of the failed node
 	- initiates resync of the bitmap on the current node
-		md_check_recovery is invoked within recover_bitmaps,
-		then md_check_recovery -> metadata_update_start/finish,
-		it will lock the communication by lock_comm.
-		Which means when one node is resyncing it blocks all
-		other nodes from writing anywhere on the array.
+	  md_check_recovery is invoked within recover_bitmaps,
+	  then md_check_recovery -> metadata_update_start/finish,
+	  it will lock the communication by lock_comm.
+	  Which means when one node is resyncing it blocks all
+	  other nodes from writing anywhere on the array.
 
  The resync process is the regular md resync. However, in a clustered
  environment when a resync is performed, it needs to tell other nodes
@@ -198,6 +245,7 @@ The algorithm is:
  particular I/O range should be suspended or not.
 
 4.2 Device Failure
+==================
 
  Device failures are handled and communicated with the metadata update
  routine.  When a node detects a device failure it does not allow
@@ -205,38 +253,41 @@ The algorithm is:
  acknowledged by all other nodes.
 
 5. Adding a new Device
+----------------------
 
  For adding a new device, it is necessary that all nodes "see" the new
  device to be added. For this, the following algorithm is used:
 
-    1. Node 1 issues mdadm --manage /dev/mdX --add /dev/sdYY which issues
+   1.  Node 1 issues mdadm --manage /dev/mdX --add /dev/sdYY which issues
        ioctl(ADD_NEW_DISK with disc.state set to MD_DISK_CLUSTER_ADD)
-    2. Node 1 sends a NEWDISK message with uuid and slot number
-    3. Other nodes issue kobject_uevent_env with uuid and slot number
+   2.  Node 1 sends a NEWDISK message with uuid and slot number
+   3.  Other nodes issue kobject_uevent_env with uuid and slot number
        (Steps 4,5 could be a udev rule)
-    4. In userspace, the node searches for the disk, perhaps
+   4.  In userspace, the node searches for the disk, perhaps
        using blkid -t SUB_UUID=""
-    5. Other nodes issue either of the following depending on whether
+   5.  Other nodes issue either of the following depending on whether
        the disk was found:
        ioctl(ADD_NEW_DISK with disc.state set to MD_DISK_CANDIDATE and
-             disc.number set to slot number)
+       disc.number set to slot number)
        ioctl(CLUSTERED_DISK_NACK)
-    6. Other nodes drop lock on "no-new-devs" (CR) if device is found
-    7. Node 1 attempts EX lock on "no-new-dev"
-    8. If node 1 gets the lock, it sends METADATA_UPDATED after
+   6.  Other nodes drop lock on "no-new-devs" (CR) if device is found
+   7.  Node 1 attempts EX lock on "no-new-dev"
+   8.  If node 1 gets the lock, it sends METADATA_UPDATED after
        unmarking the disk as SpareLocal
-    9. If not (get "no-new-dev" lock), it fails the operation and sends
+   9.  If not (get "no-new-dev" lock), it fails the operation and sends
        METADATA_UPDATED.
    10. Other nodes get the information whether a disk is added or not
        by the following METADATA_UPDATED.
 
-6. Module interface.
+6. Module interface
+===================
 
  There are 17 call-backs which the md core can make to the cluster
  module.  Understanding these can give a good overview of the whole
  process.
 
 6.1 join(nodes) and leave()
+---------------------------
 
  These are called when an array is started with a clustered bitmap,
  and when the array is stopped.  join() ensures the cluster is
@@ -244,11 +295,13 @@ The algorithm is:
  Only the first 'nodes' nodes in the cluster can use the array.
 
 6.2 slot_number()
+-----------------
 
  Reports the slot number advised by the cluster infrastructure.
  Range is from 0 to nodes-1.
 
 6.3 resync_info_update()
+------------------------
 
  This updates the resync range that is stored in the bitmap lock.
  The starting point is updated as the resync progresses.  The
@@ -256,6 +309,7 @@ The algorithm is:
  It does *not* send a RESYNCING message.
 
 6.4 resync_start(), resync_finish()
+-----------------------------------
 
  These are called when resync/recovery/reshape starts or stops.
  They update the resyncing range in the bitmap lock and also
@@ -265,8 +319,8 @@ The algorithm is:
  resync_finish() also sends a BITMAP_NEEDS_SYNC message which
  allows some other node to take over.
 
-6.5 metadata_update_start(), metadata_update_finish(),
-    metadata_update_cancel().
+6.5 metadata_update_start(), metadata_update_finish(), metadata_update_cancel()
+-------------------------------------------------------------------------------
 
  metadata_update_start is used to get exclusive access to
  the metadata.  If a change is still needed once that access is
@@ -275,6 +329,7 @@ The algorithm is:
  can be used to release the lock.
 
 6.6 area_resyncing()
+--------------------
 
  This combines two elements of functionality.
 
@@ -289,6 +344,7 @@ The algorithm is:
  a node failure.
 
 6.7 add_new_disk_start(), add_new_disk_finish(), new_disk_ack()
+---------------------------------------------------------------
 
  These are used to manage the new-disk protocol described above.
  When a new device is added, add_new_disk_start() is called before
@@ -300,17 +356,20 @@ The algorithm is:
  new_disk_ack() is called.
 
 6.8 remove_disk()
+-----------------
 
  This is called when a spare or failed device is removed from
  the array.  It causes a REMOVE message to be send to other nodes.
 
 6.9 gather_bitmaps()
+--------------------
 
  This sends a RE_ADD message to all other nodes and then
  gathers bitmap information from all bitmaps.  This combined
  bitmap is then used to recovery the re-added device.
 
 6.10 lock_all_bitmaps() and unlock_all_bitmaps()
+------------------------------------------------
 
  These are called when change bitmap to none. If a node plans
  to clear the cluster raid's bitmap, it need to make sure no other
@@ -319,6 +378,7 @@ The algorithm is:
  accordingly.
 
 7. Unsupported features
+=======================
 
 There are somethings which are not supported by cluster MD yet.
 
diff --git a/Documentation/md/raid5-cache.txt b/Documentation/md/raid5-cache.rst
similarity index 92%
rename from Documentation/md/raid5-cache.txt
rename to Documentation/md/raid5-cache.rst
index 2b210f295786..d7a15f44a7c3 100644
--- a/Documentation/md/raid5-cache.txt
+++ b/Documentation/md/raid5-cache.rst
@@ -1,4 +1,6 @@
-RAID5 cache
+================
+RAID 4/5/6 cache
+================
 
 Raid 4/5/6 could include an extra disk for data cache besides normal RAID
 disks. The role of RAID disks isn't changed with the cache disk. The cache disk
@@ -6,19 +8,19 @@ caches data to the RAID disks. The cache can be in write-through (supported
 since 4.4) or write-back mode (supported since 4.10). mdadm (supported since
 3.4) has a new option '--write-journal' to create array with cache. Please
 refer to mdadm manual for details. By default (RAID array starts), the cache is
-in write-through mode. A user can switch it to write-back mode by:
+in write-through mode. A user can switch it to write-back mode by::
 
-echo "write-back" > /sys/block/md0/md/journal_mode
+	echo "write-back" > /sys/block/md0/md/journal_mode
 
-And switch it back to write-through mode by:
+And switch it back to write-through mode by::
 
-echo "write-through" > /sys/block/md0/md/journal_mode
+	echo "write-through" > /sys/block/md0/md/journal_mode
 
 In both modes, all writes to the array will hit cache disk first. This means
 the cache disk must be fast and sustainable.
 
--------------------------------------
-write-through mode:
+write-through mode
+==================
 
 This mode mainly fixes the 'write hole' issue. For RAID 4/5/6 array, an unclean
 shutdown can cause data in some stripes to not be in consistent state, eg, data
@@ -42,8 +44,8 @@ exposed to 'write hole' again.
 In write-through mode, the cache disk isn't required to be big. Several
 hundreds megabytes are enough.
 
---------------------------------------
-write-back mode:
+write-back mode
+===============
 
 write-back mode fixes the 'write hole' issue too, since all write data is
 cached on cache disk. But the main goal of 'write-back' cache is to speed up
@@ -64,16 +66,16 @@ data loss.
 In write-back mode, MD also caches data in memory. The memory cache includes
 the same data stored on cache disk, so a power loss doesn't cause data loss.
 The memory cache size has performance impact for the array. It's recommended
-the size is big. A user can configure the size by:
+the size is big. A user can configure the size by::
 
-echo "2048" > /sys/block/md0/md/stripe_cache_size
+	echo "2048" > /sys/block/md0/md/stripe_cache_size
 
 Too small cache disk will make the write aggregation less efficient in this
 mode depending on the workloads. It's recommended to use a cache disk with at
 least several gigabytes size in write-back mode.
 
---------------------------------------
-The implementation:
+The implementation
+==================
 
 The write-through and write-back cache use the same disk format. The cache disk
 is organized as a simple write log. The log consists of 'meta data' and 'data'
diff --git a/Documentation/md/raid5-ppl.txt b/Documentation/md/raid5-ppl.rst
similarity index 98%
rename from Documentation/md/raid5-ppl.txt
rename to Documentation/md/raid5-ppl.rst
index bfa092589e00..357e5515bc55 100644
--- a/Documentation/md/raid5-ppl.txt
+++ b/Documentation/md/raid5-ppl.rst
@@ -1,4 +1,6 @@
+==================
 Partial Parity Log
+==================
 
 Partial Parity Log (PPL) is a feature available for RAID5 arrays. The issue
 addressed by PPL is that after a dirty shutdown, parity of a particular stripe
-- 
2.20.1

Powered by blists - more mailing lists