| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <1225817093-6247-1-git-send-email-bharrosh@panasas.com>
Date: Tue, 4 Nov 2008 18:44:53 +0200
From: Boaz Harrosh <bharrosh@...asas.com>
To: James Bottomley <James.Bottomley@...senPartnership.com>,
Andrew Morton <akpm@...ux-foundation.org>,
Mike Christie <michaelc@...wisc.edu>,
FUJITA Tomonori <fujita.tomonori@....ntt.co.jp>,
Jeff Garzik <jeff@...zik.org>,
open-osd ml <osd-dev@...n-osd.org>,
linux-scsi <linux-scsi@...r.kernel.org>
Cc: linux-kernel <linux-kernel@...r.kernel.org>, Sami.Iren@...gate.com,
Pete Wyckoff <pw@...d.com>
Subject: [PATCH 16/18] osd: Documentation for OSD library
Add osd.txt to Documentation/scsi/
Signed-off-by: Boaz Harrosh <bharrosh@...asas.com>
Reviewed-by: Benny Halevy <bhalevy@...asas.com>
---
Documentation/scsi/osd.txt | 198 ++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 198 insertions(+), 0 deletions(-)
create mode 100644 Documentation/scsi/osd.txt
diff --git a/Documentation/scsi/osd.txt b/Documentation/scsi/osd.txt
new file mode 100644
index 0000000..a6735ae
--- /dev/null
+++ b/Documentation/scsi/osd.txt
@@ -0,0 +1,198 @@
+The OSD Standard
+================
+OSD (Object-Based Storage Device) is a T10 SCSI command set that is designed
+to provide efficient operation of input/output logical units that manage the
+allocation, placement, and accessing of variable-size data-storage containers,
+called objects. Objects are intended to contain operating system and application
+constructs. Each object has associated attributes attached to it, which are
+integral part of the object and provide metadata about the object. The standard
+defines some common obligatory attributes, but user attributes can be added as
+needed.
+
+See: http://www.t10.org/ftp/t10/drafts/osd2/ for the latest draft for OSD 2
+or search the web for "OSD SCSI"
+
+OSD in the Linux Kernel
+=======================
+osd-initiator:
+ The main component of OSD in Kernel is the osd-initiator library. Its main
+user is intended to be the pNFS-over-objects layout driver, which uses objects
+as its back-end data storage. Other clients are the other osd parts listed below.
+
+osd-uld:
+ This is a SCSI ULD that registers for OSD type devices and provides a testing
+platform, both for the in-kernel initiator as well as connected targets. It
+currently has no useful user-mode API, though it could have if need be.
+
+osdfs:
+ Is an OSD based Linux file system. It uses the osd-initiator and osd-uld,
+to export a usable file system for users.
+See Documentation/filesystems/osdfs.txt for more details
+
+osd target:
+ There are no current plans for an OSD target implementation in kernel. For all
+needs, a user-mode target that is based on the scsi tgt target framework is
+available from Ohio Supercomputer Center (OSC) at:
+http://www.open-osd.org/bin/view/Main/OscOsdProject
+There are several other target implementations. See http://open-osd.org for more
+links.
+
+Files and Folders
+=================
+This is the complete list of files included in this work:
+include/scsi/
+ osd_initiator.h Main API for the initiator library
+ osd_types.h Common OSD types
+ osd_sec.h Security Manager API
+ osd_protocol.h Wire definitions of the OSD standard protocol
+ osd_attributes.h Wire definitions of OSD attributes
+
+drivers/scsi/osd/
+ osd_initiator.c OSD-Initiator library implementation
+ osd_uld.c The OSD scsi ULD
+ osd_ktest.{h,c} In-kernel test suite (called by osd_uld)
+ osd_debug.h Some printk macros
+ Makefile For both in-tree and out-of-tree compilation
+ Kconfig Enables inclusion of the different pieces
+ osd_test.c User-mode application to call the kernel tests
+
+The OSD-Initiator Library
+=========================
+osd_initiator is a low level implementation of an osd initiator encoder.
+But even though, it should be intuitive and easy to use. Perhaps over time an
+higher lever will form that automates some of the more common recipes.
+
+init/fini:
+- osd_dev_init() associates a scsi_device with an osd_dev structure
+ and initializes some global pools. This should be done once per scsi_device
+ (OSD LUN). The osd_dev structure is needed for calling osd_start_request().
+
+- osd_dev_fini() cleans up before a osd_dev/scsi_device destruction.
+
+OSD commands encoding, execution, and decoding of results:
+
+struct osd_request's is used to iteratively encode an OSD command and carry
+its state throughout execution. Each request goes through these stages:
+
+a. osd_start_request() allocates the request.
+
+b. Any of the osd_req_* methods is used to encode a request of the specified
+ type.
+
+c. osd_req_add_{get,set}_attr_* may be called to add get/set attributes to the
+ CDB. "List" or "Page" mode can be used exclusively. The attribute-list API
+ can be called multiple times on the same request. However, only one
+ attribute-page can be read, as mandated by the OSD standard.
+
+d. osd_finalize_request() computes offsets into the data-in and data-out buffers
+ and signs the request using the provided capability key and integrity-
+ check parameters.
+
+e. osd_execute_request() may be called to execute the request via the block
+ layer and wait for its completion. The request can be executed
+ asynchronously by calling the block layer API directly.
+
+f. After execution, osd_req_decode_sense() can be called to decode the request's
+ sense information.
+
+g. osd_req_decode_get_attr() may be called to retrieve osd_add_get_attr_list()
+ values.
+
+h. osd_end_request() must be called to deallocate the request and any resource
+ associated with it. Note that osd_end_request cleans up the request at any
+ stage and it must always be called after a successful osd_start_request().
+
+osd_request's structure:
+
+The OSD standard defines a complex structure of IO segments pointed to by
+members in the CDB. Up to 3 segments can be deployed in the IN-Buffer and up to
+4 in the OUT-Buffer. The ASCII illustration below depicts a secure-read with
+associated get+set of attributes-lists. Other combinations very on the same
+basic theme. From no-segments-used up to all-segments-used.
+
+|________OSD-CDB__________|
+| |
+|read_len (offset=0) -|---------\
+| | |
+|get_attrs_list_length | |
+|get_attrs_list_offset -|----\ |
+| | | |
+|retrieved_attrs_alloc_len| | |
+|retrieved_attrs_offset -|----|----|-\
+| | | | |
+|set_attrs_list_length | | | |
+|set_attrs_list_offset -|-\ | | |
+| | | | | |
+|in_data_integ_offset -|-|--|----|-|-\
+|out_data_integ_offset -|-|--|--\ | | |
+\_________________________/ | | | | | |
+ | | | | | |
+|_______OUT-BUFFER________| | | | | | |
+| Set attr list |</ | | | | |
+| | | | | | |
+|-------------------------| | | | | |
+| Get attr descriptors |<---/ | | | |
+| | | | | |
+|-------------------------| | | | |
+| Out-data integrity |<------/ | | |
+| | | | |
+\_________________________/ | | |
+ | | |
+|________IN-BUFFER________| | | |
+| In-Data read |<--------/ | |
+| | | |
+|-------------------------| | |
+| Get attr list |<----------/ |
+| | |
+|-------------------------| |
+| In-data integrity |<------------/
+| |
+\_________________________/
+
+A block device request can carry bidirectional payload by means of associating
+a bidi_read request with a main write-request. Each in/out request is described
+by a chain of BIOs associated with each request.
+The CDB is of a SCSI VARLEN CDB format, as described by OSD standard.
+The OSD standard also mandates alignment restrictions at start of each segment.
+
+In the code, in struct osd_request, there are two _osd_io_info structures to
+describe the IN/OUT buffers above, two BIOs for the data payload and up to five
+_osd_req_data_segment structures to hold the different segments allocation and
+information.
+
+Important: We have chosen to disregard the assumption that a BIO-chain (and
+the resulting sg-list) describes a linear memory buffer. Meaning only first and
+last scatter chain can be incomplete and all the middle chains are of PAGE_SIZE.
+For us, a scatter-gather-list, as its name implies and as used by the Networking
+layer, is to describe a vector of buffers that will be transferred to/from the
+wire. It works very well with current iSCSI transport. iSCSI is currently the
+only deployed OSD transport. In the future we anticipate SAS and FC attached OSD
+devices as well.
+
+The OSD Testing ULD
+===================
+TODO: More user-mode control on tests.
+
+Authors, Mailing list
+=====================
+Please communicate with us on any deployment of osd, whether using this code
+or not.
+
+Any problems, questions, bug reports, lonely OSD nights, please email:
+ OSD Dev List <osd-dev@...n-osd.org>
+
+More up-to-date information can be found on:
+http://open-osd.org
+
+Boaz Harrosh <bharrosh@...asas.com>
+Benny Halevy <bhalevy@...asas.com>
+
+References
+==========
+Weber, R., "SCSI Object-Based Storage Device Commands",
+T10/1355-D ANSI/INCITS 400-2004,
+http://www.t10.org/ftp/t10/drafts/osd/osd-r10.pdf
+
+Weber, R., "SCSI Object-Based Storage Device Commands -2 (OSD-2)"
+T10/1729-D, Working Draft, rev. 3
+http://www.t10.org/ftp/t10/drafts/osd2/osd2r03.pdf
--
1.6.0.1
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists