| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <1441130243-20303-1-git-send-email-matthias.tafelmeier@gmx.net>
Date: Tue, 1 Sep 2015 19:57:13 +0200
From: Matthias Tafelmeier <matthias.tafelmeier@....net>
To: netdev@...r.kernel.org
Cc: hagen@...u.net, shemminger@...l.org, fw@...len.de,
edumazet@...gle.com, daniel@...earbox.net
Subject: iproute2 v5: full ss json support and general output simplification
TLDR:
- add full JSON support for ss
- Patchset provides a general and easy to use abstraction to extend ss later
- Patchset size is large to minimize daily use ("user" should not deal with
formation (json, human readble) later on)
- Patches 7/10 and 8/10 illustrate how to extend ss for new data to support human readble and json
output.
- Example_Usages: 1. ss -jt to print out all tcp related information formatted in json
2. ss --json -a to print out all info (also summary)
STATS:
ss: rooted out ss type declarations for output formatters
ss: created formatters for json and hr
ss: removed obsolet fmt functions
ss: prepare timer for output handler usage
ss: replaced old output with new generic output mechanisms
ss: renaming and export of current_filter
ss: symmetrical subhandler output extension example
ss: symmetrical formatter extension example
ss: fixed free on local array for valid json output
ss: activate json_writer excluded logic
include/json_writer.h | 1 +
lib/json_writer.c | 7 -
misc/Makefile | 2 +-
misc/ss.c | 971 +++++++++++++++++---------------------------------
misc/ss_hr_fmt.c | 321 +++++++++++++++++
misc/ss_hr_fmt.h | 9 +
misc/ss_json_fmt.c | 475 ++++++++++++++++++++++++
misc/ss_json_fmt.h | 26 ++
misc/ss_out_fmt.c | 137 +++++++
misc/ss_out_fmt.h | 92 +++++
misc/ss_types.h | 186 ++++++++++
11 files changed, 1567 insertions(+), 660 deletions(-)
create mode 100644 misc/ss_hr_fmt.c
create mode 100644 misc/ss_hr_fmt.h
create mode 100644 misc/ss_json_fmt.c
create mode 100644 misc/ss_json_fmt.h
create mode 100644 misc/ss_out_fmt.c
create mode 100644 misc/ss_out_fmt.h
create mode 100644 misc/ss_types.h
--
Abstract:
This patch set originates from the necessity to upgrade ss with the possibility
to output in json format. Not to clutter up ss too much, the author of the
patch decided to come up with a simple distributor to handler approach. That
is, the distributor poses the mechanical interface which passes the output
requests coming from ss to the appropriate handler. This simplifies the
interaction with ss and provides a maximum of future extensiblity. Not to
forget, ss loses weight thereby since output implemented in ss itself does
migrate to the appropriate handler. Additionally, because types are shared
amongst handlers, the distributor and ss, the author conceived, that a separate
containter module for types has to be formed. In future, all type declarations
and extensins go there.
In sum, the patchset has this voluminous extent since there is no viable way
for putting out syntactically correct human readble and json in a simpler manner.
The requirement for convenient extensibility of output and data is
another justification for the patchset size.
Concept sketch:
formatter1
************
* *
* *
ss ~~~~~~~>zzzzzzz *
****************** ~ * *
* * ~ ###>fffffff *
* * ~ # * *
* * distributor~ # ************
* -------- * ********* ~ #
* - -------------- * * ~ #
* -------- * - * * ~ #
* * ---->++++ ~~~~ #
* * * * ~ # formatter2
* * ---->==== ######## ************
* -------- * - * * ~ # * *
* - -------------- * * ~ # * *
* -------- * ********* ~ # * *
* * ~~~~#~~>zzzzzzz *
* * # * *
* * ###>fffffff *
****************** * *
************
At the moment, the distributor is the ss_out_fmt module while two handlers are
up: namely the ss_json_fmt and the ss_hr_fmt (human readable). You can use
those modules as the main reference for own extensions.
Future Extension:
In the following, I will expand on the expandability of the formatter model.
The explanations advances from the minimal to the most sweeping extension in
mind.
Sub Format Handler Output
Sketch
FormatterX
***********************************
* *
* handlerX *
* °°°°°°°°°°°°°°°°°°°°°°°°°° *
* ° ° *
* ° xxxxxxxxxxxxxxc<.. ° *
* ° . ° *
* ° xxxxxxxxxxxxxxc<.. potential context
* ° new: . ° *
* ° +++++++++++++++... < * * * * * * * * * * *
* ° ° * *
* °°°°°°°°°°°°°°°°°°°°°°°°°° * *
* . * *
* . * *
* . * *
* handlerY * *
* °°°°°°°°°°°°°°°°°°°°°°°°°° * *
* ° ° * *
* ° xxxxxxxxxxxxxxx ° * *
* ° ° * *
* ° xxxxxxxxxxxxxxx ° * *
* ° ° * * expand symmetrically
* ° ° * *
* °°°°°°°°°°°°°°°°°°°°°°°°°° * *
* * *
* * *
* * *
*********************************** *
*
. *
. *
. *
*
FormatterY *
*********************************** *
* * *
* handlerX * *
* °°°°°°°°°°°°°°°°°°°°°°°°°° * *
* ° ° * *
* ° zzzzzzzzzzzzzzc<.. ° * *
* ° . ° * *
* ° zzzzzzzzzzzzzzc<.. potential context *
* ° new: . ° * *
* ° +++++++++++++++... < * * * * * * * * * * *
* ° ° *
* °°°°°°°°°°°°°°°°°°°°°°°°°° *
* . *
* . *
* . *
* handlerY *
* °°°°°°°°°°°°°°°°°°°°°°°°°° *
* ° ° *
* ° zzzzzzzzzzzzzzz ° *
* ° ° *
* ° zzzzzzzzzzzzzzz ° *
* ° ° *
* ° ° *
* °°°°°°°°°°°°°°°°°°°°°°°°°° *
* *
* *
***********************************
Explanation:
If you plan to expand a sub out handler function of a formatter, it
essentially boils down to adding a new printf with an according format and
probably a necessary predicate (condition). Nontheless, care must be taken not
to lose possible context interdependecies out of sight. An examble for the
latter would be the interdependecy of json coma setting terms – in compound
types, you do need a coma between consecutive elements.
More important is the issue about symmetric extensions. Except for the
tcp_out_fmt function implementation – where a macro (CHECK_FMT_ADAPT) is in
place to check for adaptions in the basic tcpstat data structure statically –
no general programmatic approach is in place yet which would prevent asymmetric
extensions. Up to someone devices a holistic solution, this patch relies on the
extenders to deal with asymmetries. Is the aim to have a new output feature
available in all semantically related handlers of n different formatters, then
the expander has to adapt n handlers as shown in the sketch above.
Extend with further Format Handler in Formatter
Sketch:
-provides symmetrical extension intendet
formatter1
/--------------------------\
| |
| spec_handlerX |
| ################ <** |
| ################ * |
distributor | . * |
O-------------------------O | . * |
| centr_hub | | new_spec_handler * |
| ***>01111110--------------------------+ | ++++++++++++++++<~ * |
| * 02222220----------------------+ | | ++++++++++++++++ ~ * |
| * | | | | ~ * |
| * | | | | handler_hub1 ~ * |
| * _______________ | | +--------->0#########0****~** |
| * |gen_handlerX | | | | 0+++++++++0~~~~~ |
| ********** | | | | |
| * |_______________| | | | |
| * . | | \--------------------------/
| * . | | .
| * . | | .
| * +++++++++++++++++ | | formatterN .
| ***new_gen_handler+ | | /--------------------------\
| + + | | | |
| +++++++++++++++++ | | | spec_handlerX |
| | | | ################ <** |
O-------------------------O | | ################ * |
| | . * |
| | . * |
| | new_spec_handler * |
| | ++++++++++++++++<~ * |
| | ++++++++++++++++ ~ * |
| | ~ * |
| | handler_hubN ~ * |
+------------->0#########0****~** |
| 0+++++++++0~~~~~ |
| |
| |
\--------------------------/
Explanation:
As the sketch shows, the distributor works with the help of virtual function
pointer in order to act as a call flow switch. It switches to the approriate
formatter module and its handlers depending on the chosen output format by ss
command input.
So, to add a new formatter handler symmetrically (up to now that is the only
sensibly conceivable case), the extender must implement a new generic handler
in the distributor and the specific handlers in the formatters. Then the hub
vtable structure type has to be broadend to contain the new function pointer
type for the generic handler. After that, he has to extend and update all
handler hubs with the new handlers location information (function pointer). The
latter ensures the generic switching mechanism used by the generic handler
keeps to be upheld.
Example:
Let's say we want the new "foo" data for every output format retrievable via
ss. Up to now, we have the ss_out_fmt module as the distributor and two
specific handlers: for one the ss_hr_fmt and secondly the ss_json_fmt module.
So we need a specific handler implementation in ss_hr_fmt and ss_json_fmt
modules and after that update the corresponding vtables (handler_hubs) in the
modules. After that, the distributor, namely ss_out_fmt module, has to get a
generic handler that switches via its vtable hub to either the json formatter
or the human readable formatter, depending on what fmt_type has been chosen by ss.
Before the vtables in the specific modules can be updated, struct fmt_op_hub
which is found in ss_out_fmt's interface header has get extendend with the new
function pointer type.
As soon the new generic handler has been exported via the ss_out_fmt.h module
interface, ss can use the new fmt handler to print out info. It can simply call
the generic function and does not have to deal with formatting specific issues.
Extend for another Formatter
Sketch:
The Sketch for handler extension should be sufficient for conveying the concept.
Just think of another formatter after formatterN and a new entry in the central
vtable of the distributor to reach this new formatter.
Explanation:
Nothing breathtaking has to be done when someone needs an new formatter module
for let's be image – out of pure hypothetical endeavors – xml ss output. First,
implement the new formatter with all the offered interfaces in the
distributor. Register all handlers in the local specific vtable hub.
Then, register the local vtable hub in the generic vtable hub of the
distributor to reach your new handler when chosen. Provide the client
code - here ss - with a new fmt_type option acceptance. Before the
option can do anything, you have to declare the new fmt_type. That's
it. No further adaptions in ss would be necessary.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists