Message-ID:
 <MW5PR13MB5632F54DCBDA0C74370E531FFD5C2@MW5PR13MB5632.namprd13.prod.outlook.com>
Date: Thu, 7 Nov 2024 01:26:10 +0000
From: "Bird, Tim" <Tim.Bird@...y.com>
To: "linux-embedded@...r.kernel.org" <linux-embedded@...r.kernel.org>
CC: "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: [boot-time] RFC grab-boot-data.sh - tool to grab boot time data

Fellow boot time developers,

I'm writing to introduce a tool that I'd like people to run, to gather boot-time data on
their systems.  I'd like to collect boot-time data from a lot of different systems.

There are multiple reasons I'd like to collect this data, many of which are explained
on this wiki page:
https://birdcloud.org/boot-time/Boot-time_Tools

The script uploads the data to my personal machine (birdcloud.org), and will
be accessible to the public on this page:
https://birdcloud.org/boot-time/Boot_Data

Please NOTE that this script automatically (unless overridden) sends the data to
one of my personal wikis, on birdcloud.org.  This is kind of a poor man's kcidb,
expressly intended to collect boot time data for off-DUT analysis.
(DUT = Device Under Test).  You can examine the script to see the operations
it performs, which include things like grabbing your kernel config, cat'ing 
the running processes, getting your kernel boot messages, and showing memory,
uptime, processors, mounts  and other machine info and kernel info.

If any of this might contain or reveal sensitive information (e.g. the name of a
not-yet-released product, program or peripheral, or the attributes or capabilities thereof),
then you should NOT run this script.

It is possible to run the script while suppressing the upload functionality, using
the -x option.  In that case, you can examine the actual data inside the boot-data
file, and determine if you think it contains any sensitive information before sending it.

There are multiple goals for collecting this data, including:
 * identifying printk messages (and thus boot operations) that are common to all systems
 * collecting data about "common" timings for kernel operations and initcalls
 * correlating operation timings with machine attributes (e.g. kernel version, cpu count,
cpu speed, memory size, kernel command line args, selected drivers, etc.)
 * detecting kernel operations that run before the clock is initialized on a system

Each of these goals is described in more detail on the ..Tools wiki page listed above.
Please note that this personal wiki is meant solely as a repository of boot data from this
one tool.  I plan to add some analysis tools and possibly visualization pages for
the data in the coming months.  It is NOT where I plan to put the boot-time
pages related to tools, techniques, projects, etc for the boot-time SIG.  That is still
planned to be hosted on the elinux wiki.

The script text appears below, but you can also download a copy from the ..Tools wiki
page listed above.

== Call for action ==
Please run this script on a system (or multiple systems), so that I can collect data
for this effort.

Please also provide feedback on the script, the goals, and this overall approach,
by responding to this email.  Please let me know what you think.

Thanks,
 -- Tim

---- script grab-boot-data.sh follows ----
#!/bin/sh
# grab-boot-data.sh
#  This is a simple script to grab boot time data from dmesg

# gather meta-data about the machine, so the effects of things like
# memory, cpu count, cpu frequency, bogomips, kernel command-line options,
# etc. can be correlated with the duration of specific boot operations

VERSION="1.0"

# for testing
#UPLOAD_URL="http://localhost:8000/cgi-bin/tbwiki.cgi/Boot_Data?action=do_upload"
UPLOAD_URL="https://birdcloud.org/boot-time/Boot_Data?action=do_upload"

usage() {
    cat <<HERE
Usage: grab-boot-data.sh -l <lab> -m <machine> [options]

Collect machine information and boot data, and send it to the
birdcloud boot-time wiki, for analysis.

Normally, you run this soon after booting your machine.  You should
add the following kernel command line options to your boot configuration:
  quiet
  initcall_debug
  log_buf_len=10M

You may need to add these to a booloader configuration file, such
as grub.cfg (if using the 'grub' bootloader, or if using u-boot,
by editing the bootargs variable). See bootloader-specific documentation
for how to adjust the kernel command line for your boot.

Options:
 -h,--help  Show this online usage help
 -s         Skip kernel command-line checks
            Use this if you want to collect machine data and dmesg data,
            but didn't use 'quiet' or 'initcall_debug' on the kernel
            command line.

 -l <lab>   Specify lab (or user) name for data
 -m <name>  Specify machine name for data

 -d <dir>   Output the boot-data file to the indicated directory

 -x         Don't upload the data to the wiki (but still save the data
            to a file)

 -u <file>  Upload specified boot-data file to the wiki
            This should be used when the target doesn't have networking or
            is missing the curl command. In this case, use '-x' when
            running on the target machine, transfer the resulting
            data file to a machine that does have networking and curl,
            and then upload the file using -u.

--version   Show program version information
--debug     Show debug information while running
HERE
}

timestamp() {
date +"%y%m%d-%H%M%S"
}

OUTPUT_DIR=.

LAB="unknown"
MACHINE="unknown"

SAVE_ARGS="$@"

# parse options
while [ -n "$1" ] ; do
    case $1 in
        -h|--help)
            usage
            exit 0
            ;;
        -s)
            echo "Skipping command line checks"
            SKIP_CMDLINE_CHECKS=1
            shift
            ;;
        -x)
            echo "Skipping upload"
            SKIP_UPLOAD=1
            shift
            ;;
        -d)
            shift
            OUTPUT_DIR="$1"
            shift
            ;;
        -l)
            shift
            LAB="$1"
            shift
            ;;
        -m)
            shift
            MACHINE="$1"
            shift
            ;;
        -u)
            shift
            SKIP_CMDLINE_CHECKS=1
            SKIP_GRAB=1
            UPLOAD_FILE=$1
            shift
            ;;
        --version)
            echo "grab-boot-data.sh Version $VERSION"
            exit 0
            ;;
        --debug)
            set -x
            shift
            ;;
        *)
            echo "Unrecognized command line option '$1'"
            echo "Use -h for help"
            exit 1
            ;;
    esac
done

OUTFILE="boot-data-${LAB}-${MACHINE}-$(timestamp).txt"
OUTPATH=${OUTPUT_DIR}/${OUTFILE}

# do some error checking
if [ -z "$SKIP_GRAB" -a "$LAB" = "unknown" ] ; then
    echo "Error: Please specify a lab name for the boot data"
    exit 1
fi

if [ -z "$SKIP_GRAB" -a "$MACHINE" = "unknown" ] ; then
    echo "Error: Please specify a machine name for the boot data"
    exit 1
fi

# check if 'quiet' and 'initcall_debug' are in the kernel command line
CMDLINE="$(cat /proc/cmdline)"

if [ -z "$SKIP_CMDLINE_CHECKS" -a "${CMDLINE#*quiet}" = "${CMDLINE}" ] ; then
    echo "Error: Missing 'quiet' on kernel command line"
    echo "Please reboot the kernel with 'quiet' on the command line, and run again"
    exit 1
fi

if [ -z "$SKIP_CMDLINE_CHECKS" -a "${CMDLINE#*initcall_debug}" = "${CMDLINE}" ] ; then
    echo "Error: Missing 'initcall_debug' on kernel command line"
    echo "Please reboot the kernel with 'initcall_debug' on the command line, and run again"
    exit 1
fi

out_section() {
    echo "== $1 ==" >>$OUTPATH
    $2 >>$OUTPATH 2>&1
    echo >>$OUTPATH
}

get_config() {
    # check a few different places for the kernel config file
    if [ -f /proc/config.gz ] ; then
        zcat /proc/config.gz
        return
    fi

    # check /lib/modules
    RELEASE="$(uname -r)"
    if [ -f /lib/modules/${RELEASE}/kernel/kernel/configs.ko ] ; then
        insmod /lib/modules/${RELEASE}/kernel/kernel/configs.ko
        zcat /proc/config.gz
        rmmod configs
        return
    fi
    if [ -f /lib/modules/${RELEASE}/kernel/kernel/configs.ko.xz ] ; then
        insmod /lib/modules/${RELEASE}/kernel/kernel/configs.ko.xz
        zcat /proc/config.gz
        rmmod configs
        return
    fi
    if [ -f /lib/modules/${RELEASE}/build/.config ] ; then
        cat /lib/modules/${RELEASE}/build/.config
        return
    fi

    # check /boot directory
    if [ -f /boot/config-${RELEASE} ] ; then
        cat /boot/config-${RELEASE}
        return
    fi
    if [ -f /boot/config ] ; then
        cat /boot/config
        return
    fi

    # other possibilities, though these are less likely on embedded devices
    # IMHO - it's too easy for these to be inaccurate, comment them
    # out for now
    # /usr/src/linux-$RELEASE/.config
    # /usr/src/linux/.config
    # /usr/lib/ostree-boot/config-$RELEASE
    # /usr/lib/kernel/config-$RELEASE
    # /usr/src/linux-headers-$RELEASE/.config",

    echo "Can't find kernel config data or file"
}

if [ -z "$SKIP_GRAB" ] ; then
    echo "=== Machine Info ==============================" >$OUTPATH

    out_section UPTIME "uptime"

    echo "== GRAB-BOOT-DATA INFO ==" >>$OUTPATH
    echo "GBD_ARGS=\"$SAVE_ARGS\"" >>$OUTPATH
    echo "GBD_VERSION=\"$VERSION\"" >>$OUTPATH
    echo >>$OUTPATH

    echo "== Kernel Info ==" >>$OUTPATH
    echo "KERNEL_VERSION=\"$(uname -r -v)\"" >>$OUTPATH
    echo "KERNEL_CMDLINE=\"$CMDLINE\"" >>$OUTPATH
    echo >>$OUTPATH

    out_section OS "cat /etc/os-release"
    out_section MEMORY "free"
    out_section "DISK USAGE" "df -h"
    out_section MOUNTS "mount"
    # ps -A seems to work with procps ps, busybox ps, and toybox ps
    out_section PROCESSES "ps -A"
    #out_section INTERRUPTS "cat /proc/interrupts"
    out_section CORES "cat /proc/cpuinfo"
    out_section CONFIG "get_config"
    out_section "KERNEL MESSAGES" "dmesg"

    echo >>$OUTPATH

    echo "Boot data is in the file: $OUTPATH"
    UPLOAD_FILE="${OUTPATH}"
fi

# should I also get a systemd-analyze here?

### upload the data
if [ -z "$SKIP_UPLOAD" ] ; then
    echo "Upload file=$UPLOAD_FILE"

    RESULT_HTML_FILE="$(mktemp)"
    echo "Uploading data ..."
    curl -F submit_button=Upload -F file_1=@...LOAD_FILE -F file_2= -F file_3= $UPLOAD_URL -o $RESULT_HTML_FILE
    if grep "uploaded successfully" $RESULT_HTML_FILE >/dev/null 2>&1 ; then
        echo "Data uploaded successfully."
        rm $RESULT_HTML_FILE
    else
        echo "Error: There was a problem uploading the data"
        echo "See $RESULT_HTML_FILE for details."
        echo "Don't forget to remove this file when done using it."
   fi
fi

Powered by blists - more mailing lists