netdev - Re: virtio_net failover and initramfs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Fri, 17 Aug 2018 12:09:17 -0700
From:   "Samudrala, Sridhar" <sridhar.samudrala@...el.com>
To:     Harald Hoyer <harald@...hat.com>, Siwei Liu <loseweigh@...il.com>
Cc:     Jiri Pirko <jiri@...nulli.us>, initramfs@...r.kernel.org,
        "Michael S. Tsirkin" <mst@...hat.com>,
        Netdev <netdev@...r.kernel.org>, vijay.balakrishna@...cle.com,
        si-wei liu <si-wei.liu@...cle.com>, liran.alon@...cle.com
Subject: Re: virtio_net failover and initramfs

On 8/17/2018 2:56 AM, Harald Hoyer wrote:
> On 17.08.2018 11:51, Harald Hoyer wrote:
>> On 16.08.2018 00:17, Siwei Liu wrote:
>>> On Wed, Aug 15, 2018 at 12:05 PM, Samudrala, Sridhar
>>> <sridhar.samudrala@...el.com> wrote:
>>>> On 8/14/2018 5:03 PM, Siwei Liu wrote:
>>>>> Are we sure all userspace apps skip and ignore slave interfaces by
>>>>> just looking at "IFLA_MASTER" attribute?
>>>>>
>>>>> When STANDBY is enabled on virtio-net, a failover master interface
>>>>> will appear, which automatically enslaves the virtio device. But it is
>>>>> found out that iSCSI (or any network boot) cannot boot strap over the
>>>>> new failover interface together with a standby virtio (without any VF
>>>>> or PT device in place).
>>>>>
>>>>> Dracut (initramfs) ends up with timeout and dropping into emergency shell:
>>>>>
>>>>> [  228.170425] dracut-initqueue[377]: Warning: dracut-initqueue
>>>>> timeout - starting timeout scripts
>>>>> [  228.171788] dracut-initqueue[377]: Warning: Could not boot.
>>>>>            Starting Dracut Emergency Shell...
>>>>> Generating "/run/initramfs/rdsosreport.txt"
>>>>> Entering emergency mode. Exit the shell to continue.
>>>>> Type "journalctl" to view system logs.
>>>>> You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or
>>>>> /boot
>>>>> after mounting them and attach it to a bug report.
>>>>> dracut:/# ip l sh
>>>>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
>>>>> mode DEFAULT group default qlen 1000
>>>>>       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>>>>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
>>>>> state UP mode DEFAULT group default qlen 1000
>>>>>       link/ether 9a:46:22:ae:33:54 brd ff:ff:ff:ff:ff:ff\
>>>>> 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
>>>>> master eth0 state UP mode DEFAULT group default qlen 1000
>>>>>       link/ether 9a:46:22:ae:33:54 brd ff:ff:ff:ff:ff:ff
>>>>> dracut:/#
>>>>>
>>>>> If changing dracut code to ignore eth1 (with IFLA_MASTER attr),
>>>>> network boot starts to work.
>>>>
>>>> Does dracut by default tries to use all the interfaces that are UP?
>>>>
>>> Yes. The specific dracut cmdline of our case is "ip=dhcp
>>> netroot=iscsi:... ", but it's not specific to iscsi boot. And because
>>> of same MAC address for failover and standby, while dracut tries to
>>> run DHCP on all interfaces that are up it eventually gets same route
>>> for each interface. Those conflict route entries kill off the network
>>> connection.
>>>
>>>>> The reason is that dracut has its own means to differentiate virtual
>>>>> interfaces for network boot: it does not look at IFLA_MASTER and
>>>>> ignores slave interfaces. Instead, users have to provide explicit
>>>>> option e.g. bond=eth0,eth1 in the boot line, then dracut would know
>>>>> the config and ignore the slave interfaces.
>>>>
>>>> Isn't it possible to specify the interface that should be used for network
>>>> boot?
>>> As I understand it, one can only specify interface name for running
>>> DHCP but not select interface for network boot.  We want DHCP to run
>>> on every NIC that is up (excluding the enslaved interfaces), and only
>>> one of them can get a route entry to the network boot server (ie.g.
>>> iSCSI target).
>>>
>>>>
>>>>> However, with automatic creation of failover interface that assumption
>>>>> is no longer true. Can we change dracut to ignore all slave interface
>>>>> by checking  IFLA_MASTER? I don't think so. It has a large impact to
>>>>> existing configs.
>>>>
>>>> What is the issue with checking for IFLA_MASTER? I guess this is used with
>>>> team/bonding setups.
>>> That should be discussed within and determined by the dracut
>>> community. But the current dracut code doesn't check IFLA_MASTER for
>>> team or bonding specifically. I guess this change might have broader
>>> impact to existing userspace that might be already relying on the
>>> current behaviour.
>>>
>>> Thanks,
>>> -Siwei
>> Is there a sysfs flag for IFF_SLAVE? Or any "ip" output I can use to detect, that it is a IFF_SLAVE?
>>
> Oh, it's the other way around.. dracut should ignore "master" (eth1).
In the above example eth0 is the net_failover device and eth1 is the 
lower virtio_net device.
"ip" output of eth1 shows "master eth0". It indicates that eth0 is its 
upper/master device.
This information can also be obtained via sysfs too. 
/sys/class/net/eth1/upper_eth0
>
> Can the master enslave the "eth0", if it is already "UP" and busy later on?
eth0 is the master/failover device and eth1 gets registered as its slave 
via NETDEV_REGISTER event.
dracut should ignore eth1 in this setup.