Message-ID: <20210809151529.ymbq53f633253loz@pali>
Date:   Mon, 9 Aug 2021 17:15:29 +0200
From:   Pali Rohár <pali@...nel.org>
To:     Martin Zaharinov <micron10@...il.com>
Cc:     Greg KH <gregkh@...uxfoundation.org>,
        netdev <netdev@...r.kernel.org>,
        Eric Dumazet <eric.dumazet@...il.com>
Subject: Re: Urgent  Bug report: PPPoE ioctl(PPPIOCCONNECT): Transport
 endpoint is not connected

On Sunday 08 August 2021 18:29:30 Martin Zaharinov wrote:
> Hi Pali
> 
> Kernel 5.13.8
> 
> 
> The problem is from kernel 5.8 > I try all major update 5.9, 5.10, 5.11 ,5.12
> 
> I use accel-pppd daemon (not pppd) .

I'm not using accel-pppd, so cannot help here.

I would suggest to try "git bisect" kernel version which started to be
problematic for accel-pppd.

Providing state of ppp channels and ppp units could help to debug this
issue, but I'm not sure if accel-pppd has this debug feature. IIRC only
process which has ppp file descriptors can retrieve and dump this
information.

> And yes after users started to connecting .
> 
> When system boot and connect first time all user connect without any problem .
> In time of work user disconnect and connect (power cut , fiber cut or other problem in network) , but in time of spike (may be make lock or other problem ) disconnect ~ 400-500 users  and affect other users. Process go to load over 100% and In statistic I see many finishing connection and many start connection. 
> And in this time in log get many lines with   ioctl(PPPIOCCONNECT): Transport endpoint is not connected. After finish (unlock or other) stop to see this error and system is back to normal. And connect all disconnected users.
> 
> Martin
> 
> > On 8 Aug 2021, at 18:23, Pali Rohár <pali@...nel.org> wrote:
> > 
> > Hello!
> > 
> > On Sunday 08 August 2021 18:14:09 Martin Zaharinov wrote:
> >> Add Pali Rohár,
> >> 
> >> If have any idea .
> >> 
> >> Martin
> >> 
> >>> On 6 Aug 2021, at 7:40, Greg KH <gregkh@...uxfoundation.org> wrote:
> >>> 
> >>> On Thu, Aug 05, 2021 at 11:53:50PM +0300, Martin Zaharinov wrote:
> >>>> Hi Net dev team
> >>>> 
> >>>> 
> >>>> Please check this error :
> >>>> Last time I write for this problem : https://www.spinics.net/lists/netdev/msg707513.html
> >>>> 
> >>>> But not find any solution.
> >>>> 
> >>>> Config of server is : Bonding port channel (LACP)  > Accel PPP server > Huawei switch.
> >>>> 
> >>>> Server is work fine users is down/up 500+ users .
> >>>> But in one moment server make spike and affect other vlans in same server .
> > 
> > When this error started to happen? After kernel upgrade? After pppd
> > upgrade? Or after system upgrade? Or when more users started to
> > connecting?
> > 
> >>>> And in accel I see many row with this error.
> >>>> 
> >>>> Is there options to find and fix this bug.
> >>>> 
> >>>> With accel team I discus this problem  and they claim it is kernel bug and need to find solution with Kernel dev team.
> >>>> 
> >>>> 
> >>>> [2021-08-05 13:52:05.294] vlan912: 24b205903d09718e: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:05.298] vlan912: 24b205903d097162: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:05.626] vlan641: 24b205903d09711b: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:11.000] vlan912: 24b205903d097105: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:17.852] vlan912: 24b205903d0971ae: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:21.113] vlan641: 24b205903d09715b: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:27.963] vlan912: 24b205903d09718d: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:30.249] vlan496: 24b205903d097184: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:30.992] vlan420: 24b205903d09718a: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:33.937] vlan640: 24b205903d0971cd: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:40.032] vlan912: 24b205903d097182: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:40.420] vlan912: 24b205903d0971d5: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:42.799] vlan912: 24b205903d09713a: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:42.799] vlan614: 24b205903d0971e5: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:43.102] vlan912: 24b205903d097190: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:43.850] vlan479: 24b205903d097153: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:43.850] vlan479: 24b205903d097141: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:43.852] vlan912: 24b205903d097198: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:43.977] vlan637: 24b205903d097148: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>>> [2021-08-05 13:52:44.528] vlan637: 24b205903d0971c3: ioctl(PPPIOCCONNECT): Transport endpoint is not connected
> >>> 
> >>> These are userspace error messages, not kernel messages.
> >>> 
> >>> What kernel version are you using?
> > 
> > Yes, we need to know, what kernel version are you using.
> > 
> >>> thanks,
> >>> 
> >>> greg k-h
> >> 
> > 
> > And also another question, what version of pppd daemon are you using?
> > 
> > Also, are you able to dump state of ppp channels and ppp units? It is
> > needed to know to which tty device, file descriptor (or socket
> > extension) is (or should be) particular ppp channel bounded.
> 

Powered by blists - more mailing lists