Date:   Thu, 29 Sep 2022 12:22:35 +0000
From:   "Artem S. Tashkinov" <aros@....com>
To:     Thorsten Leemhuis <linux@...mhuis.info>,
        Konstantin Ryabitsev <konstantin@...uxfoundation.org>
Cc:     workflows@...r.kernel.org, LKML <linux-kernel@...r.kernel.org>,
        Greg KH <gregkh@...uxfoundation.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        "regressions@...ts.linux.dev" <regressions@...ts.linux.dev>,
        ksummit@...ts.linux.dev
Subject: Re: Planned changes for bugzilla.kernel.org to reduce the "Bugzilla
 blues"

Hello everyone,

I'm glad the issue has been brought up again, I did it earlier but the
discussion never gained any traction:

https://lkml.org/lkml/2021/11/5/425
https://lore.kernel.org/lkml/6abc7248-efda-b569-9030-5384e5ce1f29@gmx.com/

Let me be brutally honest here, if you're working on the kernel,
specially for a large company such as e.g. Intel, you're _expected_ to
address the issues which are related to the kernel component[s] you're
maintaining/developing otherwise it's not "development" it's "I'm
dumping my code because my employer pays me to do that". That also means
you're expected to address bug reports.

It's correct I've tried to help people with bug reports posted on
bugzilla.kernel.org but it's a tough task considering that absolute most
kernel developers are not signed up, thus most bug reports are never
seen by respective developers.

How I'd go about the whole situation:

* Bugzilla must be there whether people like it or not. I've dealt with
LKML and other subsystems' mailing lists and the situation is even
_worse_: absolute most emails are simply completely ignored and _never_
replied to. The related bug reports, of course, are rarely if ever
addressed. It's so easy to say "sorry, yesterday I received 200 new
emails and simply didn't notice a new issue". That's ugly.

* All the components in the kernel bugzilla must be synchronized with
Kconfig's, in a perfect world automatically.

* Some kernel components, e.g. amdgpu, i915 and others have their own
bug trackers. Here's my proposal. People don't need to dig deep and
understand the intricacies of kernel development, all the components
must be there.

   Whenever a person tries to file a bug report for e.g. Drivers ->
Video Intel (not currently there),

   * * they either must be redirected to the appropriate bug tracker (
https://gitlab.freedesktop.org/drm/intel ) automatically, or

   * * a copy of the bug report in the appropriate bug tracker must be
created, or

   * * an email must be sent to the appropriate mailing list.

   Not sure if Bugzilla supports any of that but it's hugely important.

* Subsystem _maintainers_ must be present in the bugzilla by definition.
You're a maintainer after all. You're expected to be responsible (that
excludes the previous point if it's addressed).

* Kernel bugzilla must be opt-out, not opt-in. To be honest I'd
automatally add everyone who's commited to the kernel in the past 6
months and of course if new developers commit to the kernel, I'll add
them as well. Only if they _hate_ getting bugzilla emails, they are free
to unsubscribe.

* Speaking of a catch-all component. Mozilla does exactly that: bug
reports are filed under such a component however then AI/agorithm or a
person assigns them to a proper component. As a person someone could
certainly do that but I've not seen any open positions/vacancies for that.

TLDR: it's so easy to hate/dismiss bugzilla and say "use our mailing
list instead". Practice shows that "your mailing lists" are too often
completely disfunctional and allow [bug] reports to linger and get never
addressed which is not good for the kernel. I strongly oppose the idea
of kernel bugzilla deprecation.

AFAIK, the kernel bugzilla is a Linux Foundation project and the
organization receives funding from its very rich members including
Google, Meta, Intel, and even Microsoft. The fact that no one is
seriously working on it looks shameful and sad. We are not talking about
a minor odd library with a dozen users we are talking about the kernel.

Sorry about the tone of the message, I'm just too invested. It pains to
see how the kernel issues in regard to its use on the desktop receive
very little attention and how things which are important for major
companies (server use and Android) are all the rage and there are
specific people addressing them.

Best regards,
Artem

On 9/29/22 11:33, Thorsten Leemhuis wrote:
> [resent with the right ksummit list in CC]
>
> On 29.09.22 13:19, Thorsten Leemhuis wrote:
>> Hi!
>>
>> TLDR: Core Linux kernel developers are unhappy with the state of
>> bugzilla.kernel.org; to improve things I plan to change a few important
>> aspects of its configuration, unless somebody comes up with better ideas
>> to tackle current problems: (1) Create a catch-all product making it
>> totally obvious to submitters that likely nobody will look into the
>> ticket. (2) Remove or hide all products & components where the subsystem
>> didn't fully commit to look into newly submitted reports. (3) Change the
>> text on the front page to make it clear that most kernel bug reports
>> need to be sent by mail.
>>
>> I recently brought the state of bugzilla.kernel.org up for discussion on
>> the kernel summit and the kernel maintainers summit in sessions about my
>> regression tracking efforts. Long story short and rough: in both
>> sessions attendees were quite unhappy about the current state and wanted
>> things to change for the better. As I brought that topic up, I guess I
>> have to get things rolling now.
>>
>> But before getting into the details, a quick & rough reminder about the
>> current state of bugzilla.kernel.org:
>>
>>   * The server and the software running on it are well maintained by the
>> the infrastructure team (Konstantin et al.); many thx for this!
>>
>>   * Products, components, default assignees, et al. OTOH are heavily
>> outdated, incomplete, or wrong: maintaining this is not the job of the
>> infrastructure team and nobody else has stepped up to take care of this
>> (for a few more details see:
>> https://lore.kernel.org/lkml/20220420163223.kz32qomzj3y4hjj5@nitro.local/).
>>
>>   * To the best of my knowledge bugzilla.kernel.org was never really
>> sanctioned as the official place to report all sorts of kernel bugs:
>> only 20 (most of them from the area of ACPI/PM and PCI) out of ~2500
>> entries in MAINTAINERS currently tell users to report issues there; most
>> other subsystems just mention email contacts, a few (like the DRM
>> developers) point reporters to external bugtrackers.
>>
>>   * Developers of subsystems committed to the bug-tracker afaics usually
>> react to reports submitted in bugzilla.kernel.org. A few other
>> developers & subsystems keep an eye on reports, too; some do this
>> directly, others rely on bugzilla forwarding reports for certain
>> products/components by mail to the subsystem's mailing list. Quite some
>> or a lot of tickets are not forwarded to any developer or mailing list
>> at all.
>>
>>   * In the end lots of bug and regression reports (even good ones!) never
>> get a reply from a developer, as a brief analysis of mine showed
>> (https://lore.kernel.org/lkml/6808cd17-b48c-657d-de60-ef9d8bfa151e@leemhuis.info/
>> ). I at least currently try to work a bit against this by briefly
>> looking at each new report and forwarding any by mail that looks like a
>> regression worth forwarding (I ignore everything else). Artem S.
>> Tashkinov also looks into some (all?) reports and tries to help reporters.
>>
>> The sessions on kernel summit and the kernel maintainers summit
>> discussed the current state only for a few minutes. It's hard to
>> summarize these discussions, but let me try to mention the aspects that
>> are important for now:
>>
>>   * In both sessions members of the audience seemed pretty unhappy to me
>> about the current state of things.
>>
>>   * In the kernel summit sessions (recording:
>> https://youtu.be/e2SZoUPhDRg?t=5370 ) Len Brown stated that he and
>> fellow ACPI/PM developers rely on bugzilla.kernel.org and would need
>> some replacement if it's decommissioned.
>>
>>   * On the maintainers summit (see the last section of
>> https://lwn.net/Articles/908324/ for a brief write-up that coined the
>> term "Bugzilla blues") someone brought up the upstream development of
>> bugzilla the software seems to be dead; there was not even one strong
>> advocate for bugzilla.kernel.org and the general vibe tented into the
>> direction of "let's get rid of it". But it was also mentioned that
>> bugzilla.kernel.org does something useful which will need a replacement:
>> a place where reporters can upload big files needed for debugging problems.
>>
>> In the end that made me settle on this plan of action:
>>
>>   1. Finding a replacement for bugzilla will take a while, so for now
>> let's try to reduce some of its aspects that are bothering people:
>>
>>    1a. Create a new product/component that can act as a catch-all bug,
>> but makes it pretty clear that nobody might see the report because it's
>> not forwarded to anyone. People can use it to upload files for debugging
>> and link to them in mailed reports. People unable or unwilling to report
>> issues my mail (see 1c) could use it to submit issues, too. The outcome
>> then is the same as before, but at least people were told upfront about
>> the likely outcome; it also gives users a chance to help each other or
>> to coordinate before properly reporting an issue.
>>
>>    1b. Go through the list of products and components and hide or remove
>> *all* where the subsystem didn't fully commit to look into newly
>> submitted reports. Minimum requirements to remain listed will be along
>> these lines: subsystem mentions bugzilla.kernel.org in MAINTAINERS or a
>> developer listed in MAINTAINERS is one of the default assignees in
>> bugzilla. Subsystems where bugzilla forwards mails to a mailing list can
>> remain listed as well, if the recent history shows the developers look
>> into newly filed bugs. I'll use my best judgment in the transition
>> process and will file "anyone listening?" bugs if in a doubt.
>>
>>    1c. Make it obvious on the front-page of bugzilla.kernel.org that most
>> kernel developers want bug reports to be submitted by mail; mention the
>> subsystems that accept reports there and point to the catch-all bug (see
>> 1a) as a last straw.
>>
>>   2. See if everybody is happy with the new state for the time being; if
>> not further fine-tune things or speed up step (3).
>>
>>   3. Work out what we want as replacement.
>>
>> Anyone any comments on this or helpful ideas how to make things even
>> better? Otherwise, I'll in a week or two get down and start working on
>> realizing the points listed under step (1).
>>
>> Ciao, Thorsten

Powered by blists - more mailing lists