Date:	Wed, 21 Dec 2011 21:01:41 -0800 (PST)
From:	david@...g.hm
To:	Brian Swetland <swetland@...gle.com>
cc:	NeilBrown <neilb@...e.de>, Tim Bird <tim.bird@...sony.com>,
	Greg KH <gregkh@...e.de>,
	linux-embedded <linux-embedded@...r.kernel.org>,
	linux kernel <linux-kernel@...r.kernel.org>,
	Arnd Bergmann <arnd@...db.de>,
	john stultz <johnstul@...ibm.com>,
	Kay Sievers <kay.sievers@...y.org>,
	Lennart Poettering <lennart@...ttering.net>
Subject: Re: RFC: android logger feedback request

On Wed, 21 Dec 2011, Brian Swetland wrote:

> On Wed, Dec 21, 2011 at 5:20 PM, NeilBrown <neilb@...e.de> wrote:
>> On Wed, 21 Dec 2011 16:36:21 -0800 Tim Bird <tim.bird@...sony.com> wrote:
>>
>>> On 12/21/2011 03:19 PM, Greg KH wrote:
>>>> That all describes the current code, but you haven't described what's
>>>> wrong with the existing syslog interface that requires this new driver
>>>> to be written.  And why can't the existing interface be fixed to address
>>>> these (potential) shortcomings?
>>>
>>>>> One specific question I have is where is the most appropriate
>>>>> place for this code to live, in the kernel source tree?
>>>>> Other embedded systems might want to use this system (it
>>>>> is simpler than syslog, and superior in some ways), so I don't
>>>>> think it should remain in an android-specific directory.
>>>>
>>>> What way is it superior?
>>>
>>> Here are some ways that this code is superior to syslog:
>>
>> It is certainly nice and simple.  It really looks more like a filesystem than
>> a char device though...  though they aren't really files so much as lossy
>> pipes.  I don't think that's a problem though, lots of things in filesystems
>> don't behave exactly like files.
>>
>> If you created a 'logbuf' filesystem that used libfs to provide a single
>> directory in which privileged processes could create files then you wouldn't
>> need the kernel to "know" the allowed logs: radio, events, main, system.
>> The size could be set by ftruncate() (by privileged used again) rather than
>> being hardcoded.
<<SNIP>>
>> The result would be much the same amount of code, but an interface which has
>> fewer details hard-coded and is generally more versatile and accessible.
>
> Moving away from hard coding the names/sizes of the logs in the driver
> is something that has been on the todo list for a while.  One thing
> we'd likely want to accomplish there is avoid creating a vector for
> consuming large amounts of memory by creating new logs.
>
> One planned change (likely to happen in the Android J release
> timeframe) is to adjust permissions such that any process can write
> messages, but unless they belong to the correct group they can only
> read back messages written by their own PID.  This is to allow apps to
> grab their own log output after a crash or during a user problem
> report without needing to grant them the ability to read all log
> messages.
>
> Currently the logger driver does not provide a mechanism for allowing
> logs to survive a reboot (unlike the ramconsole), but this is
> functionality that we've thought about adding.  Generally the kernel
> logs are most interesting after an unexpected panic/reboot, but
> getting a picture of what userspace has been up to can be useful too.
>
> The goals behind the logger driver have been:
> - keep userland and kernel logging separate (so that spammy userland
> logging doesn't make us lose critical kernel logs or the other way
> round)
> - make log writing very inexpensive -- avoid having to pass messages
> between processes (more critical on ARM9 platforms where this implied
> extra cache flushing), avoid having to make several syscalls to write
> a log message (getting time of day, etc), and so on
> - make log writing reliable -- don't trust userland to report its
> timestamp, PID, or to correctly format the datagrams, etc
> - allow a log watching process (logcat) to easily pull data from all
> logs at once
> - avoid committing a vast amount of memory to logging
> - try to prevent clients from spamming each other out of log space
> (only successful on a coarse granularity right now with the
> main/system/radio/events logs)
> - ensure logs are not lost at the moment an app crashes
>
> On one hand, having each app (per PID) be able to create their own
> logs up to a specified size limit could be really useful and is
> something we've kicked around -- for one it would allow us to avoid
> the ever present request from userspace developers to increase the log
> size because of too much log spam ("reduce log spam" never seems to be
> an answer that makes them happy) -- but we haven't come up with a
> reasonable plan for dealing with "well if we allow 16KB of log per app
> and the user installs 100 apps, they may be pinning up to 1.6MB of ram
> worst case", and so on.

At this point you are starting to sound like something much closer to a 
traditional syslog daemon. you are adding so many variations, persistant 
storage, etc that you really don't want to have to have all this in the 
kernel, make these be interfaces into a userspace logging tool (ideally 
syslog compatible), and you have the option to easily have different 
policies, consuming different amounts of space, depending on the device 
and how resource limited it is.

David Lang

Powered by blists - more mailing lists