linux-kernel - Re: [PATCH 1/3] vt: avoid a VLA in the unicode screen scroll function

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <nycvar.YSQ.7.76.1807172155070.11681@knanqh.ubzr>
Date:   Tue, 17 Jul 2018 22:54:23 -0400 (EDT)
From:   Nicolas Pitre <nicolas.pitre@...aro.org>
To:     Adam Borowski <kilobyte@...band.pl>
cc:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Kees Cook <keescook@...omium.org>,
        Geert Uytterhoeven <geert@...ux-m68k.org>,
        Dave Mielke <Dave@...lke.cc>,
        Samuel Thibault <samuel.thibault@...-lyon.org>,
        linux-kernel@...r.kernel.org, linux-console@...r.kernel.org
Subject: Re: [PATCH 1/3] vt: avoid a VLA in the unicode screen scroll
 function

On Wed, 18 Jul 2018, Adam Borowski wrote:

> On Tue, Jul 17, 2018 at 09:02:40PM -0400, Nicolas Pitre wrote:
> > The nr argument is typically small: most often nr == 1. However this
> > could be abused with a very large explicit scroll in a resized screen.
> > Make the code scroll lines one at a time in all cases to avoid the VLA.
> > Anything smarter is most likely not warranted here.
> 
> Even though nr can be 32767 at most, your new version is O(nr*nr) for no
> reason.  Instead of O(n) memory or O(n²) time, a variant of the original
> that copies values one at a time would be shorter and faster.

Well... even though nr _can_ be up to 32766 to be precise, it is most 
likely to be just 1 in 99.9% of the cases. So in that case, you'll 
execute the loop only once and the code is currently optimal with O(n).

If nr > 1 then the current cost is O(n*nr) where n is the height of the 
scroll window i.e. relatively small in practice (typically between 25 
and 60). There is no point optimizing for 32767 rows as that is rather 
silly. If we had then the best solution would be a linked list rather 
than an array.

But still, if nr > 2 that means you need a temporary storage because the 
destination memory has to be preserved before the source memory can be 
moved there, and that destination memory content cannot be stored in the 
vacated source memory until the source content is moved. Copying values 
one at a time cannot work because the destination memory, the source 
memory, and the area where the previous content from the destination 
memory will end up don't overlap most of the time.

That temporary storage was that VLA. We don't want VLAs. So how do we 
efficiently allocate and deallocate memory for, say, 25 words? Maybe 
that doesn't have to be efficient because that doesn't happen very often 
as we said, at which point we can just do it in a loop with a one-line 
increment instead, as this patch does.

If you still have a more clever way of doing this then please propose it 
in code form (I'm genuinely curious of what you have in mind). But let's 
get the baseline working in an obvious "correct" way first.

> > Requested-by: Kees Cook <keescook@...omium.org>
> > Signed-off-by: Nicolas Pitre <nico@...aro.org>
> > ---
> >  drivers/tty/vt/vt.c | 18 ++++++++++--------
> >  1 file changed, 10 insertions(+), 8 deletions(-)
> > 
> > diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
> > index 2d14bb195d..03e79f7787 100644
> > --- a/drivers/tty/vt/vt.c
> > +++ b/drivers/tty/vt/vt.c
> > @@ -433,20 +433,22 @@ static void vc_uniscr_scroll(struct vc_data *vc, unsigned int t, unsigned int b,
> >  
> >  	if (uniscr) {
> >  		unsigned int s, d, rescue, clear;
> > -		char32_t *save[nr];
> >  
> >  		s = clear = t;
> > -		d = t + nr;
> > -		rescue = b - nr;
> > +		d = t + 1;
> > +		rescue = b - 1;
> >  		if (dir == SM_UP) {
> >  			swap(s, d);
> >  			swap(clear, rescue);
> >  		}
> > -		memcpy(save, uniscr->lines + rescue, nr * sizeof(*save));
> > -		memmove(uniscr->lines + d, uniscr->lines + s,
> > -			(b - t - nr) * sizeof(*uniscr->lines));
> > -		memcpy(uniscr->lines + clear, save, nr * sizeof(*save));
> > -		vc_uniscr_clear_lines(vc, clear, nr);
> > +		while (nr--) {
> > +			char32_t *tmp;
> > +			tmp = uniscr->lines[rescue];
> > +			memmove(uniscr->lines + d, uniscr->lines + s,
> > +				(b - t - 1) * sizeof(*uniscr->lines));
> > +			uniscr->lines[clear] = tmp;
> > +			vc_uniscr_clear_lines(vc, clear, 1);
> > +		}
> >  	}
> >  }
> 
> What the function does is rotating an array (slice [t..b) here), by nr if
> SM_DOWN or by -nr ie (b - t - nr) if SM_UP.  A nice problem that almost every
> "code interview questions" book includes :)
> 
> Please say if you don't have time for such games, I've just refreshed what's
> a good answer. :þ
> 
> 
> Meow.
> -- 
> // If you believe in so-called "intellectual property", please immediately
> // cease using counterfeit alphabets.  Instead, contact the nearest temple
> // of Amon, whose priests will provide you with scribal services for all
> // your writing needs, for Reasonable And Non-Discriminatory prices.
>