lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-ID: <20091019075350.GA1769@cmpxchg.org> Date: Mon, 19 Oct 2009 09:53:50 +0200 From: Johannes Weiner <hannes@...xchg.org> To: Earl Chew <earl_chew@...lent.com> Cc: linux-kernel@...r.kernel.org Subject: Re: Arithmetic overflow in may_expand_vm() Hi, On Thu, Oct 15, 2009 at 10:24:51AM -0700, Earl Chew wrote: > This code currently reads: > > >int may_expand_vm(struct mm_struct *mm, unsigned long npages) > >{ > > unsigned long cur = mm->total_vm; /* pages */ > > unsigned long lim; > > > > lim = current->signal->rlim[RLIMIT_AS].rlim_cur >> PAGE_SHIFT; > > > > if (cur + npages > lim) > > return 0; > > return 1; > >} > > If npages is stupendously large, the failure predicate may > return a false negative due to (cur + npages) overflowing and > wrapping. Can this really happen? npages always originates in a value of byte granularity, giving a theoretical maximum of ~0UL >> PAGE_SHIFT (checking for more than the number of addressable bytes just makes no sense). And mm->total_vm is always PAGE_SIZE times smaller than total user address space (which in turn is always less than ~0UL). So I can not see this overflow being possible with PAGE_SHIFT > 0. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists