linux-kernel - Re: smart cache. ist is possible?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <49BE4DCB.1000705@draigBrady.com>
Date:	Mon, 16 Mar 2009 13:02:03 +0000
From:	Pádraig Brady <P@...igBrady.com>
To:	Alexey Fisher <bug-track@...her-privat.net>
CC:	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: smart cache. ist is possible?

Alexey Fisher wrote:
> Hallo all.
> I found for my self how great is cache in linux. If read one file from
> disk, so i don't need to do it second time, chace will do the job. It
> speed up thing greatly. But i found it not working with realy big files.
> Like i have 4GB RAM, so if i read a file like 4.6GB, cache won't work.
> Is it possible to have some sort of smart cache wich will read for
> exaplme 1GB from disk and other part from cache?
> 
> here is some simple test:
> =====================cache not working==========================
> dd if=dvd.iso of=/dev/null
> 9017680+0 Datensätze ein
> 9017680+0 Datensätze aus
> 4617052160 Bytes (4,6 GB) kopiert, 90,7817 s, *50,9 MB/s*
> 
> dd if=dvd.iso of=/dev/null
> 9017680+0 Datensätze ein
> 9017680+0 Datensätze aus
> 4617052160 Bytes (4,6 GB) kopiert, 90,7817 s, *50,9 MB/s*
> ===============================================================

Right. The cache is being cycled.
I.E. the block you want is never in the cache
as it has previously clobbered be data your reading in.
That's just a consequence of preferring to cache blocks
you have recently read from file, over older blocks.
This is usually the right thing to do, but exactly
the wrong thing to do in your case.

So you would need to provide more info to the
kernel for it to behave as you want.
I.E. never evict a block belonging to the same file
as the block you're trying to insert.

I wonder should posix_fadvise(...POSIX_FADV_SEQUENTIAL)
do what you want. I don't think it does at present.
Note dd doesn't use posix_fadvise() yet, and it probably
should (at least for POSIX_FADV_DONTNEED).
Pity there is no interface like Robert Love's old O_STREAM
patch to just specify the intent for an fd rather than
worrying about ranges.

cheers,
Pádraig.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/