lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-ID: <50C647BF.4070209@gmail.com> Date: Mon, 10 Dec 2012 20:36:15 +0000 From: Denis McMahon <denismfmcmahon@...il.com> To: full-disclosure@...ts.grok.org.uk Subject: Re: Google's robots.txt handling On 10/12/12 19:25, Hurgel Bumpf wrote: > I tried to contact google, but as they didn't answer my email, I do forward this to FD. > This shouldn't be a discussion about bad practice but the google feature itself. I seem to recall that the robots.txt exclusion standard was fairly common before I ever heard of google, I'm thinking of the mid to late 1990s when alta vista (which at the time was I think a technology demonstrator for VAX Alpha RISC hardware?) was the goto of search engines. Rgds Denis McMahon _______________________________________________ Full-Disclosure - We believe in it. Charter: http://lists.grok.org.uk/full-disclosure-charter.html Hosted and sponsored by Secunia - http://secunia.com/
Powered by blists - more mailing lists