| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <1355167539.72493.YahooMailNeo@web171705.mail.ir2.yahoo.com> Date: Mon, 10 Dec 2012 19:25:39 +0000 (GMT) From: Hurgel Bumpf <l0rd_lunatic@...oo.com> To: "full-disclosure@...ts.grok.org.uk" <full-disclosure@...ts.grok.org.uk> Subject: Google's robots.txt handling Hi list, i tried to contact google, but as they didn't answer my email, i do forward this to FD. This "security" feature is not cleary a google vulnerability, but exposes websites informations that are not really intended to be public. (Additionally i have to say that i advocate robots.txt files without sensitive content and working security mechanisms.) Here is an example: An admin has a public webservice running with folders containing sensitive informations. Enter these folders in his robots.txt and "protect" them from the indexing process of spiders. As he doesn't want the /admin/ gui to appear in the search results he also puts his /admin in the robots text and finaly makes a backup to the folder /backup. Nevertheless these folders arent browsable but they might contain f(a)iles with easy to guess namestructures, non-encrypted authentications (simple AUTH) , you name it... Without a robots.txt nobody would know about the existance of these folders, but as some folders might be linked somewhere, these folders might appear in search results when not defined in the robots.txt The admin finds himself in a catch-22 situation where he seems to prefer the robots.txt file. Long story short. Although google accepts and respects the directives of the robots.txt file, google INDEXES these files. This my concern. http://www.google.com/search?q=inurl:robots.txt+filetype%3Atxt+Disallow%3A+%2Fadmin http://www.google.com/search?q=inurl:robots.txt+filetype%3Atxt+Disallow%3A+%2Fbackup http://www.google.com/search?q=inurl:robots.txt+filetype%3Atxt+Disallow%3A+%2Fpassword As these searches can be used less for targeted attacks, they more can be used to find victims. http://www.google.com/search?q=inurl:robots.txt+filetype%3Atxt+%2FDisallow%3A+wp-admin http://www.google.com/search?q=inurl:robots.txt+filetype%3Atxt+%2FDisallow%3A+typo3 <Just be creative> This shouldn't be a discussion about bad practice but the google feature itself. Indexing a file which is used to prevent indexing.. isn't that just paradox and hypocrite? Thanks, Conan the bavarian _______________________________________________ Full-Disclosure - We believe in it. Charter: http://lists.grok.org.uk/full-disclosure-charter.html Hosted and sponsored by Secunia - http://secunia.com/
Powered by blists - more mailing lists