full-disclosure - Google Sitemap Directory and File Enumeration 0day

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date: Thu, 12 Oct 2006 08:12:13 -0700
From: Adam Muntner <adam.muntner@...etmove.com>
To: full-disclosure@...ts.grok.org.uk
Subject: Google Sitemap Directory and File Enumeration 0day

While playing with the Google Webmaster tools, I came across the
“Sitemap” XML protocol which is used to inform search engines about
pages on your website that are available for crawling.

The protocol spec is at
https://www.google.com/webmasters/sitemaps/docs/en/protocol.html

Think of this as the anti-robots.txt - instead of URLs with Disallow:
tags, you have URLs for which the web administrator is saying “Index
me.”

Sitemap makes anti-forensics Google hacking more productive. It’s only a
matter of time before enumeration tools like Wikto use it the same way
that they use robots.txt to locate files.

There are two interesting security-related issues with Sitemap, one
significantly more interesting than the other.

First, you can find pages with it that aren’t indexed by Google. The
Sitemap protocol spec says “Using this protocol does not guarantee that
your webpages will be included in search indexes. (Note that using this
protocol will not influence the way your pages are ranked by Google.)”
This is the lesser of the interesting points.

Far more interesting - you can find pages in the sitemap.xml which would
not be indexed if it weren’t for the Sitemap protocol…

You can find some interesting stuff by querying for Sitemap files.

“.htaccess” inurl:sitemap filetype:xml
“global.asa” inurl:sitemap filetype:xml

Whew.

There are a LOT of automagic-generation Sitemap scripts out there which
create Sitemap.xml files not by spidering a site, as they should… but by
reading the contents of directories inside the web root from the local
filesystem and creating the Sitemap.xml file from that.

Ouch.

I don’t blame Google - I think the Sitemap protocol is a pretty good
idea, if though the guys at the Black Hat SEO forum think it doesn't
help you get better rankings. It tells the Google search engine where to
find pages which otherwise might not get indexed. 

Due to a plethora of rotten Sitemap.xml generation scripts, this is a
directory and file enumeration issue that is going to be with us for a
long, long time to come.

     Adam Muntner, CISSP | 
                 Partner | 
         QuietMove, Inc. | w: http://www.quietmove.com
Securing the Nexus Between People, Technology, and Information.
                       ((Q))

Download attachment "signature.asc" of type "application/pgp-signature" (192 bytes)

_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/