Preventing Web-based Directory Enumeration Attacks

Here are some thoughts on how to prevent directory-name guessing (e.g. checking for http://host/admin, http://host/private, etc.).

The Attack

Several enumeration tools such as http-dir-enum and DirBuster are able to quickly list directories which exist on a website. Other more general tools such as nikto and nessus can also perform this attack, but typically go into less depth.

The simplest form of the directory-enumeration attack is to make a request for each directory name in turn from dictionary file of popular directory names. For each request, note the HTTP response code, e.g.

http://host/admin (401)
http://host/cgi-bin (403)
http://host/test (404)
http://host/logs (200)
http://host/awstats (404)
http://host/scripts (404)

The attacker figures out which HTTP response code to ignore (404 in this example). All directories which don’t return a 404 can be assumed to exist.

It is this most simple form of the attack that I want to concentrate on. Other more sophisticated attacks which actually parse the text of responses to determine if the directory exists are possible (this is necessary for example when the server sends 200 responses for everything). However, I won’t be considering how to defend against these in this post. Spidering is also a good way to figure out the directory structure of the parts of a site which are linked to, but again this attack is not being considered here.

 

Why defend against this attack?

Is it really such a big problem if attackers can easily figure out the directory structure of your website? Well, sometimes yes, sometimes no. If you’ve done everything else right (e.g. disabled directory list, removed unneeded files), it’s less of a risk.

However, enumerating directories is a useful precursor to other attacks (just like port-scanning is when performing a network-based assessment). It can sometimes find the administration page for the site, an old backup directory, the logs directory, another app installed within the same site. Even if directory listing is disabled, once an attacker knows that a directory exists, they can start guessing filenames (test.txt, index.php.bak, backup.tar.gz).

I think a sensible target would be to hide any directories that an attacker doesn’t need to see. An example would be if you’re using a CMS which has a large directory structure, only a fraction of the directories actually present in the web root will need to be visible to users in URLs. We can neither remove nor deny access to these directories as they’re required for the CMS to work.

So, how can we hide directory names from simple enumeration attacks…

The Defense

We want to return identical looking 404 responses for directories that do exist (e.g. /secret) and those that don’t (e.g. /nosuchdir).

The best I’ve come up with so far is using mod_rewrite on apache to rewrite anything that looks like a request for a directory into a request for a file that doesn’t exist. Directories that do exist are rewritten in the same way as those that don’t.

$ cat htdocs/.htaccess
RewriteEngine on
RewriteRule ^([a-zA-Z0-9_/-]+)/?$ $1  [L]

In the example above, we assume that directories are composed only of letters, numbers, underscores, hyphens and forward slashes. This is a fairly good assumption. Most requests for non-directories will contain a dot (e.g. index.asp, index.php, index.html).

There’s one big problem with this Apache-technique. It only works when the enumeration attack places a forward-slash at the end of the directory name, i.e. it would defend against an attacker that was looking for:

docs/
docs/private/
stats/
cgi-bin/

However, if the trailing slash is omitted, Apache helpfully replies with a 301 redirect for directories which exist (e.g. from /foo to /foo/). I don’t know how to stop apache from doing this. The technique therefore wouldn’t defend against:

docs
docs/private
stats
cgi-bin

I’d be interested to hear from you if you can refine this defense, or perhaps apply it to another web server. Mail me. It seems that it should be possible to defend against the type of enumeration attack described. It’s just a case of figuring out how tweak Apache (or which ever server you’re using).

Setting the “Hidden” Attribite to Hide Files in IIS

Hiding directories in IIS seems to be as easy as setting the “hidden” attribute:

cd c:\Inetpub\wwwroot

attrib +h myprivatedirectory

Now when an attacker browses to http://yoursite/myprivatedirectory they will get a 404 “Not Found” message instead of a 403 “Directory Listing Denied”.  However, files inside the directory are still accessible (e.g. http://yoursite/myprivatedirectory/somefile.htm).  This is a simple way to defeat directory enumeration attacks from tools such as http-dir-enum and DirBuster.

I tested these settings on IIS6.

Limitations

Obviously, if your site deliberately links to directories (e.g. http://host/coolstuff), this defense won’t work for the directory that are linked to, i.e. the “coolstuff” page has to return a different response. That’s what it’s supposed to do. It only works if you link to pages of the form http://host/dir/file.ext.

It won’t hide directories if they appear in normal URLs, e.g. it’s obvious that “dir” exists in the URL http://host/dir/file.ext.

 

pentest

Blog