User-Agent: * Disallow: /admin/ Disallow: /archive/ Disallow: /db/ Disallow: /files/