Advanced search  

News:

cpg1.5.48 Security release - upgrade mandatory!
The Coppermine development team is releasing a security update for Coppermine in order to counter a recently discovered vulnerability. It is important that all users who run version cpg1.5.46 or older update to this latest version as soon as possible.
[more]

Pages: [1]   Go Down

Author Topic: Not sure if my robots.txt is okay  (Read 3116 times)

0 Members and 1 Guest are viewing this topic.

macsimoin

  • Guest
Not sure if my robots.txt is okay
« on: July 02, 2006, 04:46:50 am »

Since I've registered a new domain and relocated my gallery into the root of the new domain, I added a robots.txt file there as well. Can someone have a look at this and let me know if it looks okay or if it's in the right place?

Code: [Select]
#
# robots.txt for http://www.kiltphotos.com/ and friends
#

User-agent: Zealbot
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: Fetch
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: WebZIP
Disallow: /

User-agent: linko
Disallow: /

User-agent: HTTrack
Disallow: /

User-agent: Microsoft.URL.Control
Disallow: /

User-agent: Xenu
Disallow: /

User-agent: larbin
Disallow: /

User-agent: libwww
Disallow: /

User-agent: ZyBORG
Disallow: /

User-agent: Download Ninja
Disallow: /

User-agent: wget
Disallow: /

User-agent: grub-client
Disallow: /

User-agent: k2spider
Disallow: /

User-agent: NPBot
Disallow: /

User-agent: WebReaper
Disallow: /

User-agent: Konqueror
Disallow: /

User-agent: msnbot
Disallow:

User-agent: psbot
Disallow:

User-agent: Googlebot
Disallow:

User-agent: Googlebot-Image
Disallow:

User-agent: *
Disallow: /albums/
Disallow: /bridge/
Disallow: /docs/
Disallow: /images/
Disallow: /include/
Disallow: /lang/
Disallow: /logs/
Disallow: /plugins/
Disallow: /README_FILES/
Disallow: /sql/
Disallow: /themes/

Thanks. :)
Logged
Pages: [1]   Go Up
 

Page created in 0.018 seconds with 17 queries.