forum.coppermine-gallery.net

Support => cpg1.4.x Support => Older/other versions => cpg1.4 permissions => Topic started by: macsimoin on July 02, 2006, 04:46:50 am

Title: Not sure if my robots.txt is okay
Post by: macsimoin on July 02, 2006, 04:46:50 am

Since I've registered a new domain and relocated my gallery into the root of the new domain, I added a robots.txt file there as well. Can someone have a look at this and let me know if it looks okay or if it's in the right place?

Code: [Select]

#
# robots.txt for http://www.kiltphotos.com/ and friends
#

User-agent: Zealbot
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: Fetch
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: WebZIP
Disallow: /

User-agent: linko
Disallow: /

User-agent: HTTrack
Disallow: /

User-agent: Microsoft.URL.Control
Disallow: /

User-agent: Xenu
Disallow: /

User-agent: larbin
Disallow: /

User-agent: libwww
Disallow: /

User-agent: ZyBORG
Disallow: /

User-agent: Download Ninja
Disallow: /

User-agent: wget
Disallow: /

User-agent: grub-client
Disallow: /

User-agent: k2spider
Disallow: /

User-agent: NPBot
Disallow: /

User-agent: WebReaper
Disallow: /

User-agent: Konqueror
Disallow: /

User-agent: msnbot
Disallow:

User-agent: psbot
Disallow:

User-agent: Googlebot
Disallow:

User-agent: Googlebot-Image
Disallow:

User-agent: *
Disallow: /albums/
Disallow: /bridge/
Disallow: /docs/
Disallow: /images/
Disallow: /include/
Disallow: /lang/
Disallow: /logs/
Disallow: /plugins/
Disallow: /README_FILES/
Disallow: /sql/
Disallow: /themes/
Thanks. :)