Advanced search  

News:

cpg1.5.48 Security release - upgrade mandatory!
The Coppermine development team is releasing a security update for Coppermine in order to counter a recently discovered vulnerability. It is important that all users who run version cpg1.5.46 or older update to this latest version as soon as possible.
[more]

Pages: [1]   Go Down

Author Topic: Blocking lechers and site rippers with mod_rewrite and php, MySQL  (Read 17682 times)

0 Members and 1 Guest are viewing this topic.

remdex

  • Coppermine newbie
  • Offline Offline
  • Posts: 17
    • My anime wallpapers database

Hi,

Just wanned share my experience fighting against site rippers and lechers. I won't paste link to my gallery, just share some solutions protecting your own gallery's from rippers and traffic thief's.

First part solutions consists of main mod_rewrite rules:
(Dont forget replace <variable> with your site variables :) )

Code: [Select]
RewriteEngine On

#Disable HotLinking
RewriteCond %{REQUEST_FILENAME} .*jpg$|.*bmp$|.*jpeg$|.*gif$|.*png$ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !<your_site>\.org [NC]
RewriteCond %{HTTP_REFERER} !<friend_site>\.lt [NC]
RewriteCond %{HTTP_REFERER} !youtube\.com [NC]
RewriteCond %{HTTP_REFERER} !google\. [NC]
RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]
RewriteRule ^.+ - [F]

#Disable direct access to fullsize script
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{QUERY_STRING} fullsize=1$
RewriteRule ^.+ - [F]

#Image can be accesed only from fullsize script.
RewriteCond %{REQUEST_FILENAME} .*jpg$|.*bmp$|.*jpeg$|.*gif$|.*png$ [NC]
RewriteCond %{REQUEST_FILENAME} albums [NC]
RewriteCond %{REQUEST_FILENAME} !thumb [NC]
RewriteCond %{REQUEST_FILENAME} !normal [NC]
RewriteCond %{REQUEST_FILENAME} !userpics [NC]
RewriteCond %{HTTP_REFERER} !google\. [NC]
RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]
RewriteCond %{HTTP_USER_AGENT} !google [NC]
RewriteCond %{HTTP_REFERER} !fullsize=1
RewriteRule ^.+ - [F]

#Tricky rule Witch i will explain later
#In gennaraly it passes all request to fullsize images to php script where is implemented anti leeching logic.
#Comment out this rule if you wont use script fullsize.php script.
RewriteCond %{REQUEST_FILENAME} .*jpg$|.*bmp$|.*jpeg$|.*gif$|.*png$ [NC]
RewriteCond %{REQUEST_FILENAME} albums [NC]
RewriteCond %{REQUEST_FILENAME} !thumb [NC]
RewriteCond %{REQUEST_FILENAME} !normal [NC]
RewriteCond %{REQUEST_FILENAME} !resample [NC]
RewriteCond %{REQUEST_FILENAME} !userpics [NC]
RewriteCond %{HTTP_REFERER} !google\. [NC]
RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]
RewriteCond %{HTTP_USER_AGENT} !google [NC]
#RewriteCond %{HTTP_REFERER} fullsize=1
#RewriteRule ^.+ - [F]
RewriteRule ^(.*)$ fullsize.php?path=%{REQUEST_FILENAME}


#block site rippers agents
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
    RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
    RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
    RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
    RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
    RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
    RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
    RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
    RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Jyxobot [OR]
    RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
    RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
    RewriteCond %{HTTP_USER_AGENT} Twiceler [OR]
    RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
    RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Mail.Ru [OR]
    RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
    RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
    RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} \'Mozilla [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
    RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
    RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
    RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
    RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
    RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
    RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
    RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
    RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
    RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
    RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Twiceler [OR]
    RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
    RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]
    RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
    RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
    RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
    RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
    RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Yeti [OR]
    RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
    RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
    RewriteCond %{HTTP_USER_AGENT} ^wget [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Zeus
    RewriteRule ^.* - [F,L]


fullsize.php script code (put it in root directory):
Code: [Select]
<?php
$time_window_seconds
=35;
$max_hits_per_ip_per_window=2;

define('IN_COPPERMINE'true);
require(
'include/init.inc.php');
cpg_db_query("INSERT INTO ipleechcheck (ip,hittime) VALUES ('{$_SERVER['REMOTE_ADDR']}','".time()."')");
cpg_db_query("DELETE FROM ipleechcheck WHERE hittime < ".(time()-$time_window_seconds)." ");
$result cpg_db_query("SELECT count(*) as total FROM ipleechcheck WHERE ip = '{$_SERVER['REMOTE_ADDR']}' ");

        if (
mysql_num_rows($result) === 1) {
                
$row mysql_fetch_assoc($result);
                
$IPCount $row['total'];
        }

 if( 
$IPCount $max_hits_per_ip_per_window 
{

header('HTTP/1.1 503 Service Temporarily Unavailable');
header('Status: 503 Service Temporarily Unavailable');
header('Retry-After: 3600');
header('X-Powered-By:');
echo 
'<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Temporarily Unavailable</title>
</head><body>
<h1>Service Temporarily Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
</body></html>'
;
exit;

} else {
$fileName $_GET['path'];
if (file_exists($fileName))
{
$extension end(explode('.',$fileName));

switch (strtolower($extension)) {
   case 'png':
    header('Content-type: image/png');
    echo file_get_contents($fileName);
   break;
  
   case 'jpg':
   case 'jpeg':
   header('Content-type: image/jpeg');
    echo file_get_contents($fileName);  
   break;
  
   case 'bmp':
   header('Content-type: image/bmpg');
    echo file_get_contents($fileName);
   break;
  
   default:
   break;
   }
}

}
?>


In this script you can configure two parameters:
1. $time_window_seconds=35; //Perriod witch apliers rule
2. $max_hits_per_ip_per_window=2; //Number of visits

In generally this scripts checks that user cannot hit full size image, not script (mod_rewrite comes in help here :D ), twice in 35 seconds.
So normal users should not see any different unless they will very quickly view full size images. And of course you can replace 503 error with image :) Where is written you have reached bandwidth limit in some seconds etc. Or just show that he was temporary blocked.

MySQL table structure looks like
Code: [Select]
CREATE TABLE IF NOT EXISTS `ipleechcheck` (
`id` bigint(20) NOT NULL auto_increment,
`ip` varchar(255) NOT NULL,
`hittime` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

Waiting for comments :)

P.s sorry for my crappy English :D
« Last Edit: January 29, 2008, 09:41:53 pm by Nibbler »
Logged

Nibbler

  • Guest
Re: Blocking lechers and site rippers with mod_rewrite and php, MySQL
« Reply #1 on: January 29, 2008, 09:41:34 pm »

Thanks for your contribution. Few minor things:

$_GET['path'] looks like it needs to be sanitized.
You have a typo here : Content-type: image/bmpg
The rewrite code needs to be modified if you use non-default names for the albums or userpics directory or thumb_ or normal_ prefixes.
The mysql table could use some optimizations eg. field types and keys and make it a memory table. You don't need that auto_increment field at all.

Good work :)
Logged

TigerClaw

  • Coppermine novice
  • *
  • Offline Offline
  • Posts: 28
Re: Blocking lechers and site rippers with mod_rewrite and php, MySQL
« Reply #2 on: January 29, 2008, 11:05:09 pm »

Good work!

Some user agents don't have ^ before the name like:

RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} \'Mozilla [NC,OR]

I think it's an error, isn't it?
Logged

dke

  • Guest
Re: Blocking lechers and site rippers with mod_rewrite and php, MySQL
« Reply #3 on: February 24, 2008, 03:24:40 am »

very nice, will be testing this out!! thanks.
Logged

paquets

  • Coppermine regular visitor
  • **
  • Offline Offline
  • Posts: 59
Re: Blocking lechers and site rippers with mod_rewrite and php, MySQL
« Reply #4 on: August 30, 2009, 11:59:14 pm »

This mod works very nicely. Is there any way that the error message or the same protection be working on the " normal_" file as well? Right now, I allow people to search my cpg installation, so they can see thumbnails but when they click on it, it brings them to a page that promps them to login or register.

Withthis corrent mod unregistered viewers cannot see the full size but they can see the normal size...

Think this would be possible to achieve?
Logged

armus

  • Coppermine newbie
  • Offline Offline
  • Posts: 14
    • glamour galleries
Re: Blocking lechers and site rippers with mod_rewrite and php, MySQL
« Reply #5 on: January 23, 2010, 12:13:07 am »

I bugged with it too and in some cases hotlink disable has not worked at cpanel but my hosting tech just disabled via cpanel admin and palced these rewrite rules in the .htaccess

Probably you are using cPanel, cna use the fallow simply, i would like to share


Quote
RewriteEngine on


RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://youralloweddomain.com/.*$      [NC]
RewriteCond %{HTTP_REFERER} !^http://www.youralloweddomain.com$      [NC]

RewriteRule .*\.(gif|jpg|jpeg|png)$ http://www.yourdomain.com/ [R,NC]
Logged
Pages: [1]   Go Up
 

Page created in 0.018 seconds with 20 queries.