Online Advertising
Referer spam
Referer spam is a kind of
spamdexing (spamming
aimed at
search engines). The technique involves making repeated
web site requests using a fake referer url pointing to a spam-advertised
site. Sites that publicize their access logs, including referer
statistics, will then also link to the spammer's site.
This benefits the spammer because of the free link, and also gives the
spammer's site improved search engine link placement due to link-counting
algorithms that search engines use.
Technical solutions
As with e-mail spam, web site operators who receive unwanted referer spam may
respond using filtering and blocking. Some web sites receive so many referer
spam hits that they amount to a denial of service attack on the server because there are not enough
resources left on the server to handle legitimate traffic.
Word-based filtering
An example configuration fragment for filtering using the
Apache server is as follows:
RewriteEngine On
# A known spamming site
RewriteCond %{HTTP_REFERER} ^http://(.*\.)?egolddomain.(com|net)(/.*)?$ [NC,OR]
# Various "porno" words when separated by word breaks are probably spam
RewriteCond %{HTTP_REFERER} ^http://.*(\b)porn(o(graph(y|er))?)?(\b).* [NC,OR]
# Referer URLs with excessive "-" characters are probably spammers
RewriteCond %{HTTP_REFERER} ^(http://www.)[a-z]+-[a-z]+- [NC,OR]
# You can add as many rules as desired following the pattern of the previous line
# Set an environment variable "BAD_GUY" so we can send their logs to a different file
# Be sure to leave out OR on the last RewriteCond, or your RewriteRule will -always- be
# executed in some versions of Apache
RewriteRule ^(.*) %{HTTP_REFERER} [R=301,E=BAD_GUY:1,L]
# Because we have set E=BAD_GUY above, we can do this in our log file:
CustomLog /var/log/apache/access.log combined env=!BAD_GUY
CustomLog /var/log/apache/access_bad.log combined env=BAD_GUY
The "fake" web site hits will go to access_bad.log, whereas normal traffic
goes to access.log. The "RewriteCond" lines contain
Regular expressions that can be used to match any undesirable traffic,
thereby excluding it.
IP-based filtering
If most of the spam is coming from a few
IP addresses, or is requesting a certain page (that may no longer exist on the
server) the Apache server may also be configured to deny access via the configuration
file, (often named httpd.conf), based on either IP address or the name of the
requested file by adding lines like the following:
# Deny access based on the filename or path of the requested file
<Location /links>
Deny from all
</Location>
# Deny access based on the IP address or host name of the offending site
<Directory /usr/local/etc/httpd/htdocs>
Deny from 72.36.244.166
Deny from big-bad-spammer-blah-blah.com
</Directory>
A good statistics analysis program will allow you to target the worst
offenders.
Advanced filtering using mod_security
A third solution for Apache is to install ModSecurity, which allows you to
deny requests based on any variable from the server environment, such as referer, request, IP, host, etc.
See also
External links
Home | Up | Cloaking | Doorway page | Scraper site | Spam blogs | Spam in blogs | Spam mass | Made For AdSense | Bookmark spam | Referer spam | TrustRank
|
