Page 1 of 1

Spam Filters Gone Awry

Posted: Fri Apr 23, 2010 2:54 am
by Sharkey
Apparently my ongoing efforts to keep spam off the site has gone a little too far and is now barring some of our most illustrious members from accessing the site. I'm working on the problem, but if anyone here hears from another member that they are having problems seeing the forum or the rest of the site, let me know.

Hopefully, I've managed to roll back the access restrictions to allow everyone who has a right to be here back in without opening the door to crud-spreading spammers. Time will tell.

Posted: Fri Apr 23, 2010 5:45 am
by Sharkey
It looks like things are fixed enough to run properly in the short term. Apologies to those who got caught up in the spammer's net.

There are several layers of spam and hacking protection on the site and forum.

The first line of defence is at the server level. The server software (Apache 2.0) takes the information contained in each page request and compares it to a file named ".htaccess". This file can contain rules for routing of requests, page substitutions, server memory limits, etc. In the case of my domain, it also contains about 400 lines of IP addresses that are banned from seeing anything at all. This is a very efficient way to preventing access by any IP addresses used by potential hackers or spammers. It stops them dead in their tracks by giving them a "Forbidden" message. No "Not Found" page, no redirect, nothing, just a blank page. Very efficient with my bandwidth and server load. Here's a clip of some lines of banned IP addresses:

Code: Select all

deny from 217.27.208.0/20       # IEC-NET - Hungary
deny from 217.70.32.0/23        # LEVONLINE - Sweden
deny from 217.77.32.0/20        # NO-LIT - Norway
deny from 217.115.0.0/16        # NETSIGN - Germany
deny from 217.139.0.0/16        # ADSL - Egypt
deny from 217.156.0.0/16        # BIOS - Romania
deny from 217.197.0.0/16        # NWT-NET - Czech Republic
deny from 217.218.0.0/16        # Area-6 - Iran
deny from 218.7.0.0/16          # UNICOM - China
deny from 218.8.0.0/15          # UNICOM - China
deny from 218.10.0.0/16         # UNICOM - China
deny from 218.11.0.0/16         # UNICOM
deny from 218.12.0.0/14         # CHINANET
deny from 218.16.0.0/15         # CHINANET
deny from 218.18.0.0/16         # CHINANET
deny from 218.27.0.0/16         # UNICOM - China
deny from 218.28.0.0/15         # CULTURE-SPEED - China
Each of the IP addresses is expressed as a range, called a "CIDR". A trailing /16 means that 65,536 IP addresses are banned, while a trailing /14 means that 262,144 addresses are banned. It is a very effective means of keeping whole regions from accessing my site. Like I give a rip about Iran or China... Everything after the "#" on each line is ignored by the server, and serves as a "comment", helping me keep track of where each address is registered to.

.htaccess is very powerful, but it's a brute-force brick wall, and if not carefully implemented, can wreck havoc with legitimate users. A misplaced "." or left out "#" can bring the site crashing down.

The next protection is software called "Bad Behavior". This set of scripts examines the headers of page requests and makes intelligent decisions based on sets of rules. Most spammers/hackers use proxy connections to try and spread their filth. Many use browsers which send distinctive user-agent headers. It's a more precise way of pinpointing bogus page requests. So far, BB has captured and prevented about 14,000 false posts in the forum since December of 2009. The failure mode of BB is to disallow the page request, but the error message includes an email address which can be used to alert me that the denial of service is mistaken. Only a very few forumites have been caught up in this protection, and I have been able to allow them access through use of a "whitelist" that tells BB that they are OK.

The forum software itself has been modified and optimized to weed out spam attacks. The "Insult-A-Spammer" question/answer set that Guest posters and new registrants see is an example. There are many other triggers that toss fake forum posts and registrations in the digital garbage can. Mostly, they are transparent to legitimate users.

I guess the last bastion of truth is myself as Administrator, TMAX as Moderator and the users themselves, acting as the forum "Neighborhood Watch". If any questionable material or users do get through the outer protections, it doesn't take long before someone notices and reports it for action.

Anyway, the most recent outage was caused by the .htaccess file. I still haven't found the corrupted code, which was something I inserted last night. I'll be running more tests over the next couple of days to find out where the problem is. In the meantime, we are running on a week-old backup file that is only missing a few added IP addresses. I update this file nearly daily, so there's ample opportunity for me to add the spammer addresses back in, they always come back for more.

spam protection

Posted: Fri Apr 23, 2010 6:20 am
by graydawg
WOW, way over my scope of comprehension, GOOD LUCK MR. SHARKEY

Posted: Fri Apr 23, 2010 6:20 am
by Jones'n4chrome
Thanks Sharkey.

Re: spam protection

Posted: Fri Apr 23, 2010 6:24 am
by Jones'n4chrome
graydawg wrote:WOW, way over my scope of comprehension, GOOD LUCK MR. SHARKEY
It's over most of us, but if you read between the lines it says, "Sharkey's kickin ass" :D

Posted: Fri Apr 23, 2010 9:41 am
by Granny
Sharkey :cry: , sorry bout all of that trouble we cause you. I got to admit, I thought I had been booted off the site :D for some reason. Do you know how much sleep I lost over that? :shock: :roll: :oops:
I am glad you got things figured out. It is good to be back, it is even better to know "I" didn't get booted. :) Have a good day Mr. Sharkey.

Posted: Fri Apr 23, 2010 10:17 am
by Jones'n4chrome
Granny wrote:Sharkey :cry: , sorry bout all of that trouble we cause you. I got to admit, I thought I had been booted off the site :D for some reason. Do you know how much sleep I lost over that? :shock: :roll: :oops:
I am glad you got things figured out. It is good to be back, it is even better to know "I" didn't get booted. :) Have a good day Mr. Sharkey.
Ginger,

Kissing up to the boss won't help. So you better behave! :D

By the way, who is "we" anyhow?

Yours truly,
The Forbidden

Posted: Fri Apr 23, 2010 1:49 pm
by Rudy
Who is that man behind the curtain? It is the wizard. Omnipotent and ever gracious.