Help - Dealing with Abusive Aggregators
How do you deal with abusive web aggregators?
You know, those automatic programs that search out and grab web content so that others can see it? Usually, this is a good thing for a web site, since it makes your content visible to others that wouldn’t normally see it. That is, until it becomes abusive.
Do you block the entire netblock or do something else?
I did a little research on the specific program they are using and saw lots of complaints. A number of other websites had even emailed the CIO of the company asking for them to follow web standards and only pull changed data. No response was ever returned to their emails.
It was a hard decision, but we decided to block all their corporate IP addresses from accessing any of our websites. We would rather not do that. They left us with little choice due to the abuse.
/sbin/iptables -I INPUT -s xx.xx.xx.xx/28 -j DROPBased on your corporate network netmask, you must be a small operator. Goodbye abusers. We’d love some advice on dealing with abusive aggregators. Please drop a note in the comments.
Comment Spam
In the last few days, the amount of daily comment spam has increased by about 10×. Moderation is definitely good.
Trackbacks
Use the following link to trackback from your own site:
https://blog.jdpfu.com/trackbacks?article_id=732
