Block AD Networks with a custom /etc/hosts File

If you are interested in blocking advertising from specific domains, globally, you can edit your hosts file to redirect that any undesired host to your “localhost” – 127.0.0.1. This will stop your browser from visiting that advertising server completely.

Update 12/2016

There’s a project called pi-hole which does the same thing for all the devices on your network. Take a look at it if you are comfortable with Linux on any platform. A friend claims he’s running it on a raspberry pi, which I believe. Basically, it becomes the DNS server for your home network and blocks about 130K known ad, spy, tracking and malware websites. It seems to be pretty conservative. My old hosts file was blocking only 12K sites, so that just goes to show how much tracking, ads, spyware and malware sites have expanded over the years.

How Does This Work?

DNS is the telephone book of the internet. Generally, your computer/device uses 2 methods to locate the other server over the network. In the beginning, only the /etc/hosts file was used. Over time, that file became huge with all the DARPA and University hosts listed. Then DNS was created to turn that lookup into a service for everyone. Today almost every operating system performs name-to-IP lookups by

  1. checking the /etc/hosts file – if not found
  2. sending a DNS query
    Any hostname found in the hosts file prevents further queries.

More background on DNS

DNS is critical to the security of the internet too. Any machine that can alter the name-to-IP lookup can trick your system into trusting a remote server with SSL without your knowledge. The safety of every SSL certificate used to protect all your online shopping are completely depended on DNS.

So, by changing your lookup for an ad-server to point to your local machine, you’ve just shortcut their ability to do anything to your PC. Isn’t that really what you want?

Other Benefits

  • Speeds up your internet use since the local hosts file by default is checked first, before DNS is used.
  • Not just for browsers, but works for custom apps like iTunes, RealPlayer, Twitter clients or anything, since it works at the OS name resolution level. It doesn’t care if this is UDP or TCP traffic.
  • Platform independent; portables, OSX, Linux and even MS-Windows. If the system uses IP (internet protocol), then it has an /etc/hosts file somewhere.
  • Blocks spyware and malware too, if those server locations are added to the file. That increases the safety of your networking experience.

Which File exactly?

On UNIX-like operating systems, the file is /etc/hosts.
On MS-Windows, the file is “SystemRoot\system32\drivers\etc\hosts”. Usually, on the C: drive.
I don’t know where it is on OSX, but suspect it is /etc/hosts too.

Sample Entries
For example:

127.0.0.1 ad.doubleclick.com
127.0.0.1 ad.doubleclick.net
127.0.0.1 ad.ae.doubleclick.net
127.0.0.1 ad.ar.doubleclick.net
127.0.0.1 ad.at.doubleclick.net
127.0.0.1 ad.au.doubleclick.be
127.0.0.1 ad.au.doubleclick.net
127.0.0.1 ad.be.doubleclick.be
127.0.0.1 ad.be.doubleclick.net
127.0.0.1 ad.doubleclick.be
127.0.0.1 ad.doubleclick.de

BTW, this is just a small number of DoubleClick’s domains – perhaps 0.5%. There are almost 300 doubleclick entries alone. Order doesn’t matter, neither does upper or lower case letters, either is acceptable in DNS.

Stop Trackers Too

If you don’t want to be tracked by them ever again. You won’t need to use “opt-out” cookies either. My new /etc/hosts file has over 12,000 entries. That’s 12K worth of advertising network hosts that are not using bandwidth as I surf. The old one used for the last decade was only 1200 lines, yet still highly effective. That growth says something about internet tracking these days.

Stop the Bad Guys Out There

There are a few people on the internet who are simply bad. Picking them out from the, shall we say, aggressive advertisers can be difficult, but a few definitely come to mind like cnbc7.com. These guys need to be blocked on every PC, period. Add this line to every computer hosts file, every DNS, every place that you can. They seem to only push spyware, malware and viruses.

127.0.0.1       cnbc7.com

You’ll never be bothered by them again. I just received a spam email with a redirector to cnbc7 about 30 minutes ago. If I’d accidentally clicked on that link to a respectable website that happens to have a redirector inside the URL, my PC could easily have been compromised. BLOCKED.
Please don’t forget to do this for your own protection. This has nothing to do with the CNBC cable TV network.

Get a Community Created File

You can create your own, but why? You can find pre-made lists of ad network files by googling or get a community provided file from here. There are many other versions and websites producing ad-blocking hosts files. Don’t worry about a larger file making your PC slower. Even the largest files will actually make your PC feel faster.

There’s an easy way to check these files for something nasty, since you are replacing your name-to-IP lookup with them. Search for any lines that do not start with ‘127.0.0.1’ or a comment. 

egrep -v '^127.0.0.1|^#' /etc/hosts |more

I didn’t find any issues.

Caution

Some care needs to be taken when editing this file. The format across all platforms is identical, but you probably want to merge the current file on your computer/device with one that you discover online. At some point in the future, you may want to be tracked or visit a website that you previously blocked. Just use a comment the specific server in the “hosts” file. For example, I block facebook.com and www.facebook.com since I don’t appreciate their fingers on almost every page across the internet. Recently, a TV network that I like to watch on the web forced all connections through facebook, not just as a way to track, but to proxy their content THROUGH facebook servers. I must allow facebook access for that 1 hour every week to watch that show. To make this easy and reduce risk, I have 3 /etc/hosts files.
hosts, hosts.noFB, hosts.FB. I copy the .nfFB or .FB file into the real hosts file as needed.

If you do block an entire website, like, www.facebook.com, here’s what your browser will see.

Blocked Server

Any changes take effect immediately on write of the file.

If the blocked domain is just for advertisements, that part of the screen will just be blank. When I visit http://cnn.com, their website is almost useful since almost every ad is blocked.

Be certain to retain any hosts listed at the top of the file.
Be certain you leave anything that looks like this:

127.0.0.1 localhost
127.0.1.1 my-real-hostname
::1 my-ipv6-addr

since removing those lines can be really bad for your PC networking. This will probably break a few things.

On all modern operating systems, this file is protected, so you need root/administrative access to change it. On MS-Windows, be certain you use Run As Administrator to edit the file. For more help with that see Microsoft’s support site. For Linux and other UNIX-based operating systems, you’ll need to use sudo or gain root with an su -.

Some versions of OSX don’t follow the /etc/hosts method. Here’s an outdated Apple support article to explain. I’m completely unfamiliar with OSX. A number of people have had issues, so be prepared to put things back quickly.

This should work on smartphones too. My Nokia tablet has an /etc/hosts file and it works there exactly the same. I can only assume it works the same way on Android, tablets and iPhones, but I do not know.

Some Routers Will Let You Add a Local hosts

I’ve never seen a consumer router that allowed this, but some firewall software will let you centrally manage a local /etc/hosts file for all the machines/devices on your subnet. Smoothwall will. I suspect similar tools will as well. This could be really helpful, otherwise adding a modified /etc/hosts file to every PC is needed to get the full impact.

A Final Option

When you block any website with an entry in your /etc/hosts it is like a nuclear bomb. There is nothing those hosts can do to load anything on your PC.

If you don’t want to always block content from a specific site or server, then using the /etc/hosts file may not be the best choice. Currently free websites that depend on advertising revenue would quickly disappear with we blocked all advertising. You will need to determine whether an advertising network that insist on obnoxious Flash ads or JavaScript ads should be blocked completely.

Ansible to Maintain /etc/hosts

Ansible is a devops tool that is easy to setup and run when compared to the other 10 popular tools.

$ more tasks/common_etc_hosts.yml

-

- name: Copy /etc/hosts

  template: src=templates/etc_hosts.j2 dest=/etc/hosts backup=yes 

            owner=root group=root mode=0644
is my ansible task for this. templates/etc_hosts.j2 is the ansible template file that gets modified as it is transmitted to each host. The template just puts the actual hostname and hostname.domainname into the top of the file. This is a built-in feature of Ansible. If you have 4+ Unix-like machines, setting up Ansible to manage them is a good idea. BSD/Linux are included.

What’s the Difference Between 127.0.0.1 and 0.0.0.0?

In the end, it doesn’t matter what the RFCs for compliance say. How does it really work, that is the question.

On some platforms, 0.0.0.0 doesn’t work.
On others, it asks the local machine or default router to handle the request on any interface available. That probably is NOT what you want.
On others, it is faster than 127.0.0.1 and works as needed.

That’s the straight poop. On my Linux machines, I’m using 127.0.0.1 so the requests fail quickly and in the expected way. This is on desktop and server, including web servers.

On a Win7 system, the 127.0.0.1 method is working fine for years, but it hasn’t been on the internet except for patching for about 3 yrs. It is not used as a normal desktop. People at businesses in an AD-Domain have reported issues.

Other Security Pages on this site: