JDPFu.com 2024

A list, just to get started:

• remove any unused services.
• don’t leave development tools on a production system.
• stay patched. There are thousands of unknown issues on every OS. Fortunately, most attackers use known attacks.
• don’t allow required open ports to be accessed by anyone who doesn’t need that access.
• use multiple methods to secure each open port. Firewalls, authentication, key-based auth, and 2FA, encryption, you get the idea.
• use proper security and proper encryption for the likely attack vectors. What those are depends on the content and tools used on the website.
• run the minimal service needed to accomplish the job. Avoid “kitchen sink” solutions.
• don’t use passwords as a security technique. Passwords are for use the 1st time, then keys should be used and password access disabled.
• lock down the file permissions to provide just-enough permissions required for the program to do the task. No more.
• consider running each service inside a jail/container, especially high-risk services like web, email, and DNS.
• do not run any public-facing services as root.
• do not use a normal userid to run services. Use a specific userid for a specific service.
• use application and service configurations to limit access to only those clients which require access. For example, don’t let the admin pages be available over the internet.
• use kernel parameters to protect against attacks like the syn/ack style.
• never run plain FTP. Use sftp instead.

So … for each service you run, run through that list and lock each down each with multiple layers of protection. We want an edge firewall, but also a firewall on every server and application/service configurations to block unwanted client access.

Validate Any Changes

Then use the available tools to check that what you’ve done is working. Save those scripts so that you can re-run them after every patch gets applied. Consistency. If you are running just a few servers, take notes. Create scripts for the settings, and be consistent. If you run more than 5 servers, it is time to move to DevOps to create the consistency. Consistency is more important than being correct, IMHO. If there is an issue, a DevOps tool will let you fix it quickly on 5-5,000 servers. Every day, every week, make your servers just a little more secure, a little better, a little better tuned. Consistent improvements lead to exceptional systems.

Be the Hacker of Your Site

And learn how to hack your own site. Not all attacks come straight on. Crackers are a craft bunch and they will avoid the front entrance with the steel door when a small hacksaw through the sidewall will work easier and still get inside. If you have a banking website, the protections need to be much greater than for a cat-picture website.

Plus I’m a real believer in 1 service per system, so the idea of having the DB running on the same machine as the website seems just wrong. I put webservers behind a reverse proxy so another layer of protection is available. Limit the attacks that the real webserver had to contend with. DB servers only allow connections from IPs that need to connect, not the entire subnet.

Every Service has a Security Best Practices Guide Somewhere.

Here’s mine for ssh/sftp – basically, don’t use passwords, don’t let root ssh anywhere, proactively block most of the internet from even making a connection attempt, watch the log files.

Look for apache, mariaDB (why do people still use mysql?!), php, perl, python, ruby, versions of these things. If you run DNS, you’ll need to secure that too.

Keep Learning – Attack Vectors Are Constantly Changing

Over the last 20+ yrs as a server admin, I’ve learned a few things. Some by reading. Some from mentors, some from mistakes.

Versioned Backups

Make daily, versioned, backups and store them elsewhere. Pull the backups, don’t push them. THAT IS THE SINGLE MOST IMPORTANT security ADVICE THAT I HAVE.

Leverage Knowledge From Others; Avoid Noob Tips

Lots and lots of people have written “Ubuntu server guides” – read those. Think through what they say. Implement what makes since and research things that don’t until you get why they are important. Next look for apache security guides, then mysql/mariaDB security guides, then find everything you can about the “P” you’ve chosen to use. IF that is php, good luck. Writing secure php is non-trivial and seems to take years of study to become proficient. Review the OWASP guides for each of these things too.

Assume your server has been hacked daily. Act accordingly. There are over a million LAMP servers currently hacked. Don’t be one of those for lack of trying.

Real Security Is Hard

Security is hard. It changes every day thanks to different attacks being invented all the time. Anything written is out of date, but at least that level of security is necessary. There isn’t a security checkbox because every system is different, every attacker is different and every web-app is different.

Nginx Stuff … incomplete

add_header X-Frame-Options SAMEORIGIN; add_header X-XSS-Protection "1; mode=block";
Find a best practices guide and look for penetration testing security guides. Blocking XSS isn’t hard anymore. Both the server and the client should be providing protection for this.

Server Installations …

The options for installing a server are many. Step-by-step example for a non-trivial setup. This example shows LVM, fsarchiver, other backups, automatic file system growth, samba, and other configs. It also shows how scripting for many tasks is possible to make the admin’s life easier.