JDPFu.com 2024

Got a new toy recently, an HDHR4-US (also known as an HDHR-Connect) from SiliconDust. These are the guy who have been making the HD-Homerun network TV tuners for years.

Scripting 101

• always fully specify paths to all programs and files (input/output) inside any script. NEVER trust the PATH.
• always set any environment variables necessary to the script. Don’t trust the userid environment to be available during cron. JAVA_HOME is an example, but there are thousands of others – most scripts need fewer than 5 environment variables.
• use the -x to see what a script does. bash -x
• use built-in “verbosity” settings for any specific command; often -v or -vvvv for more output.
• Unix uses file permissions to control if a script is "executable or not. Extensions mean ZERO. Is your script permissions set to allow the userid trying to run it to see “execute” permissions?
• check the log files – client-side AND server-side.
• break down the problem inside the script to the simplest command.
• do NOT use GUI programs inside any scripts that need to run automatically. Look for the CLI version of the tool and use that instead.
• For scripting used by the public – websites – always review the code against the OWASP checklist for that language. They have a nice Top 10 list too. Guides for C, C++, Perl, Bash, Php, Python, Ruby …. try to learn from
• For bash scripting, the ABSG covers almost everything possible. For other languages, there are many, many books, websites, webpages to help.

Recently update the System Maintenance for APT-based Linux PCs article here. Seems that some things that used to be handled automatically are NOT handled automatically anymore.

BTW, the article was published on Lifehacker a few years ago. I based it on their Maintenance for MS-Windows PC article. As you know, maintaining Linux systems ist 100x easier than maintaining MS-Windows.

For all you fellow Perl lovers, Chromatic has posted an update to the old Modern Perl book. Get the 2011-2012 version there.

I haven’t read the new version yet, but have dropped it onto my tablet for those times when stuck waiting someplace. It should be considered alpha, so expect typos at this point. Read it for the ideas, not to copy/paste code.

Perl Dancer WebApp Framework

If you would like to use a really solid, web-app creation, perl module, I can completely recommend Perl Dancer. Wrote a little server a few months ago and it has been running non-stop without issues all this time. It makes the simple things trivial and … well, I’ve only done simple things so far. ;) That entire app (including the DBIx backend) is only 350 lines of code. Modern Perl rocks.

If you are like me, you are always running low on disk storage. After all, a hard drive that isn’t nearly full isn’t living up to the full potential. About every month, I need the find the largest files and clean them up – taking me back to 75% full on the “temporary storage” HDD.

Here’s a little script to make finding the largest files on your system easy. I call it hogs.sh.

$ find /home -type f -size +2G -exec ls -lh {} \; | cut -d" " -f5,8 | sort -t" " | tee ~/hogs.txt

May I suggest:

• Drop these commands into a bash/sh file
• Call this from the root crontab weekly, perhaps 3am Mondays
• Redirect the output into a file for later viewing as needed
• Overwrite the file with every run

The script isn’t perfect. The sorting fails but does group files nicely. It also takes awhile to complete, so running it at 3am is nice. Tweak the directory and files-larger-than parameters for your location.

It seems that all the software developers are using git DVCS these days. I haven’t done serious software development in many years, so I’ve been using RCS all this time for my system admin scripting needs. With my new development work, I need to upgrade my toolset to a DVCS – Distributed Version Control System. There are many reasons to do this even if you don’t want to publish all your code on the internet. Below I’ll show how to setup an internal git server that can be shared inside a company or just between friends on the internet.

I’ll assume:

• Your git server will be on a Linux/Unix system someplace where
• all the developers will have ssh connection access.
• You have git installed on the server and the clients already.
  Those server connections may allow full shell access or be limited to support just git. Regardless, setting up ssh-keys – Ssh Config Setup – is a good idea between the client(s) and the server computers.

Keep reading to learn about Git setup.

For years scripting languages like perl and add-ons like perl modules were supposed to be installed as part of the OS. This worked when the server administrator was the primary programmer on the system who could control all that stuff. In these days of role-based administration, the users and the administrators are hardly ever the same person. Administrators have different needs for systems than users or programmers and often those needs conflict.

How do we solve this problem for Perl scripts? Perlbrew.

Updated 10/2019

Use ed25519 keys, if you can:

ssh-keygen -t ed25519
ssh-copy-id -i ~/.ssh/id_ed25519.pub userid@remote

Updated 10/2015

If you have an ssh server running on your network that is accessible to the outside world, on the internet, chances are your systems are being attacked. If you aren’t aware of this, just take a look at your ssh logs in /var/log/auth.

$ egrep -i Failed /var/log/auth.log*

This article is for Linux/UNIX users, but the ideas should apply to any OS running an ssh daemon.

Ssh Setup For Higher Security

The order below based on how easy it is to accomplish or setup. None of these configuration changes are hard. All of them can be accomplished in under 5 minutes if you know what you’re doing or 15 minutes if you need to read up a little.

• Listen on a non-standard port
• Use ssh-key-based connections
• No remote root logins with a password – without-password
• Allow only key-based logins from non-LAN IPs (basically any remote ssh connection cannot use a password)
• Lock account after X failed attempts – Fail2Ban
• Automatically block IPs with login failures – Fail2Ban
• Monitor hack attempts – Fail2Ban

By some standards, my 10,000+ digital photo collection is either very large or trivial. I suspect that professional photographers probably have hundreds of thousands of photos. Many of those will have different post-processing.

Organizing, backing up and archiving digital photos and images doesn’t have to be complicated to do well.

Organization

As you take the photos, place them into your organization. If you delay more than a week after returning from that once-in-a-lifetime vacation to Africa, then you probably will never perform any useful organization. Below are 8 steps to help you organize your photos efficiently.

Last night I did a little presentation on Using VirtualBox on a Desktop. I’d give myself a D+ for a grade on the presentation. Fortunately, it was a small and highly interactive crowd. I tried to cover too much stuff. Also, I showed how to do this on a Windows host OS with a Linux client OS to a Linux-specific crowd. Initially, I’d planned to show an install on a Linux host OS too. The physical machine had a really slow disk controller, so I wasn’t able to create a virtual disk to install the OS into. I tried it a few weeks ago on the test machine and it took 45 minutes to create a 10GB .VDI file. On my home machines, doing this is just a few minutes.

The good thing was that I covered some of the key performance choices in virtualization – multiple times. The good news is that the newer VirtualBox releases choose most of these settings automatically. I should probably create a blog entry for each of the different client OSes that covers performance choices. Anyway …