JDPFu.com 2013

This Saturday (tomorrow) I’ll be presenting to the DC404 crowd how to get system monitoring working on Linux in 5 minutes. Sadly, the presentation is running about 30 minutes because I really only need 5 minutes to show a complete install. Come and check it out, say hello.

There isn’t really much to it because SysUsage is really easy to setup and run. It works on desktops, laptops and servers, but there is no GUI for setup so you will need to use a terminal or console. With newer releases (OS or SysUsage) the dependencies may change, but it is still really easy to setup. This is so simple there really isn’t any excuse NOT to have performance graphs for all your Linux machines.

These days, there is a real desire to have videos on your network, but not loose any of the features that the source media provides. Things like multiple languages tracks, director’s comments and other interesting audio tracks. I like to listen to the Spanish soundtrack and have the Spanish subtitles displayed for movies that I already know. It has been possible to have all this by ripping the full DVD contents and using an appropriate playback device for a long time, but that uses a bunch of storage – perhaps 4x more than needed if modern video codecs are used.

If you’re like me, you take notes all the time. You may have tried using paper, todo lists, spreadsheets and even personal wikis like TiddlyWiki or SoloWiki to accomplish this. Having the notes available at home, at work and when you aren’t connected is a requirement. Well, I accidentally came across a new browser/javascript tool today called Memonaut

I’ve just played with Memonaut a few minutes. It does bulleted lists, indented lists, numbered lists and generic notes all with HTML and JavaScript. There’s some lite formating too – bold, italics, strike-through, etc. The setup is like TiddlyWiki – you open a specific HTML file and start adding content. Then you “Save” the file to store the updates. Certainly there will be performance/size issues when lots of content is added, but then you just archive the file and start a new one, perhaps monthly? TiddlyWiki was used at a previous employer to share How-To Team information after we were merged into an other group. That data sharing probably saved lots of jobs because about 30 people who were trying to understand how things worked AND who to call for “x” were able to centrally store and access that data.

Did I mention that this works when you’re offline?

Anyway, Memonaut could be worth a look. Obviously this is cross platform AND it requires JavaScript to be enabled in your browser for local HTML files (which could be a security concern).

Enjoy.

I watch Hak5 on my S2 TiVo. This season, DK is doing everything on Linux. He’s covered a fairly powerful home router already with smoothwall. The home-made commercials are usually the best parts of the show m(seriously!), but episode 804 covers the easy and slightly harder way to record your Linux desktop for screen casts.

gtk record my desktop for you GUI-liking people and ffmpeg for the shell folks like me that prefer a little more control are demonstrated.

The guys over a CommandLineFu also covered the ffmpeg way of doing it with a little more finesse. Don’t you just love that domain name?!!!

ffmpeg -f alsa -itsoffset 00:00:02.000 -ac 2 -i hw:0,0 -f x11grab -s $(xwininfo -root | grep 'geometry' | awk '{print $2;}') -r 10 -i :0.0 -sameq -f mp4 -s wvga -y intro.mp4

Hak5 isn’t the first to cover this topic. A quick search finds thousands of others demonstrating how to do this as well, but a few of the Hak5 commercials are GREAT!

For the last few years, I’ve heard powershell for scripting by IT professionals in the MS-Windows community. Never looked at it until a few days ago. See, I confused cscript with PowerShell scripting. I couldn’t get the hang of cscript and wasn’t able to get it to accomplish what I needed the last time I tried. In June, I actually wrote some .BAT scripts for a client. If I knew then what I know now about PowerShell, I’d have been much happier and written those scripts in PowerShell, assuming it is pre-installed on Win2003 Server.

In my limited time with PowerShell, here are my impressions.

• UNIX has text processing, MS-Windows uses object processing – sorta like the difference between PERL and Ruby scripting.
• Most of the syntax is Perl-like – with some differences. Get used to $var.action calls.
• Pipes work like you’d expect from UNIX, except ….
  • generally, we don’t use `grep`, rather, the regex is used as part of the prior command
  • | `wc -l` becomes action.count – basically, if you want to get a count of something.
  • functions are Perl/Bash-like, except the argument passing is C-like (int x, int y). That is better than Perl, IMHO.
• Objects mean access to the registry, COM, and WMI. That’s a plus AND a minus. How to query a GUI program? That’s the problem.
• As usual, rather than reuse existing commands, Microsoft decided to use, longer, more complex commands. `ls` is `get-selections`, but they did include aliases in the tool. That means that many of the normal UNIX commands used in shell scripts have aliases, but those are not the preferred method name to be used.

As long as you don’t need to press a button inside a GUI, PowerShell can probably be useful.

Mindset Change

To me, the main limitations of PowerShell really aren’t with the language, rather, it is with my mindset and having to change from UNIX/File processing to object processing. Also, the tools on UNIX/Linux systems tend to support command line options to get things done. In MS-Windows, command line options to GUI programs are usually extremely limited – next to worthless. In Linux/UNIX, there are hundreds and hundreds of small applications and tools that come with the system suitable for script use.

References

A few years ago, PowerShell was called msh, Microsoft Command Shell. Here’s an overview.

Here’s a PowerShell Tutorial if you are interested in going further.

Read where someone is porting PowerShell to UNIX/Linux systems. Yep, it is called Pash. According to the project page, it is 40-50% complete and running on Windows, Linux, Mac and Windows Mobile platforms.

VMware is using PowerShell (they call it PowerCLI) as an interface into their ESX virtualization system.

Summary

In summary, it seems that Microsoft has been working hard for years to remove some of the biggest complaints against their systems – the lack of real, usable scripting. Seems they could advertise it a little more widely. Personally, I’d prefer a Perl library that encapsulated all this and for Perl to be included with MS-Windows installations so I don’t need to learn yet another language. Still, msh/psh/pash or whatever they call it this year is a step in the right direction.

Below are a few incidents that I’m personally aware of which impacted a few different projects. Some are from my personal desktop to production dispatching systems with 20K+ users to some that impacted a space shuttle launch data.

People like Top 10 Lists, but I could think of only 9 near disasters. Perhaps something interesting will happen this week? ;)

Ooops – beep, beep, beep ….

Today a friend sent an email with a Gnome helper app to setup a panel so remote ssh logins could be added to the Gnome Panel. There are lots of applications, or applets, like this out there. They all read and edit config files and provide a GUI to do something that has been possible for years and years. I guess if you are new, then having a program that edits configuration files before you’ve learned to use a UNIX editor is a good thing. Noob-friendly editing is good and reduces the perceived learning curve for Linux. Long-time users know that having a program to edit simple configuration files isn’t needed. You can edit them yourself and accomplish amazing things.

Some background reading on X/Windows. Here’s an architecture image as a reminder:

Recall that the X-Server runs on the desktop and that the X-client runs on the other, sometimes remote, machine sending requests for to the specific screen to be displayed. Also, you can run many X-servers on a single machine, even if they are not physically displayed.

Nothing is New

Do a google search on “fvwm screenshots” to see what I mean. This one or one very similar has been displayed as long as I can recall using fvwm. fvwm has been around since before I started using UNIX/Linux in 1993. I didn’t find it until 1995 when it was a pioneer in virtual desktop capable window managers. At that time, people were still using wmw and twm, yes, people actually used twm. When you first started using FVWM, you wanted to configure the menu for your local needs. It was easy to get going quick and setup remote logins to other systems for everything, including telnet, ssh, email, web browsing, editing files, running desktop word processors … whatever. Today the GUI settings are still maintained inside text files and these can be customized manually. Sometimes there are a few more steps since GUI programmers today like to take a simple concept and turn it into an environment that requires many, many more config files. Still, manually adding menu items to a panel for Gnome, KDE, XFCE, or LXDE is relatively easy.

The Matroska container format for video has become popular over the last few years because it merges a number of good ideas and let’s a single file contain multiple video, audio and subtitle streams. No longer do you have to keep multiple .vob/.mpg/.avi.mp4, and multiple .sub/.srt/.idx files to have 3 different audio and subtitles. All of them can be placed into a single .mkv file. For me having the subtitles efficiently contained inside the same file is good, but the real, fantastic reason that MKV containers are brilliant is you can easily correct aspect ratio issues without re-encoding.

Here’s another article for why you should also use MKV if you’d like more specifics.

There’s another nice bonus. In my testing, the .mkv files are always smaller than the .avi files from which they are made.

Jumping into the How-To

Blog Spam

I’ve noticed the number of blog spammers have increased significantly in the last 2 months. They use general “good job” or “nice work” comments, then leave their email and commercial weblink. I assume these are spam-bots – automatically doing it.

Moderated Comments

Since all comments are moderated here, I’ll do my best to weed them out if they aren’t related to the posted article. Only on-topic links will remain and generic posts will not be allowed. It isn’t like there are hundreds of spam posts daily. I know this will reduce the number of comments, but that is the price for non-spam comments today. Sorry. If you’re comment is on topic, it will be posted. Basically, any comment that is remotely on topic will be posted. Just those that are commercial or links to unrelated content will not be posted. For example, if the post is about virtualization and you provide a comment with links to an online vitamin store, that will not be posted. OTOH, if links in comments are to other articles on virtualization or even commercial virtualization products, then it will be allowed. The decision of moderators leans towards posting comments when in doubt.

Test Messages

I guess some people don’t want to bother writing a longer message if it won’t be posted. I get that, but a test message is not on topic either and won’t be posted. How does that comment add to the conversation?

Further, I’ve disabled comments for older articles. I don’t recall the actual cutoff day. It is probably 90 or 120 days, so it won’t impact the few, loyal, readers. Those articles do not have any way to enter any comments. If there is a comment field displayed, then your comment will be seen by the moderators.

Hello, Nice Article and other non-related comments were allowed previously, but are not going forward. Sorry. Those do not add to the conversation.

English Only Please

This is an English language blog. While we like worldwide viewers and understand that not everyone reads English, that is simply a limitation of our skills. I have translated some non-English comments previously. None were on-topic to the post. We may attempt to translate comments again, but you can visit translate.google.com just as easily as we can.

No Sign-up Required

We do not require any sign up to post comments. Heck, we don’t really want your email address either. An alias is preferred. If you leave an email address or web address, it will probably be included in the comment and publicly seen. That seems to be the way this software works. Our systems do log IP addresses, just like every other system out there does.

Example Blocks

A few of these spammers have been blocked at the router. Sure they can come from a different subnet, but I bet they won’t.

The financial planning and foreign internet diamond sellers are the funniest. Blocked.

Automatic Moderation

I’ve looked into viable solutions to allow non-moderated comments here and didn’t find one that I was willing to implement.
Here’s a site from 2005 with specific ideas to reduce, if not eliminate internet marketing on blogs. About a year ago, I came across another site where the blogger had placed a static Captcha with an simple arithmetic problem inside the image. The answer was always “42.” He never changed it, Never, yet it prevented 100% of the blog spam. I may introduce that here.

If I were running MT or blogger or some other highly popular blog tool, then I’d have a bigger issue. Since I’m running a little used Ruby blog with few internet users, I’m fairly safe just like Linux and Apple are safe compared to Microsoft.

Today, we are manually moderating comments about once a day.

Comment Edits

Occasionally, comments may be edited by a moderator to remove offensive content. We will say in the post that it was edited. Cuss words will probably be removed or exchanged for #$#%. Keep it clean, please.

Exceptions

We are people and regardless of the statements above, there will be exceptions for posting and not posting. Friends who post can say almost anything.

Here’s a little program that automates Video Redo Plus Quick Stream Fix, QSF, processing.