JDPFu.com 2025

Everything is fine. No news. Both of them are just working.

VirtualBox 2.0.6 was released this week. I haven’t updated.

Xen on Ubuntu 8 LTS is working better and better. Performance is fine. Networking is fine. I have 6 VMs running on a single box. Most are 64-bit images. The list:

1. Zimbra
2. Alfresco, Apache, mediaWiki, dotProject, samba, mySQL
3. Typo (32-bit)
4. SugarCRM
5. PKI
6. Monitoring

Automatic full backups happen nightly and get mirrored to 2 other disk subsystems. The total size of the TGZ files are under 9GB, so network traffic is fairly trivial on our GB network.

Most of the full VM backups take 6-8 minutes. Zimbra takes about 25 minutes. That’s a full VM shutdown, copy to another disk, VM startup. rsync of the VM files should get this to just a few seconds for each. That’s the next phase to improve our backup system.

As far as VirtualBox goes. I boot WinXP once a week to get patches and run a few WinXP only apps. The rest of the time, it runs Ubuntu 8 desktop – very happily. Video transcoding, rsync, testing new software before deploying into the Xen infra, open office, surfing, rss, email … all the normal stuff we use computers for.

Anyway, everything is fine. Nothing to see here. Move along.

So, I’ve had a TiVo series 2 with lifetime support since 2003-ish. When wireless support became useful – TTG – I bought a supported D-Link USB 122 adapter. I’ve been using that adapter until today. It still works, but doesn’t support WPA and is only 802.11b – WEP. It was kinda slow too. 350kbps – uh, on a wireless network that should have a REAL speed of about 2500kbps.

Anyway, I broke down and ordered the TiVo branded G device a few weeks ago. It arrived today. I didn’t believe the instructions.

1. Plug it into the USB port.

That’s it.

Damn, if it didn’t work. No reboot. No reconfigure of wifi settings.
Most impressive – my download speed has doubled to 650kbps. Oh, and that’s with a connect to TiVo test going.

The other scary thing is that my signal has gone from 85%-good to 100%-excellent. All I did was unplug the old USB wifi adapter and plug in the new. That’s it.

Happy. If I knew how easy this was, I would have changed years ago.
Later, after my current download finishes, I’ll change my wifi network from WEP to WPA. My network-mooching-neighbors are gonna hate me.

VirtualBox has a way to share host file directories with client VMs. Under Windows, it is fairly intuitive. Under Linux client OSs, the steps are less than clear. I don’t know whether they are documented in the users guide or not. I don’t recall it however.

1. Load the Client OS Extensions.
2. Verify that `lsmod | grep vbox` shows the vboxfs module.
3. Configure the extension to export a directory – I told it to share c:\Data from my Vista-64 host as “Data”
4. sudo mount -t vboxsf Data /Data – yes, the device is just the plain “Data” and it doesn’t show up in /dev like it should.

Run that last command and BAM, it mounted. I updated the /etc/fstab with the necessary info, dismounted the drive and ran `mount -a` to test it. Yippy!

I have access to 150GB+ of data storage on my laptop that is shared with the host and all guest machines. Sweet! This Virtual Box thing is good, but you already know that I’m easily amused.

I’m currently running VirtualBox 2.0.4.

I read yesterday that AVG on Vista had been causing network issues for some people. I’ve had some of those issues with my WiFi connectivity – disconnects as often as every 15 minutes. Usually, it stays connected about 45 minutes then disconnects unless I force the connection to stay active by moving files around my network. Sadly, moving large files started failing about and hour into the move – disconnection.

Last night, I de-installed AVG from my Vista-64 host. At the end, a reboot was required, but the machine refused to come back up, BSOD, twice. Safe mode boot did work so I forced a checkdisk. Over 2 hours later, the system rebooted and seems to be working. I haven’t been wifi disconnected at all since then. Everything seems just a tiny bit faster. I’m not really worried about viruses on that box since it is used to run virtual machines, VirtualBox. I’ve been running an Ubuntu VM for about 3 hours now. No issues.

Update: About an hour later, the network got slow and I lost my ssh connection to another box on my network. This issue never existed with WinXP. It appears that AVG may not be the cause … just Vista-64 Home Premium remains. That OS sucks.

Show your CEO the cost savings between Microsoft solutions (MS-Office, Exchange, AD, CALs) as you propose replacing them with Open Source Software, OSS, that include commercial support.

• MS-Exchange —> Zimbra
• MS-Exchange Calendaring —> Zimbra
• MS-Active Directory —> OpenLDAP
• MS-Outlook —> whatever client you like that’s free; Try Thunderbird or whatever Apple gives away or Enlightenment
• MS-Sharepoint —> Alfresco
• Shared Drives —> Samba and/or Alfresco
• MS-Print Servers —> Samba/CUPS with printer driver support
• MS-Office —> OpenOffice

Add up all that you’re paying for this stuff. Spend 2 hours researching what commercial support for the FOSS stuff costs – estimate $15k for each except those items that just work like OO, Samba and OpenLDAP.

MS Costs	FOSS Costs
$4.5M	$60k

Enough said? No? Ok, use Xen to host all these things. Your disk costs for EMC/NetApp/Whatever won’t change. Your clustering costs will require consultants … 1 time to setup until your team becomes knowledgeable.

Hardware? Use this as the call to virtualization that your organization needs already. When you’re all done, you’ll have half as many servers as before. You’ll probably save many, many more in actuality.

Training? When the CEO shares the overall cost savings with his team and it gets out to the every day workers, they may still complain a little, but they were complaining about MS stuff before.

Outcomes:

• huge cost savings for very minor changes to capability – you’ll need to judge that for yourself
• 2x hardware (or more) reduction
• ongoing software license costs cut by over 50% (just the MS stuff here – this doesn’t save on your CRM, SAS, Oracle costs)
• No Vista upgrade looming
• No desktop CAL and no desktop MS-Office licenses; well, you’ll probably have to retain a few. Or perhaps the CEO will mandate 1 Exchange server for him and his team, but everyone else needs to migrate to Zimbra.

Further, if you can replace WinXP with Linux (not everyone can), you’ll see

• No more “hardware is too slow” upgrades
• “Hardware broken” replacement cycle every 5-7 years
• Employees will use work computers for work, since the same programs they use at home won’t run without tinkering at work.
• There are many, many excellent FOSS programs that businesses use today. MS-IIS —> Apache; PBX —> Asterisk; many, many, many, many, many, many, many, many, others.
• WINE will run many, many Win32 programs.

All of these replacements have support contracts available, so there is strong commercial support, should you need it. OTOH, if you have a small team of experts, you can probably get the FOSS versions up and running in less than a week and never need to pay anyone for support outside your current staff. This is idea for proof of concepts and other trials. If you are an enterprise looking for support, that’s fine – compare $15k-$30k per year to what ever you’re currently paying MS. I worked at a place where we paid MS over $5M/yr in support and licenses just for desktops (OS, MS-Office, CALs, AD).

Best of all, all those replacements don’t require MS-Windows. They work with whatever clients you like … well, that isn’t entirely true. Zimbra FOSS doesn’t do any calendaring with Outlook 2003 and perhaps 2007. Is that a bad thing?

Lastly, use this swap as a way to improve the way your company does things. Don’t just replace shared folders with samba – use a document management system like Alfresco and migrate users into it. To start, they can use it just like shared folders. Over time, you’ll get better as it and implement triggers for when documents arrive or leave the system. For example, you can have every document loaded into directory XYZ automatically converted into HTML and dropped into another folder. Or implement document review work flows. Start small but grow into the full capabilities.

The key thing with all these solutions is that the data is yours. It isn’t locked up in some proprietary format.

• Don’t like Alfresco? Connect to the repository with CIFS and copy all the documents out and load them into Documentum or back to shared drives.
• Don’t like Zimbra? The message store is just IMAP. Calendars can be exported with ICS. Contacts can be exported too. I haven’t found where the distribution lists and aliases are stored, but the DBs are OpenLDAP and MySQL. How tough will it be?

Whatever … it’s your data.

I feel bad for those who can’t throw MS completely out for whatever reason. For example, many of the telephony, scanning and fax servers only run on Win32 servers and only connect with Exchange.

Blackberry users – take heart – Zimbra has a Blackberry interface. Many other smartphones also have interfaces and there is a generic J2ME interface for all the rest.

How to install Zimbra on Ubuntu 10.04 LTS

Another gmail outage
this week …. this is exactly why you shouldn’t trust a service provider with this sort of stuff without an SLA.

• Sure, it’s free.
• Sure, it’s in beta … and has been for 4 years.
• Sure, nobody cares that you AND your users can’t get your email.

Let’s not forget that google stores and searches all your email and builds a profile. I saw an estimate that had google making $245 per profile per year on targeted ads due to that extra information. You may not be paying for google apps, but you are paying for them with your privacy.

Java programmers have had a bad rap for years. Your programs are slow and use too much memory has been the cry from compiled binary language programmers.

It is less true today. A good JIT compiler will be within 10% of the speed of C++ for an equally skilled programmer. That doesn’t say that an excellent Java programmer can’t make better code than a poor C++ programmer, they can. It just isn’t as likely for programmers of similar skill in their selected language.

Most of the time, code speed isn’t really that important. 80% of programs are used by less than 10k users, so the performance will be key only where I/O is concerned, not in calculations or GUI code. Calling a function 10k times when you only need to call it once is bad in any language – that’s understood.

In summary, Java programs are less efficient than C++ which is less efficient than C which is less efficient than ASM – provided that all are efficiently written.

Speed will be greater and memory use will be less as you move towards ASM. This is obvious.

OTOH, the speed of development, complexity management and platform flexibility that Java has is probably greater than the other choices and (in your environment) may outweigh the speed and memory differences.

There’s an old programming rule … don’t bother optimizing code that only runs once. Spend 80% of your time optimizing only the code that needs optimization.

Personally, I like cross-platform C++ code as the best of efficiency, memory footprint and code portability when a complex program is being created. But more and more, Ruby, Python, PHP or other scripting languages are faster to create than lovingly crafted C++ or Java and get the job done. That’s what programming is all about – getting the job done.

Many of us have lots of music on our home network. I spent months converting my audio CD collection into MP3 and organizing the files.

Organization

At the time, I didn’t trust ID3 tags inside MP3 files.
My organization was/is directory based. /{genre}/{artist}/{album}/{track_no-title}.mp3

It was a lot of work to keep that since “genre” is often subjective. Is it lite rock or easy listening? OTOH, almost every player supports “random” and you’ll get only artists that you like since you’ll probably only convert music you like.

Streaming

I installed gnump3 on my Linux server. This is an-all-in one perl-based media streaming server. No external web server used – it has a built in streaming server. It supports simple authentication and blocking/allowing access by IP address. Of course, when you travel, allowing any IP access, but retaining the login is a good idea. Here’s 1 screen, there are many different skins available

This is really old news. Almost everyone who wants a streaming media server probably already has one setup. Why bother with this? Well, if you have a Nokia N800 (or similar), then perhaps you’d like to stream audio anywhere in your home. The browser-based interface is simple and will pull a M3U file that the N800 Media Player app works with. Other computers and media devices in your home can also listen to this music. Imaging, Sunday morning, when you’re reading the newspaper and listening to Bach. Nice.

Backups!

Obviously, if you spend the time to convert a large audio collection into digital formats, you’d like to never do it again. Definitely back up those files. I took an old IDE disk drive and copied all those media files over to it. Then that disk is placed on a shelf. This is the best backup method – better than burning DVDs, CDROMs, or even tape. Hard drives effectively have no end of life when they aren’t used. Realistically, the EOL happens when you no longer have a PC that can connect to the drive. As a simple example, can you read a 5.25" floppy drive today? Anyway, when you spend as much time and convert all your own music personally, you will want a backup, period.

I’ve been running Xen and VirtualBox VMs here. We’ve been experiencing VM performance issues on 2 VMs under Xen on a fairly powerful box. Basically, the DomUs get really slow – slow.

Here’s what I know.

1. while connected via ssh to a DomU, sometimes my shell becomes non-responsive for 30+ seconds.
2. A php network monitor too shows the listeners aren’t answering in a timely fashion

Suspects:

1. sparse disk files for the images this is the issue – COMFIRMED
   1. I used sparse files for the OS images. This is the default instead of preallocated or LVM based disks. I need to migrate from sparse to full ASAP! `hdparm -t` shows Dom0 with 60-80 MBps throughput. On DomU it is 55-65MBps which should be good enough.
2. virtual network issues
   1. In older network bridges under Xen, there were real problems. The IO reported by others in tests were 3-4 Mbps over GigE networks. By using a virtio driver instead, they got 750+Mbps throughput. That’s what I’m talkin’ bout!
   2. There could be issues with my cheap switch and cheap router too. IP sessions may not expire quickly enough.
   3. I didn’t change any of the default network settings from the xen install. Perhaps this was a bad idea?
3. poor kernel settings for Dom0 and DomUs
   1. I’ve check the scheduler for issues – the defaults in Hardy and Xen 3.2 seem to follow the suggestions I’ve found thru google.
4. Similarly mis-configured DomU OS installations
   1. Well, it is believable that I made some other bad choices related to OS installation. I don’t know what they could be.

I have some work to do. Check back here for the results.

Results

1. I happened to be watching disk I/O and CPU when another lockup/slowdown happened. loop0 was the top process at the time. That means – disk is the issue. Nice.
2. Ok, after huge amounts of research, I was finally able to mount the sparse and full disk file images. Then I attempted to dump/restore the old system onto the new files (full/non-sparse). Done. Then I took a huge step and started the VM up – the new VM. Everything worked. FAST and with only 128M of RAM instead of the normal 512M that the old image had. It really is amazing … so far.
3. So assuming that actually worked perfectly – 10 minutes of use only – now it is time to do the same thing for my Zimbra installation. I am feeling lucky, punk. Get ‘er done. I’ll report back here in an hour or so.
4. So I swapped the Zimbra installation from a sparse file to a FULL file system. I’ll watch the monitoring software to see whether the 30+ reported outages per day keep happening. My blog installation definitely is showing higher responsiveness, definitely. I haven’t seen the slowdowns on a console, but I haven’t sat behind the server for hours yet either. We shall see.

See google is evil …. or just doesn’t want any hassles over what they index going forward.

Basically, google wants to not be sued over any content they come across – however they come across it.
Like lambs to the slaughter, we are.

I’ve pointed out how google wants to know their customers previously