JDPFu.com 2025

rdiff-backup rocks, mostly. But there are times when it doesn’t work as expected or doesn’t work at all. Usually, the not working at all part is a cockpit error, but sometimes not.

Key rdiff-backup features

1. Simple 1 line backup command; rdiff-backup source target
2. Reverse Incremental backup sets
3. Extremely FAST backups. Entire server installations are just a few minutes, after the initial backup set is created.
4. Last backup set is available as a complete copy of the files. Need to recover? Just copy the file(s) back.
5. Control over how old backup sets can be. Deletion of “older than x days” sets is trivial.
6. Compressed older differential backup sets
7. Current backup is 1-for-1 sized. Older backups are tiny. As an example, a 5GB backup with hundreds of files with 30 days of incremental backups only takes 6GB total. Each daily backup is relatively tiny and based on changes made that day. Usually those changes are just a 10-40MB. Impressive.
8. Recovery by date/time
9. FOSS – we like Free and Open Source Software
10. Cross platform – Unix, Linux, MacOS and MS-Windows.

Things that just work

1. Linux local rdiff-backup runs, just work. Backing up a directory structure or an entire VM (not as a single huge file) to another mounted disk works very nicely with all the key features listed above.
2. Win32 local rdiff-backup runs, provided there isn’t any networking involved nor huge files.
3. Recovery of an entire VM fileset. I’ve needed a few recoveries the last 6 months due to user error. They worked flawlessly and only took 20 minutes from problem discovery to full recovery. That was manual recovery. If this were scripted, it could be less than 5 minutes.

Things that don’t work or work poorly

This is mostly on MS-Windows platforms, but some Linux stuff doesn’t work nicely either. Windows howto that wasn’t really much use.

1. remote transmission over ssh on a non-standard port is broken regardless of platform. That doesn’t mean it can’t work, but I’ve never been successful in getting it to work. Neither push, nor pull command versions have worked. It shouldn’t be this hard.
2. Large file differencing doesn’t seem to work on Linux or Windows. In theory, that means 4GB files, but smaller files get confused and end up as a complete copy too, not a block level differential copy.
3. MS-Windows network backups don’t really work, even over samba connections. Ok, there are many strange things about rdiff-backup on Windows. For example:
   1. You have to `cd` to the drive and directory of the source if you want it to work.
   2. You have to use ‘/’ instead of ‘\’ characters, most of the time. This is a python thing, I guess.
   3. Backups to samba shares may or may not work. I haven’t figured the reason why or why not yet.
   4. Backups over ssh require less than trivial setup. Only push will work from windows unless an ssh server is setup. Then the complexity is exponentially more difficult.
   5. Many people use cygwin with all those faults (slow, heavy, bad directory access) to get around the win32/64 API issues.

So, rdiff-backup is great for local Linux system backups, but for remote backups, you’ll want to use different technology, like rsync. If you’re on Windows and want remote backups, check out some other solutions.

Good writeup on rdiff-backup features, method, and algorithm.

S1/Disk1 -rdff-backup→ S1/Disk2 -rsync/ZFS send→ R2/Disk1

If you have a lead or solution for my woes, please let me know! I often miss trivial solutions.

Here’s an actual rdiff-backup set to clarify:

        Time                       Size        Cumulative size

-—————————————————————————————————————

Fri Jul 10 01:32:13 2009         4.37 GB           4.37 GB   (current mirror)

Thu Jul  9 01:32:13 2009         36.3 MB           4.41 GB

Wed Jul  8 01:32:13 2009         37.5 MB           4.45 GB

Tue Jul  7 01:32:14 2009         31.4 MB           4.48 GB

Mon Jul  6 01:32:13 2009         31.1 MB           4.51 GB

Sun Jul  5 01:32:13 2009         27.0 MB           4.53 GB

Sat Jul  4 01:32:14 2009         41.3 MB           4.57 GB

Fri Jul  3 01:32:12 2009         33.9 MB           4.61 GB

Thu Jul  2 01:32:13 2009         37.9 MB           4.64 GB

Wed Jul  1 01:32:14 2009         35.4 MB           4.68 GB

Tue Jun 30 01:32:13 2009         37.3 MB           4.71 GB

Mon Jun 29 01:32:14 2009         38.9 MB           4.75 GB

Sun Jun 28 01:32:13 2009         38.7 MB           4.79 GB

Sat Jun 27 01:32:15 2009         42.0 MB           4.83 GB

Fri Jun 26 01:32:13 2009         49.3 MB           4.88 GB

Thu Jun 25 01:32:13 2009         37.3 MB           4.92 GB

Wed Jun 24 01:32:14 2009         36.4 MB           4.95 GB

Tue Jun 23 01:32:13 2009         43.0 MB           4.99 GB

Mon Jun 22 01:32:15 2009         33.4 MB           5.03 GB

Sun Jun 21 01:32:15 2009         31.0 MB           5.06 GB

Sat Jun 20 01:32:13 2009         41.6 MB           5.10 GB

Fri Jun 19 01:32:14 2009         31.7 MB           5.13 GB

Thu Jun 18 01:32:14 2009         32.0 MB           5.16 GB

Wed Jun 17 01:32:15 2009         31.0 MB           5.19 GB

Tue Jun 16 01:32:17 2009         31.6 MB           5.22 GB

Mon Jun 15 01:32:16 2009         31.7 MB           5.25 GB

Sun Jun 14 01:32:14 2009         31.3 MB           5.28 GB

Sat Jun 13 01:32:14 2009         30.7 MB           5.31 GB

Fri Jun 12 01:32:14 2009         31.3 MB           5.34 GB

Thu Jun 11 01:32:15 2009         32.3 MB           5.37 GB

I’ve been running Vista on a laptop since Sept. Generally, I don’t really use Vista for anything other than a platform to run virtual machines. What I need from Vista is

1. a stable host platform
2. Disk access and storage for VMs
3. Network access and bandwidth for VMs
4. Video access for VMs
5. Accurate Time Service since VMs get time from the host OS. There’s no way to disconnect time from the host provider.

Most new IT efforts have a Green Component to them. At a minimum, the amount of power used is to be minimized for any new equipment. Other projects involve consolidating servers with a 4:1 or higher reduction. That reduction, when combined with fewer total number of CPUs, hence fewer licenses, more efficient storage architectures and significantly more efficient servers from a power consumption perspective really lowers the power usage overall.

As an example, a recent server review showed 14 servers. Our efforts so far had added 2 new highly efficient servers and removed 4 older servers. The two new servers deployed VMware and were using approximately 20% of the CPU and 50% of the RAM (generous RAM allocations to start) after migrating the 4 physical servers into virtual servers. We are targeting a peak CPU utilization of 60%-80%, so a few more virtual servers may be added to each system. That will reduce the total server count from 14 to 8 (14-4+2-4) with more manageable systems overall, easier disaster recovery, and significantly lower power consumption – perhaps 2/3rds less power.

One of the older, already retired systems used over 500VA of power just for server idle with nominal processing. The new servers at idle use approximately 80VA each. On average, the other servers used about 300VA of power, 3x more than the new servers.

The savings aren’t just in direct power use reductions. The UPS system has costs too. As the required power load is reduced, it is important to resize down the UPS system so battery charge maintenance and AC—> DC conversion loses are minimized. Required cooling is also reduced, which is important in warmer clients.

At my home, I’ve used a Kill-A-Watt to measure power use of my systems and a TV.

New Core2Duo system with 4GB RAM – Peak 126W, Nominal 80W, off 8W

3 Yr Old Core2Duo system with 2GB RAM – Peak 173W, Nominal 126W, off 0W

Toshiba 37" LCD – Peak 147W, Nominal 104W, off 0W

Clearly newer systems are more efficient than older systems. Similar performing nVidia graphics cards are in each server, both with 2 internal disks as well. A RAID card is installed in the older system, which could be the main power impact. Motherboards are different between each box, but other than age, believed to be comparable. Also, the older box has a high efficiency power supply – over 90% efficient with the newer server using a normal case power supply.

Everything is fine. No news. Both of them are just working.

VirtualBox 2.0.6 was released this week. I haven’t updated.

Xen on Ubuntu 8 LTS is working better and better. Performance is fine. Networking is fine. I have 6 VMs running on a single box. Most are 64-bit images. The list:

1. Zimbra
2. Alfresco, Apache, mediaWiki, dotProject, samba, mySQL
3. Typo (32-bit)
4. SugarCRM
5. PKI
6. Monitoring

Automatic full backups happen nightly and get mirrored to 2 other disk subsystems. The total size of the TGZ files are under 9GB, so network traffic is fairly trivial on our GB network.

Most of the full VM backups take 6-8 minutes. Zimbra takes about 25 minutes. That’s a full VM shutdown, copy to another disk, VM startup. rsync of the VM files should get this to just a few seconds for each. That’s the next phase to improve our backup system.

As far as VirtualBox goes. I boot WinXP once a week to get patches and run a few WinXP only apps. The rest of the time, it runs Ubuntu 8 desktop – very happily. Video transcoding, rsync, testing new software before deploying into the Xen infra, open office, surfing, rss, email … all the normal stuff we use computers for.

Anyway, everything is fine. Nothing to see here. Move along.

VirtualBox has a way to share host file directories with client VMs. Under Windows, it is fairly intuitive. Under Linux client OSs, the steps are less than clear. I don’t know whether they are documented in the users guide or not. I don’t recall it however.

1. Load the Client OS Extensions.
2. Verify that `lsmod | grep vbox` shows the vboxfs module.
3. Configure the extension to export a directory – I told it to share c:\Data from my Vista-64 host as “Data”
4. sudo mount -t vboxsf Data /Data – yes, the device is just the plain “Data” and it doesn’t show up in /dev like it should.

Run that last command and BAM, it mounted. I updated the /etc/fstab with the necessary info, dismounted the drive and ran `mount -a` to test it. Yippy!

I have access to 150GB+ of data storage on my laptop that is shared with the host and all guest machines. Sweet! This Virtual Box thing is good, but you already know that I’m easily amused.

I’m currently running VirtualBox 2.0.4.

I read yesterday that AVG on Vista had been causing network issues for some people. I’ve had some of those issues with my WiFi connectivity – disconnects as often as every 15 minutes. Usually, it stays connected about 45 minutes then disconnects unless I force the connection to stay active by moving files around my network. Sadly, moving large files started failing about and hour into the move – disconnection.

Last night, I de-installed AVG from my Vista-64 host. At the end, a reboot was required, but the machine refused to come back up, BSOD, twice. Safe mode boot did work so I forced a checkdisk. Over 2 hours later, the system rebooted and seems to be working. I haven’t been wifi disconnected at all since then. Everything seems just a tiny bit faster. I’m not really worried about viruses on that box since it is used to run virtual machines, VirtualBox. I’ve been running an Ubuntu VM for about 3 hours now. No issues.

Update: About an hour later, the network got slow and I lost my ssh connection to another box on my network. This issue never existed with WinXP. It appears that AVG may not be the cause … just Vista-64 Home Premium remains. That OS sucks.

I’ve been running Xen and VirtualBox VMs here. We’ve been experiencing VM performance issues on 2 VMs under Xen on a fairly powerful box. Basically, the DomUs get really slow – slow.

Here’s what I know.

1. while connected via ssh to a DomU, sometimes my shell becomes non-responsive for 30+ seconds.
2. A php network monitor too shows the listeners aren’t answering in a timely fashion

Suspects:

1. sparse disk files for the images this is the issue – COMFIRMED
   1. I used sparse files for the OS images. This is the default instead of preallocated or LVM based disks. I need to migrate from sparse to full ASAP! `hdparm -t` shows Dom0 with 60-80 MBps throughput. On DomU it is 55-65MBps which should be good enough.
2. virtual network issues
   1. In older network bridges under Xen, there were real problems. The IO reported by others in tests were 3-4 Mbps over GigE networks. By using a virtio driver instead, they got 750+Mbps throughput. That’s what I’m talkin’ bout!
   2. There could be issues with my cheap switch and cheap router too. IP sessions may not expire quickly enough.
   3. I didn’t change any of the default network settings from the xen install. Perhaps this was a bad idea?
3. poor kernel settings for Dom0 and DomUs
   1. I’ve check the scheduler for issues – the defaults in Hardy and Xen 3.2 seem to follow the suggestions I’ve found thru google.
4. Similarly mis-configured DomU OS installations
   1. Well, it is believable that I made some other bad choices related to OS installation. I don’t know what they could be.

I have some work to do. Check back here for the results.

Results

1. I happened to be watching disk I/O and CPU when another lockup/slowdown happened. loop0 was the top process at the time. That means – disk is the issue. Nice.
2. Ok, after huge amounts of research, I was finally able to mount the sparse and full disk file images. Then I attempted to dump/restore the old system onto the new files (full/non-sparse). Done. Then I took a huge step and started the VM up – the new VM. Everything worked. FAST and with only 128M of RAM instead of the normal 512M that the old image had. It really is amazing … so far.
3. So assuming that actually worked perfectly – 10 minutes of use only – now it is time to do the same thing for my Zimbra installation. I am feeling lucky, punk. Get ‘er done. I’ll report back here in an hour or so.
4. So I swapped the Zimbra installation from a sparse file to a FULL file system. I’ll watch the monitoring software to see whether the 30+ reported outages per day keep happening. My blog installation definitely is showing higher responsiveness, definitely. I haven’t seen the slowdowns on a console, but I haven’t sat behind the server for hours yet either. We shall see.

Steps to clone/backup a VirtualBox domain

that can be restored to the same or a different machine. Start by shutting down the VM to be backed up/cloned. Then …

$ VBoxManage clonevdi Master.vdi Clone.vdi

and the (undocumented) function

$ cp Master.vdi Clone.vdi
$ VBoxManage internalcommands setvdiuuid Clone.vdi

The 2nd command stuffs the clone with a new SID.

Warning – this doesn’t backup any SNAPSHOTs

you may have made below the {root} VDI file. Here’s a directory view

 C:.

 ├───Machines

 │   ├───Ubuntu Upgrades

 │   │   └───Logs

 │   └───WinXPPro

 │       ├───Logs

 │       └───Snapshots

 └───VDI

VDI/ contains the base installations. Machines/ contains each XML DomU description file and snapshots where they exist. You need these for your backup too. Copying the .vdi files is not enough to back them up. While it will probably work on the same PC with VirtualBox, taking those files to another machine with a slightly different configuration may not work. Use the VBoxManage command.

_* I’m not responsible if this doesn’t work for you. Always work with test data when you are doing any of this until you prove it works on your machine.

Vista is slow even when you get a new machine. I bought a new Dell Studio 1535 laptop last week with this config:

• Core 2 Duo T8100
• 4GB DDR2 RAM
• Intel x3100 Graphics 1280×800-eh
• 320GB disk
• Vista Home Premium 64-but

Out of the box, this config should be FAST, VERY FAST. It isn’t. It’s slow, really slow. I will admit, it is pretty, but I didn’t buy this $1k machine to be pretty. Vista is slow on my 4 day old laptop with 4GB of RAM and no extras installed. Unacceptable – Dell and MS, are you listening?

Run WinXP in a VirtualBox VM

Preliminaries

1. Do you know what a virtual machine is? "Check out this Wikipedia article ":http://en.wikipedia.org/wiki/Virtual_machine to start.
2. You’ll need to be committed to this process. This is not a quick fix. You’ll be living with this configuration going forward. Every 3-7 days, you’ll want to reboot Vista for sanity.
3. I got a BSOD on Vista last night after I’d shutdown the VMs and VBox; no other data loss that I can tell
4. You’ll need a legal WinXP license that can be activated with MS.
5. Read all the directions through before you start. I’m not responsible if you fail and destroy your only working PC.
6. Get any WinXP network drivers before you start. Actually, download every thing you will need before you start since your networking may not work for a few hours. A better idea is to have another PC with network connectivity ready to get the stuff you forget to get.
7. If you change between WiFi and wire LAN connectivity, be prepared to use `arp` and `ipconfig /renew` alot. Vista seems to disconnect from networks whenever it feels like it.
8. Expect this to take a good 1-2 hours to setup the first VM not including the 2+ hours to install WinXP. Later, setting up VMs takes about 5 minutes when you know what you are doing.

Suggestions for your new Vista box

If you can’t or are unwilling to wipe vista off and start over with a fresh XP or Linux install, like I was, perhaps …

1. Buy more RAM, 3+GB – 4GB if you have 64-bit Vista. I don’t think that will help with the speed, but it will help with what we have planned. Vista “likes” at least 2 GB of RAM to itself, so if you only have 2GB of RAM, move along, there’s nothing to see here.
2. Configure Vista to be a minimal OS, UNinstall every app that you aren’t using. The goal is to have the minimum disk, memory and CPU footprint, while leaving enough to
   • control your networking
   • not get hacked
   • Uninstall Norton, MacAffee, and all the other crap “included”, especially the media center garbage. Leave an antivirus scanner – I"m using AVG Free version with URL checking disabled (for privacy)
   • Set the machine for “best performance” to turn off all the fancy shading and other effects and set the GUI for “Classic mode”.
   • Defrag the disk if that’s an issue – probably not.
   • Reboot – go into the BIOS and enable the Hardware Virtualization Instruction support. AMD and Intel call these different things. This is needed to improve performance.
3. Grab the non-OSS VirtualBox – the license will work for non-Corporate IT supported installs and personal use. This version supports USB devices, the OSS version doesn’t. You can use it on company equipment provided IT doesn’t install it.
4. load VirtualBox on your Vista machine – vbox going forward
5. In vbox, use Host Networking (that means you’ll need to be comfortable setting up “Bridged Networls” – without this, you’ll end up with a PC that isn’t connected. You can use NAT connections, but that wasn’t useful to me, so you’re on your own.
6. In vbox, use the Intel PRO 1000 Network setting. Linux has drivers for this built in, but you’ll need to download the drivers from Intel for WinXP. This is important as the other network options gave me problems with disconnects, etc.
7. In vbox, 20GB Disk – use which ever disk type you like, but know that the choice will impact performance and backups.
8. In vbox, Change the video memory from 8M to 128M. I think this could be important for running at higher resolutions with true color graphics. Most of you won’t recall the days of 16 or 256 colors. All PC graphics cards support 16M+ colors at 1024×768 resolutions now.
9. Put your WinXP install CD into the PC and configure this new VM to boot from it. There’s 2 places to be certain it boots the CDROM.
10. Boot the VM
11. load WinXP into a VM under it. Use an old XP license from an old machine that you don’t run anymore. My old Dell 600M laptop XP disk installed flawlessly and the MS registration went through easily, no phone call needed. I had another WinXP Home license ready.
12. Don’t do anything with the screen resolution yet. Get get the networking in the VM working. I wouldn’t load any software at this point either – now WinXP Patches or SPs.
13. Load the vbox client addons to get higher resolution graphics and other nice additions.
    1. The client addons disk was included in your Vbox install,
       1. just mount the CDROM image on the VM setup menu (shutdown the WinXP VM, modify the CDROM ISO and reboot the VM). Don’t boot from it.
       2. Boot using the HD, not the CDROM
       3. Find the loaded CD and run the install program.
    2. Now configure the local drive access, screen resolution and I changed the default get out of VM key from to F11.
14. Install all the WinXP patches (SP3 etc.) then shutdown the VM and back up VM files off to another disk or server. Don’t just copy the files.
    1. Create a “checkpoint”, if you like.
15. Next I installed many, many of my apps, from MS-Office 2003, Firefox, Thunderbird, Divx, VideoRedo Plus, VLC, Opera, TrueCrypt, etc… and made another backup.
    1. Create a “checkpoint”, if you like.

Results

1. I’m getting nearly native performance on my XP VM. Oddly, Vista is still slow on the host system.
2. When I’m running full screen, I forget I’m in a VM even when doing video editing or encoding. Same for Linux VMs.
3. Unless you are running fairly new games, this setup works with the extra niceness of trivial system backups, checkpoints, etc. If you use new games, forget VMs, this solution is just for productivity workers, lite gamers. I don’t know what level of DirextX is supported inside a VM, if any.
4. Keep your “data” outside the VM on the host drive. The VM just holds programs and settings – setup around 20GB for XP, mine is using around 10GB now with MS-Office and about 20 other programs installed.
5. Backups – perfect, recoverable systems to any VirtualBox running host. Back to exactly where you were in 2 hours or less on a completely different machine. This is the programs, not the data.
   1. Backing up VMs is not a trivial `copy`.
   2. Data backup – use MS-SyncTool or some other way to back it up daily over your network.

Update 9/2/08: If you get stuck, use google and read the virtualbox wiki carefully. I am not available as your personal guru without an hourly contract + expenses.

Update 9/3/08: Today is the first time I’ve connected to my KVM and used my desk with this new laptop. Everything works good, but it doesn’t exceed my expectations. The screen resolution changed to the native 1920×1400 of my 24" monitor. There is lite video flickering, that may drive me nuts; the laptop X3100 graphics chip isn’t the best, but the additional screen real estate is much needed. I suspect a better host graphics card would be significantly better.

More on Virtual Machines like VMware, VitrualPC and VirtualBox.

10-19.2008 update

So the lockup issue for vbox under Vista-64 was due to a huge memory leak in some Microsoft code. MS has a hotfix, but not a patch for this. That means you have to register, agree not to share the fix and use a password to access the fix. It is fairly easy to know which hotfix – the vbox guys have it documented clearly.

Basically, the VirtualBox guys started using a screen refresh routine that nobody else had used in Vista. A few dot releases later and vbox will stop using that call. It isn’t available in a formal released version yet.

Since applying the MS hotfix, my vbox system has been fairly stable. Zero BSOD.

The network disconnects still happen, but have been reduced significantly. Rather than every 15 minutes, they are every few hours. I changed my wireless router out (d-link —> linksys). That is the only noticeable change besides the memory fix. The disconnect happens under vista without any virtualization running at all. Bad Microsoft, bad Dell. Bad. My old laptop (WinXP and Dell) had no issues with either wifi router and disconnects.

I had a need to upgrade from a 6.xx release of Ubuntu this week and decided to do that rather than perform a fresh install of the current distro. My server has 3 years of tweaks – I’d hate to have to recall what they are, much less attempt to reproduce them on a fresh install. Not gonna happen.

All the instructions online explain how to do this with a GUI. What do you do when the X/Server doesn’t come up? That happened on my test machine on 7.04 – well, that isn’t exactly true. X did come up, but nothing else did. No menu, no left or right click on the desktop. Nada. Networking was up – I’m glad that I bothered to get it working on the first install – this was just a test machine anyway.

So the shortcut for each upgrade 6.xx>7.04>7.10==>8.04 is:

1. sudo vi /etc/apt/sources.list
   1. edit edgy ==> feisty
2. sudo apt-get update
3. sudo apt-get dist-upgrade
4. sudo apt-get -f install
5. sudo dpkg —configure -a
6. sudo lsb_release -a

Ok, so if you lose your desktop, add in at the end and reboot.

1. sudo apt-cdrom add
2. sudo aptitude update
3. sudo aptitude install ubuntu-desktop
4. sudo /etc/init.d/gdm start

Output should show a new release.

Simple?