JDPFu.com 2025

Enable ssh access to ESXi.

Instructions for the VM Backups using GhettoVCB script from William Lam

The GhettoVCB script. The link to the script is about 5 pgs down, but nowhere near the end of this web page.

General VMware ESXi scripting page.

Essential VCB Backups.

Sometimes I come across a story that combines multiple technologies. This story has many things that I love.

1. Japanese – Check
2. Sushi – Check
3. Space Station – Check
4. Astronauts – Check
5. Experimental – Check
6. Long term space travel; 4.5 months – Check
7. Danger – Check
8. Underwear – Check

There’s an older, NASA-style article on underwear too.

Got dressed early this morning, drove to the local quickie mart and bought the VERY thin AJC newspaper this morning. See, I haven’t bought a newspaper in about 20 years, so I wasn’t certain that this tiny, thin thing was the version I needed, the version with the Fry’s Tax Free Holiday Sale Ad in it. Abdullah said that was the normal paper. I quickly flipped looking for the Fry’s ad – didn’t obviously see this main item of my quest, but bought it anyway. The AJC doesn’t put commercial ads online, which sucks. Other cities have Fry’s Electronics Ads online, but not Atlanta.

Got home … and started searching the ad group. Nothing. Started flipping pages, 1 by 1 hoping for at least a half page ad with motherboards, CPUs, etc…. eureka! An insert that didn’t look like an ad insert, but was more than just a few pages – 8 pgs! Excited, I grab my morning tea and start flipping anxiously through those wonderful pages looking for the components to build a new PC. TVs … keyboards …. laptops, monitors, cameras, printers … ah – RAM! The RAM I need – at a good price, but not a great price. Then GPSs, vacuums, software and games. Are these tax free? 1 hard drive too small and slow to be considered. A few non-interesting motherboard/CPU combos and telephones. Eh.

No relevant CPUs, motherboards, power supplies, or great RAM deals. No trip to Fry’s needed today.

I want my $0.75 back.

See, in previous years, I’ve been able to build an entire PC for $100 less than normal using Fry’s ads on the tax free weekend. This year, I won’t even make the 45 minute drive to visit the store. Sure they had 8GB USB thumb drives for $19, but not the type that spreads access across multiple internal bays to make them last longer and be much faster. Nothing of real interest or a deal I can’t pass up.

At least I’ll get to sip my tea slowly this morning as I build an order from NewEgg and Amazon. Microcenter didn’t appear to have any deals either. A motherboard I found for $109 at newegg was $145 at Microcenter. Sorry guys.

To add more injury – the Fry’s in other cities have Intel Core 2 Quad Processor Q9400 & Gigabyte GA-EP45-UD3P Motherboard – exactly the deal I want. EXACTLY. Too bad it isn’t available local ‘cause they’d already have my money.

We’ve all had to learn about disappointment in our lives. Today was just a small reminder.

As I ponder how to build a redundant file server that serves Linux, Solaris, VMware, Xen, VirtualBox, FreeBSD, FreeNAS, TiVo and Windows systems, a few interesting articles have come to light.

• A home file server using ZFS
• Snapshots into the cloud
• Backups with versioning
• 10 Reasons You need to Consider ZFS

Requirements

Basically, I’d like

1. reasonable amounts of redundancy
2. hardware agnostic
3. FOSS (non-commercial)
4. Enterprise ready – support for iSCSI, CIFS, Samba, NFSv4, RAID levels, snapshots, and versioning
5. remote backup capabilities – rdiff-backup would be ideal
6. Offsite backup capabilities – any type of external storage “in the cloud”
7. Encryption of offsite backups
8. high performance capabilities
9. Suitable for file system, database and raw disk device access

More on this as I work through the solution over the next few days and weeks.

BTRFS

Of course, I came across this article on btrfs a few days later explaining the it will likely be the default Linux file system in a few years. It also explains that any file systems created prior to kernel 2.6.30 are incompatible and with later kernels. Today, I’m running 2.6.24-24-generic SMP. No go.

Clearing some Brush

In May, I decided to use a 10+ year old lawn mower to clear some brush in my back yard. This was after I’d manually removed all the brush that was larger than 0.25" in diameter. I did this knowing that it would probably destroy the mower and I’d need a new one. That old Sears $189 mower did clear everything and ran for a few more weeks before it died.

New Mower

So, I do a model little research and head back to Sears for a new mower expecting to get 8+ years of use out of it. It is a $212 purchase. It runs for 2 mowings beautifully before I attempt to cut some of the new brush growth at the edge of my lawn. By accident, it lands on a 1.5 inch stump and stops cold. That error caused the blade to deform badly, but since I’m not finished cutting the yard, I continue mowing the front yard without knowing how much the blade was actually deformed. The mower strikes a few tree roots strictly due to the deformed blade. The mower runs, but very rough. So I

1. replace the out of balance blade and it still runs rough
2. take it to a repair center for an estimate

Estimate for the Fix

A new minutes ago, the repair center called with a $315 estimate to fix my newly destroyed lawn mower.

Brand new, it cost $212 about 6 weeks ago. I guess they shouldn’t be surprised when I don’t come and pick it up the old junk?

The good news is that my yard will get mowed a week sooner, since buying, yet another lawn mower is required. I probably won’t buy that same model again. I’ll find one with a crankshaft disconnect bolt that fails before engine damage can happen. Mowers are supposed to have this safety bolt built into them to prevent engine damage.

Happy Mowing!

And watch out for roots and costly repairs!

I was catching up on the FINA World Diving Finals in Rome today and found the best video, without any breaks, in high def at Universal Sports. Every dive from every competitor in HD. No commercials. WOW.

Many of you know that I’ve found the coffee mug that keeps me happy and my tea/coffee hot for hours a few years ago.

It is the Contigo Extreme Insulated Mug. Get the vacuum insulated version, not the foam type.

I got lucky and found a pair together at a local store, perhaps 4 years ago, for $20. I’ve told a few friends about my cups but didn’t know the name or model. Random searching today found them – at least some mugs that look like my favorite hot liquid mugs. Further reading was required.

Seems they make 2 versions.

1. Foam insulated
2. Vacuum insulated <— recommended

They don’t appear to sell the 2-pak any more. Too bad.

I like:

• Handle clips onto a bag so you won’t lose it when empty
• Rubberized base – nonskid.
• 1/4 turn lid to close
• Spill-proof lid
• Lid doesn’t melt in the Georgia summer heat (lost a few mugs with cheap rubber seals this way)
• Keeps liquid HOT for hours with lid, warm for hours without
• Easy clean, but not dishwasher safe
• Two for $20 – impossible price to get now

I didn’t find the same model, but the description on this seems really good for $35.

Software Usability is an art form. It isn’t easy, but here’s a suggestion for all the product managers out there.

Give the software to your Mother. Can she use it? Can she figure it out with 10 minutes of 1-on-1 training?

If she can’t then, you’ve failed. Try again.

If there is a trick, then you’ve failed too. Apple likes their tricks. The first time I used an ipod, I couldn’t turn it off or change tracks because they were too lazy to properly mark the buttons.
The first time I used a Mac after college, I needed to format a floppy disk and couldn’t figure out how. The menus didn’t have “format” on them until the disk was inserted.

rdiff-backup rocks, mostly. But there are times when it doesn’t work as expected or doesn’t work at all. Usually, the not working at all part is a cockpit error, but sometimes not.

Key rdiff-backup features

1. Simple 1 line backup command; rdiff-backup source target
2. Reverse Incremental backup sets
3. Extremely FAST backups. Entire server installations are just a few minutes, after the initial backup set is created.
4. Last backup set is available as a complete copy of the files. Need to recover? Just copy the file(s) back.
5. Control over how old backup sets can be. Deletion of “older than x days” sets is trivial.
6. Compressed older differential backup sets
7. Current backup is 1-for-1 sized. Older backups are tiny. As an example, a 5GB backup with hundreds of files with 30 days of incremental backups only takes 6GB total. Each daily backup is relatively tiny and based on changes made that day. Usually those changes are just a 10-40MB. Impressive.
8. Recovery by date/time
9. FOSS – we like Free and Open Source Software
10. Cross platform – Unix, Linux, MacOS and MS-Windows.

Things that just work

1. Linux local rdiff-backup runs, just work. Backing up a directory structure or an entire VM (not as a single huge file) to another mounted disk works very nicely with all the key features listed above.
2. Win32 local rdiff-backup runs, provided there isn’t any networking involved nor huge files.
3. Recovery of an entire VM fileset. I’ve needed a few recoveries the last 6 months due to user error. They worked flawlessly and only took 20 minutes from problem discovery to full recovery. That was manual recovery. If this were scripted, it could be less than 5 minutes.

Things that don’t work or work poorly

This is mostly on MS-Windows platforms, but some Linux stuff doesn’t work nicely either. Windows howto that wasn’t really much use.

1. remote transmission over ssh on a non-standard port is broken regardless of platform. That doesn’t mean it can’t work, but I’ve never been successful in getting it to work. Neither push, nor pull command versions have worked. It shouldn’t be this hard.
2. Large file differencing doesn’t seem to work on Linux or Windows. In theory, that means 4GB files, but smaller files get confused and end up as a complete copy too, not a block level differential copy.
3. MS-Windows network backups don’t really work, even over samba connections. Ok, there are many strange things about rdiff-backup on Windows. For example:
   1. You have to `cd` to the drive and directory of the source if you want it to work.
   2. You have to use ‘/’ instead of ‘\’ characters, most of the time. This is a python thing, I guess.
   3. Backups to samba shares may or may not work. I haven’t figured the reason why or why not yet.
   4. Backups over ssh require less than trivial setup. Only push will work from windows unless an ssh server is setup. Then the complexity is exponentially more difficult.
   5. Many people use cygwin with all those faults (slow, heavy, bad directory access) to get around the win32/64 API issues.

So, rdiff-backup is great for local Linux system backups, but for remote backups, you’ll want to use different technology, like rsync. If you’re on Windows and want remote backups, check out some other solutions.

Good writeup on rdiff-backup features, method, and algorithm.

S1/Disk1 -rdff-backup→ S1/Disk2 -rsync/ZFS send→ R2/Disk1

If you have a lead or solution for my woes, please let me know! I often miss trivial solutions.

Here’s an actual rdiff-backup set to clarify:

        Time                       Size        Cumulative size

-—————————————————————————————————————

Fri Jul 10 01:32:13 2009         4.37 GB           4.37 GB   (current mirror)

Thu Jul  9 01:32:13 2009         36.3 MB           4.41 GB

Wed Jul  8 01:32:13 2009         37.5 MB           4.45 GB

Tue Jul  7 01:32:14 2009         31.4 MB           4.48 GB

Mon Jul  6 01:32:13 2009         31.1 MB           4.51 GB

Sun Jul  5 01:32:13 2009         27.0 MB           4.53 GB

Sat Jul  4 01:32:14 2009         41.3 MB           4.57 GB

Fri Jul  3 01:32:12 2009         33.9 MB           4.61 GB

Thu Jul  2 01:32:13 2009         37.9 MB           4.64 GB

Wed Jul  1 01:32:14 2009         35.4 MB           4.68 GB

Tue Jun 30 01:32:13 2009         37.3 MB           4.71 GB

Mon Jun 29 01:32:14 2009         38.9 MB           4.75 GB

Sun Jun 28 01:32:13 2009         38.7 MB           4.79 GB

Sat Jun 27 01:32:15 2009         42.0 MB           4.83 GB

Fri Jun 26 01:32:13 2009         49.3 MB           4.88 GB

Thu Jun 25 01:32:13 2009         37.3 MB           4.92 GB

Wed Jun 24 01:32:14 2009         36.4 MB           4.95 GB

Tue Jun 23 01:32:13 2009         43.0 MB           4.99 GB

Mon Jun 22 01:32:15 2009         33.4 MB           5.03 GB

Sun Jun 21 01:32:15 2009         31.0 MB           5.06 GB

Sat Jun 20 01:32:13 2009         41.6 MB           5.10 GB

Fri Jun 19 01:32:14 2009         31.7 MB           5.13 GB

Thu Jun 18 01:32:14 2009         32.0 MB           5.16 GB

Wed Jun 17 01:32:15 2009         31.0 MB           5.19 GB

Tue Jun 16 01:32:17 2009         31.6 MB           5.22 GB

Mon Jun 15 01:32:16 2009         31.7 MB           5.25 GB

Sun Jun 14 01:32:14 2009         31.3 MB           5.28 GB

Sat Jun 13 01:32:14 2009         30.7 MB           5.31 GB

Fri Jun 12 01:32:14 2009         31.3 MB           5.34 GB

Thu Jun 11 01:32:15 2009         32.3 MB           5.37 GB

As a technical architect, I’m pulled in many different directions when working on solutions design. Some include:

1. Solution meets the majority of must have requirements
2. Solution provides exceptional value to the customer
3. Solution needs to meet security requirements
4. Solution advances long-term technical needs
5. Solution can be leveraged for future, unknown, needs
6. Solution is open whenever possible to avoid single-vendor lock-in
7. Leverage existing available infrastructure, unless it has been proven a poor choice
8. Costs – deployment and annual maintenance must be considered
9. Solution must include the smallest number of components; the fewest moving parts. KISS, Keep It Simple Stupid, methodology.
10. Minimize support complexity. Complex solutions that can’t be easily supported will eventually fail. They cause too much trouble to use.
11. Ensure customer satisfaction by clear, concise communication with the customer and team on the expected outcomes, issues, and schedule. Provide public and private points for feedback from anyone on the team to say good and bad things. Sometimes failures can be avoided in this way.
12. Solution must support future migration to a different, competing vendor. Be certain that customer data isn’t locked up inside a complex system that can never be moved.

So, when you begin working a project that includes updates to the technical infrastructure, consider that your solutions designer will be trying to merge all these interactions into the best possible solution for your needs. Sometimes there is not a clear best solution and just a non-worst solution must be selected.

As a customer, I often find that our needs are ahead of the solution availability curve. No solution currently exists so we need to revisit the solution space in a year or two.