JDPFu.com 2025

I’m asking for help again with my Windows7 final installation. See, Microsoft gave the 32-bit version, not the 64-bit version. This puts a wrinkle in my original plan to host Win7 on the laptop because about 0.6GB of RAM cannot be used. On a system with only 4GB, 0.6GB is a bunch, perhaps too much to waste.

The current goal is:

JeOS/Linux-Host
|____Win7-VM (MCE)
|____xubuntu-VM
|____WinXP-VM (Visio / MS-Office / Quicken)

RAM allocation plans

JeOS – 512MB

Win7 – 1GB

WinXP – 1GB

xubuntu – 1.5GB

If Media Center in Win7 doesn’t work well enough in a VM; safe to leave on 24/7 with USB support, this plan will be trashed. The QAM recording is nice. For me, it is about the recording, not the playback or other features.

There are other complications in using Win7 Media Center. The recorded file format, for example. That’s something for another story.

32-bit DVD – ouch.

So I opened the Windows7 Ultimate DVD and uncovered that it only contains the 32-bit version. After swapping the old/Vista drive with the new/Empty drive in the laptop, I elected to install Win7 even though I’d end up unable to use about 0.5GB of RAM. I wanted to give the new OS a fair chance and gain some experience. The setup was fairly easy, but dumbed down too much for my liking. I actually installed the OS to the wrong partition (280GB), wiped it and reinstalled to the other partition (30GB), that was planned for OS and Apps. Then I proceeded to setup WMC – Media Center.

Windows Media Center – ClearQAM Supported!

I’d heard that ClearQAM was supported and looked forward to using it. My cable system switched almost all channels to QAM 2 months ago. I’d hoped there was an automatic translation between QAM channels and normal cable channels so guide data can be used. I haven’t found that, if it exists. I AM recording a movie as I write this. There’s no noticeable performance hit. Nice.

Data Migration

Overnight, I copied the data and virtual machines from the older drive to the new drive, about 150GB. I split the disk into 2 partitions – C: and D: . C is for the OS and programs. D is for data and virtual machines. This config should make data backup much easier.

I dislike the whole Library BullShite that this new OS forces. I also dislike the new Explorer look and feel. Is there a way to default all Explorer views to Detailed?

VirtualBox Migration

So, after getting the new OS installed, the very first program installed was Sun’s VirtualBox. Initial attempts to migrate all the settings and virtual disks didn’t work as well as I’d hoped. However, I did get 1 VM up and running with a small amount of effort. I’ll write up the actual steps and things that didn’t work in another post. There may be another way to migrate the settings, I did retain both XML files for Vbox and for each VM, so seeing the specific differences should be easy.

The next trick is to migrate a vbox image that includes a snapshot image. I’m cautiously hopeful for a good outcome with that.

I need your help deciding how to use the free Windows7 Ultimate license Microsoft gave away yesterday. I want to use it on my laptop but need some considered feedback on how would be best?

Current Laptop Config

1. 4GB of RAM – may put 8GB in later
2. 320GB disk
3. Main OS is Vista-64bit Home Premium
4. VirtualBox 3.0.6 for Virtual Machines
   1. xubuntu
   2. WinXP Pro
   3. Ubuntu
   4. OpenSolaris
   5. FreeBSD

Initial Thoughts

My initial thoughts are to

• replace Vista with Win7-64
• eventually remove my WinXP-Pro VirtualBox
• use the built-in WinXP Compatibility layer

I spend 14 hrs a day in the xubuntu VM and only boot WinXP to run Quicken, a few MS apps and access TrueCrypt data. Perhaps 3 times a week.

Questions?

1. How good is the USB support in the WinXP VM?
2. HDMI output?
3. GigE networking – WiFi networking?
4. How good is the driver compatibility for Win7-64? All-in-One Fax, printer, scanner, old Creative Xen and built-in laptop camera are the only devices I see using, in addition to normal flash and ext USB disk drives.
5. Hauppauge 950Q ClearQAM TV tuner must work.
   • Does Media Center work with this TV tuner and ClearQAM? The current MCE doesn’t.
6. Can I consider Win7-32bit at all. Does it access the full 4GB of RAM? Is an upgrade to Win7 64-bit easy?
7. TrueCrypt, MS-Visio, MS-Office 2007, and VideoRedoPlus are the only uses for Windows that I have. No gaming, er … very little gaming.

Choices

1. Run Win7 in a VM, get used to it. Decide later
2. Backup the data and VMs, repartition the disk for OS, Apps, Data, and install Win7 ??-bit as the main OS
   • 32-bit or
   • 64-bit?

Thoughts and suggestions? Did I miss an option?

I just got back from the Microsoft Windows7 Launch in Atlanta for developers – the IT Professional track was full.
There wasn’t much new to see since there were only 2 tracks for developers (8am-noon) and I decided to learn more about MS-Server 2008-R2, 08R2

Session 1 What’s New in Server 2008 R2 – Murray Gordon

The main things covered that interested me about 08R2

1. Upto 256 CPUs supported
2. Only 64-bit (32-bit Server OS is dead)
3. Win7 and 08R2 share the same core OS
4. IIS 7.5 is included
5. PowerShell with remote execution
6. VHD file support included
7. Virtualization is built-in, not an add-on
8. Boot from an OS install or a VHD (this is nice)
9. During installation, you decide how much of the Server OS you want even to the point of ZERO GUI installs

Session 2 Parallel Programming – Glenn Gordon

Glenn talked about the differences in sequential, threaded, and parallel/task programming.

1. Win7 and 08R2 are needed for the new “Task” APIs
2. VisualStudio 2010 includes .NET v4.0 with parallel extensions
3. It is unclear on any backward compatibility to prior OSes with .NET 4
4. Parallel Extensions make it easier to use whatever CPUs, Cores, hyper-threading an OS can support.
5. There are 3 parallel programming models – I didn’t bother writing them down since the programming examples seemed contrived and didn’t include any concern about data homogeneity. Scary.

Were I windows-only developer, they certainly made using multi-core systems much easier. The downside appears to be that only Win7+ and 08R2+ can use these extensions. I could be wrong. Let me know if there will be backward compatible solutions.

Session 3 IIS 7.x Features – Glenn Gordon

Basically, IIS 7.5 is a modular system (gee, like apache?) now and you can place your code almost anywhere in the pipeline, extending IIS with your code. He went on and on about some trivial examples that placed copyright notices into JPG files as the file was being served. You could, for example, tag every image with the authenticated userid, website and timestamp without touching the source image files.

The thing that really seemed neato to me was that MS finally is building a web application easy-installation community that makes it trivial to package solutions, deploy them or simply publish them for others to use. http://www.microsoft.com/web/ is the community portal and Web Platform Installer v2 is the enabler. It wasn’t clear whether these tools worked with any IIS older than 08R2 or not. The packager creates a ZIP with everything needed from the development machine and is a 1 file package provided to the IT production support guys for deployment. It modifies those settings that are different between developer and production servers during installation. Nice.

To make a good point, Glenn pulled both a DasBlog engine and MediaWiki from the community and deployed both to his server in real-time. The blog engine came right up and was ready for blogging. I didn’t see whether MediaWiki worked as easily – I run a mediawiki for the company. There were many well know open source projects in the list.

Of course, there were the normal give aways – Win7 Ultimate license and all the 180 day server stuff you can stand. the 180 day stuff you are free to download anyway.

Nice job from Microsoft and the Gordons who presented. I wish I could have attended the afternoon sessions that were more about enterprise deployments.

I came across an old article that I wrote on backups that had some clock times for the different VMs. Since that article was written, I’ve changed the backup methodology from rsync to rdiff-backup.

dms44 → 1m:52s   Alfresco

crm46 → 3m:36s    vTiger

xen41 → 3m:10s    Typo

pki42 →  1m:17s

mon45 → 1m:8s

zcs43 →  3m:53s   Zimbra

Those are real “downtime” numbers to ensure completely safe backups were made with all files closed. Actually, the virutal machine is shutdown during the backup periods. Email is unavailable for 4 minutes at around 2am daily. We can live with that. Recovery works perfectly too. I’ve recovered the largest VM twice in under 20 minutes after some cockpit errors.

This works because we use Xen virtual machines and rdiff-backup. Most of the VMs are 20GB in disk size, but use less actual storage.

Getting Syslog, Pound and Mongrel to work with Awstats

If you run the Ruby on Rails blog, Typo, it is likely you are using Mongrel as a cluster server and not Apache. Mongrel is easy – really easy. If you need 5 backend ruby servers, change 1 entry in the mongrel_cluster.yml file and restart.

Pound is a very simple load balancer written in perl. Many very busy websites use it. Slashdot for example gets 40M pages a day, all going through pound. Scalable? Check.

I’ll assume you already have syslog, awstats, pound, and typo/mongrel installed and simply want better logging. Explaining the nontrivial setup of these, sometimes complex, systems is not something handled in a blog. You’ll need to be root or have root editing via sudo to make this happen. A knowledge of manpages won’t hurt either.

So now you have two non-standard programs handling your web traffic, mogrel and pound. You’d like to get some normal website statistics about your users. By default, pound logs via syslog. We love syslog, but we don’t like that pound doesn’t use a separate file, by default. All your web traffic logs get intermixed with login, attempted hacks, disk failures and other system messages.

Below we’ll

1. describe the syslog setup to trap pound messages and drop them into a new logfile – /var/log/pound.log
2. setup the new logfile to be automatically created, should it disappear
3. setup pound to write the the new pound.log file via syslog
4. automatically perform log file rotation, in the normal way
5. create a custom log_file_format so awstats gets all the data it can from the logs
6. be certain that restarting pound gets syslog to bounce or restart too

Here are the changes specific to the logging changes. You should have other changes for your server/domain already in these files. Before starting, you probably want to run an

awstats.pl -config=domain.com -update

to capture the latest stats before you move them all into a new location.

/etc/awstats/awstats.domain.com.conf:

 LogFile=“/var/log/pound.log”

 LogFormat=“%time3 – - %host_r %host – - %time1 %methodurl %code %bytesd %refererquot %uaquot”

 SkipHosts=“REGEX[^192\.168\.]”

/etc/syslog.conf

 .;auth,authpriv.none,local0.none              -/var/log/syslog

 local0.*                        -/var/log/pound.log

See, local0 – local7 are approved syslog classes. They are meant to be used just like this. Syslog know about them, we need to be certain that pound will use local0 too. If your system is using local0, then select 1, 2, 3 … local7, which ever isn’t already in use.

/etc/pound/pound.cfg

LogFacility local0

 LogLevel        3

You’ll need to have your Service, Redirect, and BackEnd stanzas too.

/etc/init.d/pound

 if [ ! -e “/var/log/pound.log” ] ; then

   log_warning_msg “Creating pound.log …”

   touch /var/log/pound.log

   chmod 0644 /var/log/pound.log

   chown syslog.adm /var/log/pound.log

   /etc/init.d/sysklogd reload > /dev/null

 else

   log_success_msg “pound.log was found”

   /etc/init.d/sysklogd reload > /dev/null

 fi

/etc/logrotate.d/pound

/var/log/pound.log {

        daily

        missingok

        rotate 14

        compress

        delaycompress

        notifempty

        create 640 syslog adm

        sharedscripts

        postrotate

                /etc/init.d/sysklogd reload > /dev/null

        endscript

 }

Good enough? Now just restart pound with

 sudo /etc/init.d/pound restart

The next time your awstats is updated, you’ll see more and better stats. Note that we didn’t touch any of the old rrd data that awstats may have been able to parse.

This worked on an Ubuntu server 8.04 LTS running in a Xen virtual machine. There are other ways to do this and some settings can be changed without impacting whether this continues to work or not.

Obviously, your situation will be a little different and you’ll need to figure out which differences matter and which don’t. Did I miss something important or does anything need clarification? Use the comments or talkback to let me and other readers know, please.

Overview of LinuxFest Atlanta 2009

I attended LinuxFest Atlanta 2009 with
700 like-minded people. Lots of good information for the price –
basically free.

There were about 42 sessions organized
for all levels from beginngers (I didn’t count them) from Fixing
Audio in Ubuntu/Linux to
multiple Kernel Hacker sessions (Debugging the Kernel,
4 Driver Writing Sessions,
etc.). There were more sessions offered than I could hope to attend.
Due to many late sign ups (about 300 extra), many of the sessions
were standing room only and overflowed into the hallway. I was able
to get a seat by going directly from session to session quickly.

We
need to thank IBM http://www.ibm.com

for providing facilities to this conference. There wasn’t any IBM
advertising that I saw. A
BIG THANK YOU, IBM, from
me. There were other supporters too with tables in the common areas.
Linux Journal, SuSE, LinuxPro and Cononacal are a few from memory.
Many companies hosted extremely informative sessions.

My session attendence:

• What Community Has to Offer – OpenSuSE

  
  

• Linux, Hadoop, and Amazon Web Services: Crunching the Big Data in the Cloud

• Free Software Development with Clouds

• Securing Your Network wth Open Source Technologies

• Running and Open Source Business

• The Weather Ahead: Clouds

There were other
sessions I would have liked to attend, but the conflicts prevented
it.

What Community Has to
Offer – OpenSuSE

Presenter: Chuck Paynehttp://opensuseterrorpup.blogspot.com/

Slides:http://www.magidesign.com/download/alf.odp

The presenter is an OpenSuSE
evangelists and works at the Travel Channel IT in Atlanta as a
sysadmin. He provided a survey of the different tools and
distributions that OpenSuSE provides.

OpenSuSE Studio:

Using the OpenSuSE Studio tool, you can
build a specialized distribution for your team, clients, family,
school. A concrete example was that you could build a server and
desktop distributions for students to perform homework with identical
software available to all from a Live CD boot.

See the
“slides”:http://www.magidesign.com/download/alf.odp for much
more.

Linux, Hadoop, and
Amazon Web Services: Crunching the Big Data in the Cloud

Presenter: John Willis
http://www.johnmwillis.com/

Slides: not available.

Basically, this talk was a list
of companies, FOSS tools, and techniques around dealing with huge
data sets in parallel on cloud infrastructure. It started with the
NIST definition of Cloud Computing and
ended with how to monitor and merge data from hundreds of individual
systems for an overview. My notes are just a list of tools that I
found interesting during the talk.

Libvirt, OpenNebula, OpenQRM,
Cobbler

RightScale.com

Nanite, Capistrano, ControlTier

Eucalyptus, Enomaly, Nimbus

OpenVPN, CloudNet

Splank

Chef from Opscode, Puppet,
Cfengine

CollectD, jCollectD

Big Data Frameworks: Pig, Hive,
Cascading

It’s 2 days laters and I’ve
checked out RightScale and collectD. We use SysUsage

for monitoring our small group of systems. I must have missed the
main points of this talk. Lots of data, but nothing that made me want
to change jobs.

Free Software
Development with Clouds

Presenter: Deryck Hodge
(Canonical) http://www.devurandom.org/

https://launchpad.net/

is a Canonical-backed software collaboration website. The goal is to
provide everything except compilers for software development
projects. Here’s a bullet list:

• Blue Prints – architecture
  diagrams

  
  

• Version Control via Bazaar
  with branching and merging

  
  

• Bug Tracking

  
  

• Threaded discussions

• Release Management

• Collaborative Translations –
  language files

• Karma system

• Code Reviews can be
  mandatory – PQM-based

• Open Source, but getting it
  running inside your company isn’t easy and they won’t help you. They
  said it would require 15+ servers. Get the source here:
  https://dev.launchpad.net/

While the website has things for
project management, it is tailored to software development projects.
A comment from the croud that tracking server deployment with it was
very possible. Free accounts let anyone have access to view your
project details. Paid versions provide project privacy, if you like.

Securing Your Network
wth Open Source Technologies

Presenter: Nick Owen
http://www.wikidsystems.com/ Lots of how-to guides.

Lots of detailed information, a
little too fast for me, about securing your network, applications,
and users. Here’s a link to the presentation. Basically, use RADIUS and 2-factor authentication.
RADIUS is supported by every vendor and standards were created before
anyone wanted a niche. RADIUS works with Apache, PAM, Microsoft, and
many routers.

Admins are happiest when there are
no users.

Tell all your passwords to go to
hell.

I need to check on

• RADIUS support in pound (a
  load balancer)

• Remote Desktops support
  RADIUS

• Using RADIUS in OpenVPN

• Apache front ends – don’t
  allow anyone to our apache services until they network authenticate
  via RADIUS

• One Time Passwords –
  WikID, Opie, FreeToken, OTP Auth

• FreeRADIUS – AIS
  (Microsoft)

This session provded the greatest
value for me.

Running and Open
Source Business

Presenter: Tarus Balog
http://www.opennms.com/

Basically, this was a talk on how
to start a business with a slant on FOSS. Get a laywer, CPA,
insurance and all the other things you need for a business. Give the
software away and encourage a community to form that provides patches
and modules back to you. He only knows how to make money selling
services for tools, not applications. How much are you willing to pay
for OpenOffice support and installation? $0. OTOH, how much are you
willing to pay to monitor your servers with a great tool that is
complex to install, but easy to run? $10,000/yr?

Main tips:

• Don’t quit your day job

• GET A TRADEMARK and copyright everything -

  $300
  and a year of your life

  
  

• Build an awesome app or tool

  
  

• Start a foundation and get a
  company to fund it. IBM funds lots of foundations that Microsoft
  hates.

• If you use GPL for your
  license, anyone that steals your code must release their code too.
  If you use BSD or Apache or other do-what-you-like licenses, they
  can be secret.

• Copyrights

• Owner can change the license
  at any time

• Defend the code from license
  abuse

• Sun started theee Dual
  Copyright

• Have a Contributions Agreement

  that gives you and the contributer both copyright ownership. This
  lets you change the license in the future without asking permission
  from everyone that contributed 15 years ago. Clone the Sun
  agreement.

  
  

• Get ramen
  profitable
  – earn the amount of money to life.

  
  

• Spend less than you earn

  
  

• There’s
  a diagram in the book – Crossing the Chasm -
  

  The difficulty is in getting
  enough customers to be #1 or #2 in your market and becoming an Early
  Majority solution.

  
  

• Release
  code early and often – The Practice Effect

  
  

• Create
  products that are easy to buy – not things that are easy to sell

  
  

• Create
  a website

  
  

• Separate
  work from life.

• Create
  a blog http://www.adventuresinoss.com/

• Be
  results driven, not effort driven – my addition

• Build
  CRM, Trouble Ticketing, and bug tracking BEFORE you need them

• Create
  a mailing list and/or forums to let your community chat

• Participate
  in the community – go to conferences and give talks

• Twitter,
  facebook, whatever for marketing

• Get
  Paid:

• Easy
  pricing – “bundle of knowledge consulting”

• Get
  customers – don’t do free stuff

• Net-30
  – offer a discound, 2%, for paying early

• Statements
  of Work – SoW or do time and materials, T&M

• Annual
  Renewals include consultations, upgrades, etc. If you charge
  $15k/annual support and have 100 customers, you have a business.

• Value
  your employees – 401(k), Health Insurance, Payroll Service; People
  are your company

  
  

• Use
  the Bowling Pin model; after you sell 1 pin, discover 9 other things
  each customer needs and offer it.

• Grow
  or die

• Fire
  a bad customer – life is too short for work you really hate to do.

• How
  to get out?

• IPO

• Make
  a great lifestyle company

• Sell
  to a big company – If someone offers $30M, do you take it?

  Obviously
  from my notes, I liked this talk.

The Weather Ahead:
Clouds

Presenter: John

Ubuntu Jaunty includes a cloud API identical to Amazon S3 and EC2
serivces. This means you can build and test internally, then deploy
with binary compatibility to Amazon or other compatible cloud
providers.

Today, cloud computing is like electricity; turn it on when you need
it. Turn if off when you are done.

No capitol costs.

Ubuntu1 – storage

Landscape – SaaS – stats, hw, sw, trending, patches

AMI – Amazon Machine Image

I need to research switching from Xen to KVM for our internal VM
systems. Managing a cloud is less like managing a group of VMs.

Always migrate forward, never go back. If you have an issue, grab
the next machine, migrate and get it working. Later, you can go back
to the non-working version and figure out what happened or destroy
the VM.

The key things to consider when purchasing a Windows/Linux Laptop are:

• Screen Size / Resolution
• OS
• CPU
• RAM
• WiFi / Wireless
• External ports (USB/Firewire/eSATA)
• Disk Size
• Memory Card Reader
• Install CD / Recovery / Backup
• Warranty / Type of Support
• Applications Preinstalled

Enable ssh access to ESXi.

Instructions for the VM Backups using GhettoVCB script from William Lam

The GhettoVCB script. The link to the script is about 5 pgs down, but nowhere near the end of this web page.

General VMware ESXi scripting page.

Essential VCB Backups.

As I ponder how to build a redundant file server that serves Linux, Solaris, VMware, Xen, VirtualBox, FreeBSD, FreeNAS, TiVo and Windows systems, a few interesting articles have come to light.

• A home file server using ZFS
• Snapshots into the cloud
• Backups with versioning
• 10 Reasons You need to Consider ZFS

Requirements

Basically, I’d like

1. reasonable amounts of redundancy
2. hardware agnostic
3. FOSS (non-commercial)
4. Enterprise ready – support for iSCSI, CIFS, Samba, NFSv4, RAID levels, snapshots, and versioning
5. remote backup capabilities – rdiff-backup would be ideal
6. Offsite backup capabilities – any type of external storage “in the cloud”
7. Encryption of offsite backups
8. high performance capabilities
9. Suitable for file system, database and raw disk device access

More on this as I work through the solution over the next few days and weeks.

BTRFS

Of course, I came across this article on btrfs a few days later explaining the it will likely be the default Linux file system in a few years. It also explains that any file systems created prior to kernel 2.6.30 are incompatible and with later kernels. Today, I’m running 2.6.24-24-generic SMP. No go.