The book I read is Jez Humble and David Farley’s Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. I’m not sure if the weaknesses are due to Amazon’s ebook platform, this particular title, or some combination of both, but these were the issues I had:

1. The table of contents as displayed by the PC reader application, isn’t detailed enough.  The book is divided into three sections, and the sections appear in the ToC, but none of the chapters in the sections do. One of the entries in the ToC is the Contents as it appears in the book itself (which is also where you go if you choose Table of Contents from the iPad menu). It has links to everywhere, but is extremely detailed and spans 19 pages, making it cumbersome to use. There’s no easy way to just jump to a particular chapter.
2. Cross references don’t tell where they’re going in a meaningful way. They reference other parts of the book by page number, but Kindle books don’t have page numbers. There is a link, so not all is lost, but I couldn’t tell if a cross reference was to some part of the book I had already read (and that I ought to therefore nod and say, “Oh, yeah”) or somewhere later on. Maybe just a little arrow icon pointing left or right would have been a good hint.
3. The index is cumbersome. There aren’t any links to jump to the different letter sections, so if you want to look up “Tests”, be prepared to flip through a lot of pages to get to the T’s.
4. The figures are too low-res. One in particular, 15.1, was almost completely unreadable. It sounded like a really interesting chart, but it was just too small and couldn’t be zoomed so I feel kind of cheated out of that.
5.  Compound words were problematic because the dashes were usually dropped and the words mashed together. Apparently someone/something confused them  with dashes as a result of layout (to split words across lines) and dropped them all.

It seems possible that Apple, with their new iBooks 2 platform and its textbook focus,  could end up being a better experience for technical books like this. I’ll be interested to give it a try.

Amazon Linux (Basic 64-bit Amazon Linux AMI 2011.09 AMI Id: ami-1b814f72) currently comes with Python 2.6.7 on it. At first, I thought I’d just be able to update to 2.7 with yum, but as far as I can tell, there isn’t a package for that already.

Joshua Holmes wrote about updating Python, which was a great help. My experienced varied a little, so I thought I’d share in case it is useful to someone else. For example, I didn’t want sqllite, but I did want ssl support. Also, there were some cases where I needed to use sudo when logged in as ec2-user.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

sudo yum install make gcc gcc-c++
sudo yum install openssl-devel.x86_64
 
cd ~
wget http://python.org/ftp/python/2.7/Python-2.7.tgz
tar xfz Python-2.7.tgz
cd Python-2.7
./configure --prefix=/opt/python2.7 --with-threads --with-ssl --enable-shared
make
sudo make install
 
cd ~ 
echo '''
 alias python='/opt/python2.7/bin/python'
 PATH=$PATH:/opt/python2.7/bin
 ''' >> .bash_profile
source .bash_profile
 
echo '''
 /opt/python2.7/lib
 ''' >> opt-python2.7.conf
sudo mv opt-python2.7.conf  /etc/ld.so.conf.d/
sudo ldconfig
 
python -V
Python 2.7

Notes

Line 26 is output from Python — not something you input (if that wasn’t obvious).

The configure script (line 8 ) specifies which optional libraries will be included in Python. I’m not sure what the typical list includes (for example, which are included in the official Python install for Windows?), but when you run make (line 9), it will report all the missing libraries for Python modules (“Python build finished, but the necessary bits to build these modules were not found”). If you use a module with a missing library in a Python program, you’ll get a runtime error.

The echo lines (13-16) are editing your user profile file for bash so that when you type ‘python’ you’ll get the 2.7 version instead of the default 2.6.7 version. I had a situation where I needed to run python under sudo, but root doesn’t have the alias, so it kept running 2.6.7. After some unfruitful attempts to add the alias to /etc/profile (sudo doesn’t execute that), I just added a symlink and invoked ‘python2.7′ explicitly (thanks to Chris Brinker):

sudo ln -s /opt/python2.7/bin/python /usr/bin/python2.7
sudo python2.7 -V
Python 2.7

I guess I’m not the only one with this problem. A friend sent me a message thanking me for a gift basket I’d sent him a couple of weeks ago. He hadn’t opened it until it was time to start wrapping all the presents he’d ordered himself from Amazon. Hopefully the food was still edible.

At first I was thinking that Amazon needs to figure out how to solve that problem. If I were Amazon, I might just suggest gift wrapping presents to others so that it is always safe to open the box, but gift wrap starts to get expensive — especially if you have several less-expensive items.

Another solution is to address the packages in a way that differentiates them. I’ve changed my wishlist address so that it includes “(Wishlist)” after my name. My regular ordering address now has “(Self Order)” after my name. Now maybe I just need to add “(A gift for you)” after the names of my friends to tip them off.

I guess I’ll see how it goes…

The currently winning smartphone platforms are the gated and guarded community of iOS, and the wild-west of Android. Palm/HP had the vertical integration of Apple with the openness of Google. Maybe in today’s polarized society, the middle has no appeal. But don’t think that when I say “middle” I mean not as good as either extreme — it’s more like the best of both worlds.

Here are some of my favorite features of webOS:

1. Fantastic muti-tasking that no one else has yet matched.
2. A UI that is is richer than iOS and more beautiful than Android.
3. Over-the-air activation/updates/etc. Attaching it to a PC is never required.
4. Unobtrusive notifications that can be ignored without being lost.
5. Merging all of your contacts/calendars from wherever they happen to live.
6. Acceptance that Flash is still a significant web technology.
7. No jailbreaking required – but rather a completely sanctioned developer mode that gives you root access to the device.
8. HTML 5 application development as a first class citizen.
9. C++ application development for gaming, etc.
10. A cloud-based IDE allowing app development in a browser.
11. It is Linux, so skills are transferable.
12. Inductive charging, so you don’t have to plug in cords — just drop it on the charger (OK, technically a hardware feature rather than webOS per se, but still awesome).

I have applications that I wrote myself or with co-workers in both the HP App Catalog and the Apple App Store. I definitely prefer coding in JavaScript as opposed to Objective-C, but the thing that makes my blood boil in comparing the experience is Apple’s provisioning profile system. It’s as if Apple said, “Please poke yourself in the eye at regular intervals so that we can be sure you’ve paid us your developer program fees.” Having developed and submitted for webOS, it becomes obvious how pointless the Apple system is. I wouldn’t go to the trouble for a personal project — but what does Apple care about that?

I haven’t developed for Android, but I like Java, so it could be a good experience. But the Android ecosystem seems like total chaos.

I did do some preliminary investigation into developing a video-centric application for Android, but got stuck when trying to figure out what video decoder support exists for the platform. Sure, there’s something that says H.264 is supported, but at what level and profile? Those are kind of critical to know, but the information just isn’t available. I guess Google just left it up to the device manufacturers to do what they wanted, which means trying to figure it out for every device. Compare that to the webOS information on video formats.

So I find myself in an unfortunate situation, hanging on to an aging phone and not sure what I’d do if I accidentally dropped it on the sidewalk tomorrow. Maybe, like I did the last time Palm imploded (with Palm OS), I’d have to give Windows a try, and hope that it isn’t the soul-less experience it was last time.

A while back, I used WiX to build an installer, which was handed off to IT to deploy a web site. After a fair amount of anguish and frustration, I eventually realized that my installer was essentially a difficult to create and maintain batch file. Half the project was just custom command calls out to appcmd.exe anyway. If you’re so unfortunate as to absolutely have to create an MSI install, WiX isn’t a bad way to go, but it was a layer of massive unnecessary complexity in my case. Instead, I ended up creating a 7zip self-extractor that ran an actual batch file when launched.

One challenge is that I actually had a few installers, and I wanted to share code between them, which I was able to do after discovering the CALL command. Here’s an example:

call lib.cmd CopyFiles "C:\ServiceInstallDir"

This is essentially a function call, where “CopyFiles” is the name of the function, and “lib.cmd” is the file where it is implemented. Arguments follow the name of the function, the destination directory being the single argument in this case.

The lib.cmd file looks something like this:

@echo off
REM Within the "function" the args will be shifted, so 3 here becomes 2, 2 becomes 1, etc.
call :%1 %2 %3 %4 %5 %6 %7 %8 %9
goto :EOF
 
:Heading
REM %1 - Text for heading
echo ------------------------------------------------------
echo ^| %~1
echo ------------------------------------------------------
goto :EOF
 
:CopyFiles
REM %1 - Install directory
call :Heading "Copying files..."
if exist "%~1" rd /s /q "%~1"
md "%~1"
xcopy *.* "%~1" /s 
goto :EOF

Each label in this file acts as a function call. The CALL at the beginning passes control off to the label, including the additional arguments. The function ends by jumping to the special :EOF label (which exits the batch context). Note also that functions can call each other, as where the “CopyFiles” function calls the “Heading” function.

It’s a great way to encapsulate little bits of functionality in a common place.

General rant aside, I also get tired of paying $60 for TurboTax every year, so I cancelled my automatic subscription, determined to try something else this time around. After reading some reviews, I decided to go with TaxACT Online, which looked to be reasonably good and quite a bit less expensive. The nice thing is that you can use the application for free until you’re ready to actually file or print your return.

There were a few times that I felt unsure about what I was doing while doing my return — I had some employee stock purchase program things last year to complicate matters. I eventually made it through, though, itemizing my deductions since I’ve got mortgage interest and lots of charitable donations. My remaining federal tax liability at the end was around $500.

That felt wrong to me. I don’t think I’ve ever owed federal taxes beyond what had already been withheld. State taxes, sure — I frequently owe some there — but owing federal taxes was enough to encourage me to take a run through TurboTax’s estimator. That gave me hope that I might be able to get a more favorable result.

Doing taxes is far from fun, but it was definitely easier with TurboTax Online. I felt more like I was being guided along rather than being presented with a bunch of text to read and some form inputs to fill in. I was also able to import a lot of data electronically, which helped with both speed and accuracy. When I finished my federal return (for the second time), I came out with about a $1500 refund.

Maybe I did something wrong with TaxACT. Maybe I did something wrong with TurboTax, but the refund was more in line with previous years. Certainly for my situation, TurboTax was clearly worth the price of admission.

Before addressing the tedium, though, why does it even matter whether you check for null arguments and throw a ArgumentNullException? You’ll just get a NullReferenceException if you don’t anyway. Is the ArgumentNullException somehow better?

The ArgumentNullException has a couple of benefits: It clearly indicates that the caller is to blame and by including the parameter name quickly narrows down the problem instead of requiring you to figure out which reference within the body of the method is null.

Although an ArgumentNullException is preferable, the null checking can overwhelm a short method. Here’s a contrived example:

public static void PrintThings(string[] things)
{
  if (things == null)
  {
    throw new ArgumentNullException("things");
  }
 
  foreach (var thing in things)
  {
    Console.WriteLine(thing);
  }
}

Half the method is dedicated to checking the parameter for null. Now imagine if a relatively short method had three parameters to check. It would be all null checks and hardly any code.

In the spirit of conciseness, I’ve frequently had some kind of helper function that does the null check:

public static void PrintThings(string[] things)
{
  Arg.NotNull("things", things);
 
  foreach (var thing in things)
  {
    Console.WriteLine(thing);
  }
}

The trouble is, Visual Studio’s static analysis doesn’t recognize this as a parameter validation of “things”, so you’ll get the CA1062 warning.

The work-around is to create a ValidatedNotNullAttribute and include it in the helper method, which signals to static analysis that, trust me, I really am checking the parameter. Here’s a complete example:

using System;
 
namespace CA1062Test
{
    public static class Program
    {
        static void Main()
        {
            PrintThings(new string[] { "one", "two" });
        }
 
        public static void PrintThings(string[] things)
        {
            Arg.NotNull("things", things);
 
            foreach (var thing in things)
            {
                Console.WriteLine(thing);
            }
        }
 
    }
 
    static class Arg
    {
        public static void NotNull(string name, [ValidatedNotNull] object value)
        {
            if (value == null)
            {
                throw new ArgumentNullException(name);
            }
        }
    }
 
    sealed class ValidatedNotNullAttribute : Attribute
    {
    }
}

Notice the trivial ValidatedNotNullAttribute implementation — it doesn’t matter what it does so long as it is named “ValidatedNotNullAttribute”. The attribute is applied to the parameter being checked in the NotNull function, so it only has to be specified once in the helper declaration, and not on each use of the helper.

1. Ubiquity
2. Price (free)

OneNote’s list is shorter:

1. It’s a truly excellent Windows application

The folks at Evernote can’t quite seem to get their act together. I think the Windows application has been through at least two complete rewrites since I’ve been using it, each time with the promise that this version will finally be the solid platform on which fantastic new features will be added. But the editing abilities still compare unfavorably to 1990′s-era HTML editors.

I can expect, though, that I’ll be able to find an Evernote client for pretty much any platform I want, with data synchronized between all of them because of Evernote’s cloud focus. Not all of the clients are created equal — Evernote for the iPad simply refuses to edit notes with “rich” formatting — but they aren’t bad.

OneNote’s anchor to Desktop Windows has been its biggest downfall. The cloud-stored notebooks of OneNote 2010 are a great advancement, but Microsoft needs clients for other platforms — and good clients at that. Even when I had a Windows Mobile phone a few years ago, the OneNote client there was pathetic. I regularly run Evernote clients on Windows, Mac, iOS (iPad) and webOS, and until I can do the same with OneNote, it’s hard to imagine it becoming my primary exocortex. That Microsoft has released an iPhone client is promising. Hopefully they’ll continue to add other platforms.

Using OneNote on Windows is a delight. Formatting works as expected instead of suddenly flaking out like it does in Evernote. Tables are easy to work with and look good. Images can be resized in-line. Screen clips go into the note I’m editing instead of a separate note, which is the behavior I want 95% of the time.

That Evernote is free is nice, but I’ve got 80 bucks sitting right here for OneNote if it can make that final jump to universal access.

Of course, the easy way is to just get the latest from the TFS repository and add all of the files to the new SVN repository, but change history is valuable and I didn’t want to:

1. lose it
2. be required to go back to TFS to get it

I found tfs2svn, a tool written by Kevin Colyar expressly for the purpose of migrating while preserving the change history. If you’re going to run this on a big, mature TFS repository, there are a couple of things you should know:

1. It’s going to take a long time.
2. Stuff is going to go wrong.

This is no disrespect to Kevin — that the tool is available at all is a great help. I also have to confess that I hacked the source code a bit, so there’s a possibility that some of what went wrong was my own fault.

My Scenario

The repository I migrated has about five and a half years of history, comprised of nearly 21,000 changesets. It also had code from several projects, and I really only wanted to migrate a subset of the tree. We already have a Subversion repository with some source in it, so I wanted to get the TFS source into there.

Strategy

My plan was to create a local, temporary Subversion repository into which I could do the conversion. This would let me mess up in various spectacular ways without risk to the real SVN repository that would be the code’s ultimate home. It also seemed like it would be faster than running all of the SVN commands across the network to the Subversion server. Once the migration was done locally, I’d use svnadmin dump to generate a version of the repository that could be imported into the “real” repository with svnadmin load.

I hoped that I’d be able to use tfs2svn to limit the migration to only the parts of the tree that I wanted, but the tool resisted my attempts to do so. I was thinking that I’d run it multiple times, changing the paths to get one relevant top-level  folder at a time. It’s not clear which path to change, though. You’ve got the TFS repository path (which would be a must), the SVN repository path and the SVN working copy path. It turns out that all of those paths are used in different situations by the tool, and I eventually thought it might be best to just migrate the whole thing.

It would be tragic to actually include everything in the final SVN repository, though, given that Subversion doesn’t have a permanent delete feature (that is, “obliterate”). All the stuff I didn’t want to migrate would be forever in the repository. But since I was going to do a dump anyway, I could run dumpfilter to remove the stuff I didn’t want.

The trouble then becomes time. It takes so long to migrate all those TFS changesets into SVN that I very quickly decided I needed to push the filtering forward in the process. This is where the hacking began.

I added a “IncludeChange” check into TfsClientProvider’s ProcessChange method:

private bool ProcessChange(Changeset changeset, Change change)
      {
          if (IncludeChange(change))
          {
              // Process file change.
              if (change.Item.ItemType == ItemType.File)
                  ProcessFileChange(changeset, change);
 
              // Process folder change.
              else if (change.Item.ItemType == ItemType.Folder)
                  ProcessFolderChange(changeset, change);
 
              return true;
          }
          return false;
      }

IncludeChange has a bunch of lines that look like this:

if (change.Item.ServerItem.StartsWith("$/Projects/Main/Applications", StringComparison.OrdinalIgnoreCase)) return false;

This combination skips any changes that were in areas that I didn’t care about. ProcessChange also now returns a boolean to indicate whether anything from the changeset needs to be committed in Subversion, saving additional time by skipping an unneeded operation.

Problems

One of the really good things about tfs2svn is the Start at Changeset# field. When something goes wrong, you can always start back up where you left off — even if you have to exit the program. Sometimes tfs2svn will even offer to fill in the Start at Changeset# field for you when it’s about to report an error.

These are some of the things that went wrong.

At first I was getting an obscure Error resolving case of [some path]. Hard to figure out, but easy to fix: I just needed to take the trailing backslash off the Working copy folder.

I also ran into a problem when a changeset moved some files from a place that I was excluding to a place that I wasn’t excluding. That didn’t turn out so well since the source files didn’t exist, having been skipped by my IncludeChange method. This was fairly early on, so I just decided to start over and not exclude the source path. I might have been able to get a copy of the files and add them to SVN in the appropriate place and retry, though.

Directory deletes in TFS frequently caused trouble in the conversion. They would result in an unresolved conflict on the directory and the commit would fail. In these cases, I just marked the conflict resolved, committed manually and skipped to the next changeset. You lose some of the history in these cases by doing that, though.

Directory adds also caused troubles at times. The tool would try to add a file to a directory that hadn’t been created yet. To fix those, I just created the directory by hand and added it to SVN and retried the problematic changeset.

There was one situation where some file renames weren’t working at all. The original files didn’t exist and the destination files did. I kind of mangled that changeset, but fortunately I knew that those files were obsolete and would all be deleted in a later changeset, so I didn’t care much about them anyway. I think this was probably caused by a situation where part of a changeset had been processed, but then there was an error. Retrying the changeset meant it tried to redo some moves that were already done.

Finally, there were some times when the network would timeout, or Subversion would report that a file or directory was corrupt or unreadable. In those cases, I just tried again and was always successful the next time.

With these errors popping up, tfs2svn required a lot of babysitting to notice the error, make a fix and get it going again. That unfortunately added to the total time span since it would invariably stop when you were busy with something else, or when you hoped to make good progress letting it run overnight.

Finish

Once the migration was done, svnadmin dump was relatively quick and completely painless. Once I could get some of IT’s time (someone with direct access to the Subversion server), the load also was quick and easy.

Update (4 Feb 2011)

I’ve noticed another problem in the migrated history. Sometimes the author of a change seems to be incorrect. According to the history, a developer from another team did some work that would be extremely unlikely if it were true. I haven’t figured this one out yet…

Update (22 Mar 2011)

A possible explanation for the “The file or directory is corrupted and unreadable” errors is a bug in Windows 7. The bug fix is included in Windows 7 SP1, which you can get through Windows Update.