Xenocode Postbuild: Bad UI or Intentional Deception?

I've been looking at application virtualization products recently and I just keep getting more and more. Seriously, I'm just not `getting` this new business model of `Give us thousands of dollars for the tool AND give us a cut ( tens of dollars ) for every single client you deploy your application to. Seriously, there must be some real discounts out there somewhere if they really think and ISVs will spend boat loads of money just to virtualize their package instead of spending fractions of that to hire quality setup developers to create properly behaving MSI's.

But anyways, my real annoyance came when I was playing with Xenocode Postbuild. I downloaded the eval and went to virtualize a simple winforms app. I got a dialog saying I needed to activate ( which my firewall at work said HELL NO to ) but look at the color scheme:



I loaded this screenshot as 24bit uncompressed so you could see exactly what I see. Look at that cyan looking small point text. What the hell is that? I had to flood fill it to be able to read it:



This just seems sleazy to me and reinforces my perception that something just doesn't quite smell right in the virtualization market.

How DTF is going to help me become a better .NET Developer

I've been doing .NET for a couple years now, but I mostly use it for small tools/automation development. Things like the occasional MSBuild or NAnt task here and there, a SOAP web service that encapsulates a REST web service, or perhaps a custom action that consumes a SOAP web service. The only application work I've ever done in C# was a client/server program that configured a existing system. The application used the ADO.NET 2.0 Factory model, had a framework consisting of a base UI and several application services. I did it pretty cleanly but at the end of the day it was just a bunch of look a like modules using DataGrids. Big deal

So here I am playing with DTF and I'm going back to various CAs that I've written over the years and seeing how they could be done better in C# The first story that I decided to look at is dynamically updating ListBoxes at install time.

I started with a simple statement like this:

session.Database.OpenView("INSERT INTO `ListBox` (`Property`, `Order`, `Value`, `Text`) VALUES ('TESTPROP', 1, '1', 'One') TEMPORARY").Execute();


This is problematic because if you get any of the SQL statement wrong everything will blow up. Trust me, trying to do this in VBScript really sucks big time. There is a reason Script CAs are said to be evil and it’s not just because of virus scanners.

So then I noticed that the DTF Database object has a TableCollection of TableInfo objects that each have built in SqlStatements strings. So I tweaked it up a little:

Database db = session.Database;
string sqlInsertTemp = db.Tables["ListBox"].SqlInsertString + " TEMPORARY";
View view = db.OpenView(sqlInsertTemp );
view.Execute( new Record( new object[] { "TESTPROP", 1, "1", "One" } ));
view.Close();

This is a little bit better but it doesn’t solve a more fundamental problem. I’m trying to manage a list of items, why should I have to care about how to get the data stored in MSI? It started to become obvious a bit of abstraction was needed. The result is creating and invoking a custom class that looks more like this:

ListBox listBox = new ListBox(session.Database, "TESTPROP");
foreach (Process proc in Process.GetProcesses())
{
listBox.AddToEnd(new ListBoxItem(proc.Id.ToString(), proc.ProcessName));
}

Now I'm just instantiating a ListBox class and during it's constructor setting up database and property collection association. I've also defined a ListBoxItem class that strongly types the Value and Text fields. Finally I have an AddToEnd method that takes a new listBoxItem.

This class is by no means complete or bug free. I'll probably want to do things implement capabilities like Add, Insert, Move, Remove, Sort, IEnumerable etcetera. Then I'll probably want to go on to other common stories such as making it easier/more generic to build data driven custom actions that change system state. Serializing/Deserializing CustomActionData. Building ICEs. And so on. Basically the plumbing details that many CAs have to do but really aren’t relevant to the problem trying to be solved.

Yes, I know… these are OO principals that I've learned over the years. I can sit down with a developer and read what he's doing and talk about it but frankly I've never taken the time to hone these skills because it was orthoganal to the problem of writing setup declaratively. After all, when your life is breathing MSI it's kind of hard to get worked up over things like NHibernate vs EF debates. But now that C# is my language of choice for custom actions, it's time to catch up on things I should have mastered years ago.

PS- Does anyone have any suggestions on good books to read? Please, nothing too heavy... go easy on me.

Red Vs Blue Death Match

For some reason Rob Mensching has decided once again to bring up the subject of Red vs Blue. I'm a little confused because I thought he disliked troll posts.


Anyways, he says he's `joking` so I figured I'd share a little bit of my humor also. Now what was AJ saying about taking it out to the playground?





Seriously though, Rob and I have a lot of common friends. I've spoken to them and the general feeling is that I'm being too hard on Rob (misdirecting my frustration), that Rob was out of line for his comments and that it's disappointing that we aren't getting along.

Rob has claimed on his blog that he is leaving the lines of communication open. Believe me... I've tried. However, he doesn't seem to be serious as it takes him a week to answer an email. Oh well, it took Sinan 1 1/2 years to get his apology, why should I expect one any sooner?


PS- Rob says he doesn't read my blog anymore ( actually he sent me an email to that effect a year ago ) but my statcounter tells me otherwise. Anyways, I'm just kidding here also. One day the two of us are going to be slamming some beers and this will be really funny in an alpha geek sorta way.

More Information on the .NET 3.5 Client Profile

I noticed a link over on the WiX-Users list for even more information on the .NET 3.5 Client Profile. I've not had a chance to absorb it all but I wanted to share the link ASAP.

Edit: I've absorbed a bit and I'm a little less impressed with this servicing mechanism. It seems that this is really aimed at XP SP2 with no framework. XP machines ( and all Vista machines ) with the framework can't go the client route but must instead get the full install.

Thinking back to my originial problems deploying .NET 3.5 applications, I'm not sure if this will really help us or not.

Also I noticed this little quote from the MS blog

"As is illustrated in the preceding table, the Client Profile deployment is focused on Windows XP SP2+ with No Framework components installed. This platform configuration is still the predominant platform configuration and will continue be so for the foreseeable future."

Ouch. I'm sure the SlashDot and Apple fans would love that quote.

Cancer Update

I haven't spoken about my wife's battle with cancer in a while but I just read an article about Senator Kennedy being diagnosed with a brain tumor that brought it back to my mind.

Cheryl had a seizure at about 2am one night back in October. I guess I'd blocked out how scary that day was until I read the article today. It was absolutely horrifying to wake up to her having a convulsion and then spend hours and hours in the emergency room as they did countless tests to rule out every possible reason. Believe me, some of the things they suspected were truly bizarre. It was almost like watching house only it isn't funny because it's YOUR wife on the table.

I'll never forget how relieved we were when finally one of the many doctors came to her room, picked up her hand and said `You don't have brain cancer.`

Anyways, 7 months have passed since that night and I'm currently tearing up as I write this. Cheryl has chemotherapy (yet again) tomorrow and only one more treatment scheduled a couple weeks later.

After that it's a matter of wait and see. Only we don't plan on doing much waiting as we are currently busy planning a few summer getaways.

LINK: Annoying software: a rogues' gallery

I just came across this article on ZDNet. When I read the title I just knew that most of the complaints would be about the installs. Sure enough, as I clicked through through the first 6 annoying applications the complaints were aimed squarely at installs that treat customers computers like it's ISV owned equipment.

It's a really good read and it's a good reminder why auto update patterns, unwanted bundled software, annoying reboots and ordering customers to widen the attack surface of their system just to use these applications is NOT Setup Goodness.

Unfortunately we are going to see even more of this garbage going forward as it's sadly bound to be getting easier for ISVs to hoist crap like Yahoo! Toolbar on users machines.

Data Driven Managed Custom Actions made easy with DTF

One of the shortcomings of Windows Installer is no built in ability to perform math functions. Fortunately this is made easy in C# thanks to the AxLibrary project from Ron Ayoub found on The Code Project. Now with DTF's ability to easily and reliably write Manged Custom Actions creating a simple math evaluation pattern is easy. Consider this database snippet in WiX:

(Note: I originally wrote the table using InstallShield but decided to re implement it in WiX since it's easier to read without visualization. But it sure is more difficult to author. In InstallShield it only took a few clicks and fill in the form typing in the schema wizard. )

The goal is to query the Math table and process math statements with conditions that evaluate to true. Using C#, DTF and AXLibrary it's as simple as this block of code:

The result is a log file like this:

Wow, is that easy or what? I'd love it if someone would write the same functionality in C++ and show what it would look like. BTW, even though DTF comes from WiX, this CA was actually executed via InstallShield. After all, it is just a Type 1 exported stdcall function.

.NET 3.5 SP1 Client Profile Information

I wanted to point out an excellent blog post talking about a new .NET Framework 3.5 `Client Profile` installation that weighs in at around 25mb and had the ability to update itself to the full framework via Windows Update.

This is obviously a HUGE improvement over the previous options. Combined with various developer websites that picked up my articles about the massive size of the framework it is comforting to know that I'm not alone and losing my mind in thinking that it was just too friggin big.

In the article there is also a mention that Microsoft is working with InstallShield to provide support for the client profile so I only have one question: When will we get to see a beta prereq?

Deployment Tools Foundation (DTF) Managed Custom Actions

Background

Note: For the purpose of brevity, this blog post is going to assume that the reader already has a strong understanding of the Windows Installer architecture and philosophy. I will not attempt to fully cover the declarative and transactional design goals of MSI or the sordid details of the pro’s and con’s of different custom action types. If you do not have this knowledge, I highly advise that you obtain it prior to writing custom actions for Windows Installer packages.

Despite Windows Installer rich feature set of Standard Actions, there often comes a time when a custom action is required to accomplish the deployment goals for an application. Windows Installer originally wanted to be 100% declarative, but it came to be understood that this was not achievable. Thus the Windows Installer team created 3 primary mechanisms for developers to inject code into the installation process: Win32 DLL, ActiveScript (VBS/JS) and EXE. As the years went by, there came to be an understanding that the Script and EXE hosting models had some serious design limitations and points of failure. These CA types were then largely discouraged. The sole remaining technique of C++ Win32 exported DLL functions made perfect sense back in the mid 1990’s when MSI was created but with today’s generation of developers and platform capabilities exposed solely in .NET base class libraries, it simply no longer could meet application deployment needs in the 2000’s. At first the community struggled with this because there was still a strong desire to return back to a 100% declarative design and there was some technical problems with consuming managed code from within the Windows Installer native unmanaged engine.

Deployment Tools Foundation: A Strong Solution

First made available to the public in WiX weekly release 3.0.4116.0 (http://wix.sourceforge.net/releases/3.0.4116.0/ ), DTF provides a framework for easily and reliably writing managed code custom actions for the Windows Installer. In a nutshell, it provides a robust set of interop classes to simplify communicating with MSI and a hosting model to abstract the CLR code from the MSI process. At runtime, MSI thinks it’s calling a Win32 DLL in it’s own sandbox but in reality the CLR is being fired up out of process and communicated with through a named pipe. From your managed codes perspective, you are simply communicating with MSI through the interop classes with no need (generally) to be concerned with all of the nightmares of unmanaged code.

Simple Example:

Consider this code snippet that assigns a random number to a public property.

Notice the Microsoft.Deployment.WindowsInstaller reference, the [CustomAction] attribute and the Session class. Let's build the assembly and look at it in Dependency Walker:



There's a couple problems. The first is that there aren't any exported functions that MSI could understand, the second is the reference to MSCOREE.dll reference that would cause MSI to be stuck on a particular CLR version if it had been ran. We can fix this though by running a postbuild step with a DTF utility called MakeSfxCA.exe. This will package will parse the assembly for methods with the [CustomAction] attribute, package it and the dependencies into a near self extracting wrapper for consumption by MSI. Let's look at it again in Depends:



Notice the exported stdcall function now exists, there is a dependency on MSI and MSCOREE is no longer required (in this context).

Debugging

DTF supports two ways of debugging your managed code custom action. One is to attach a debugger to the process via a MessageBox and the later is a new environment variable called MMsiBreak ( not to be confused with MsiBreak ). I tried it and it worked the first time.

Things To Come

In DTF SDK has an expiremental namespace that provides LINQ capabilities to Custom Actions. This promises to greatly simplfy the process of querying the MSI database and should encourage developers in creating data driven CA's. For example, in my simple example I merely assign a random number to a single property. An improved custom action would have a RandomNumbers table with n number of rows specifying properties to be assigned a random number and perhaps a Condition column for evaluating if the assignment should occur.

Summary

If you don't mind adding the .NET framework as a dependency to your install and you want to write custom actions in managed code, DTF rocks. I really don't know any other way to put it. There were some bugs that I encountered ( mostly release defects with missing files and mismatches of strong name keys ) but nothing that I couldn't easily overcome to create this sample project. The resulting custom action can then be consumed in virtually any MSI editing tool. This is clearly a best in class solution for managed code custom actions that is long overdo.

On the other hand, if you still aren't sold on .NET and also don't want to resort to C++, then my best of class reccomendation for unmanaged code custom actions continues to be InstallScript from Acresso.

The Price of Ideology and a Great New Hope

This week, Jason Gincherau (MSFT) released a new SDK called Deployment Tools Foundation (DTF) that I can only term the New Holy Grail of Managed Custom Actions. It implements just about every single concept that I've tossed around over the last 5 years but that I couldn't bring to fruitition because I simply didn't have the prerequisite C++/.NET skills. DTF is truly amazing and in my next blog article I'm going to do a walk through of what it is and how to use it.

But first, I've got to get something off my chest: I am extremely frustrated with members of the setup community at Microsoft on how this SDK came to finally roam the wild. For years, there has been a debate on the roll of custom actions in setup. Certain people had formed very strong ideological positions that basically make the case that if you wrote a custom action, you had generally failed in application and declarative setup design. In the few cases your decision was somehow justified, if you wrote the CA in any other technology then a C++ style Type 1 CA, you had committed an evil act.

In my professional opinion, that believe has cost us, the worldwide setup community, at least 1 year of delay in getting managed custom action goodness... and quite possibly many more years.

If you read Rob Mensching's post here, Jason Gincherau's post here and the DTF.CHM in WiX, you can put together a 5 year time line of the motivations and accomplishments of DTF. You can put together that DTF was properly handling the technical CLR issues about 1 year ago, just one month after Rob posted his `no support and here's why` blog. As an aside, in the blog I left a very simple comment:

For the rest of Microsoft, managed code is clearly marketed as the future of software development. Windows Installer should adopt this strategic initiative and simply overcome the technical hurdles. This way we can have really good custom action patterns to play with until the day comes that MSI natively supports the patterns that we are writing custom actions for. Then we can gladly dump our CA's and refactor using the built-in standard actions.

While it took the obviously brilliant Jason a mere month to overcome the technical hurdles, it took the setup community at Microsoft an additional year ( according to their own writings ) to
overcome their zealous belief that consumers of the MSI SDK simply couldn't be trusted to write custom actions. As recently as 2 weeks ago, Rob wrote in the WiX Users list:

Every suggested implementation that I've seen on the internet that describes how to incorporate managed code custom actions into an MSI package does not address that problematic interaction.

Notice that he cleverly wordsmithed the words `on the internet`. Even though he knew DTF had been in development since 2003 and that the CLR issues were resolved, he didn't bother to enlighten the community of this fact. In fact, one of his many fan boys ( who happens to be an MSI MVP ) parroted the party line:

There is a reason that the experts are all unanimously saying that managed
custom actions are a bad idea. You are free to ignore all that accumulated
expertise but it doesn't seem like a wise thing to do.

And here we are in May of 2008. After years of a collective effort to silence the opinion of many, many setup experts who requested managed code custom actions, and years of me beating the drum to not ignore the 800lb gorrila that is .NET, suddenly Rob has switched his opinion and released DTF as a glorious addition to WiX goodness. Clap, Clap, Clap.

I'm sorry, but I have a problem with this. Now Rob can just simply attempt to dismiss my opinion by personally attacking me as a troll... however, I feel that I've made some important and accurate points in this article and that Rob has some serious soul searching and explaining to do. Then again, I probably give entirely too much power to Rob in even caring about his opinion anymore.

Finally, I am extremely happy that Jason has developed and released DTF. I am also very relieved that the Managed Custom action debate is finally over. Considering Jason's role in integrating WiX into Visual Studio, I have a great new hope that progress can overcome ideology.

100% Frustrated

A very interesting thread is going on over at the WiX-Users list: `yep - back to being 100% frustrated` It's a very good read.

Basically it started with Chris Mumford complaining that MSI sucks and that WiX doesn't make it any easier. At first there was just your typical rude fan boy comment ( yes Rob, WiX fan-boys DO exist and not everyone who disagrees with you is a troll or a newbie ):

well.. the local supermarket here .. is looking for a Security Guard???
if u want i can help pass ya resume...

Then follows a back and forth an interesting discussion of the pains of Windows Installer and WiX.

[More] Problems with MS VC++ 8 SP1 Merge Modules

[updated 5-16-08, see end of blog]
I've had to explain this problem in detail to Microsoft Tech Support so I thought I'd go ahead and blog the details in case anyone else is using the MS VC++ 8 SP1 merge modules in their MSI installation and are unable to reference the assemblies at runtime after updating their product using an MSI major upgrade on a Vista [pre-SP1] system.

Now, in fairness to the Visual Studio team, this problem is most likely an issue with MSI 4.0. I say 'most likely' because I don't know that for sure yet - MS tech support is still working with the MSI team on my case (# SRZ080507000538). However, I feel it's an educated guess based on the fact that installing Vista SP1 fixes the issue and that Vista SP1 contains fixes to MSI 4.0 [nice of the MSI team to slip undocumented fixes to MSI 4.0 into Vista SP1 - see the MSI team blog for a list of changes].

So, we're in the final stages of testing an update to our application (I'll call it OurApp SP1 and you should note that it is an MSI major upgrade with the Upgrade table entry set to uninstall the previous [FCS] install) when QA discovers that running OurApp SP1 on a Vista [pre-SP1] system where OurApp FCS is already installed results in the Win32 assemblies from the MS VC++ 8 SP1 merge modules (Microsoft_VC80_CRT_x86.msm and policy_8_0_Microsoft_VC80_CRT_x86.msm) not being published globally on the Vista system. This was causing one of our java apps to fail because it couldn't load msvcr80.dll when it started [and I know I can install the msvc*80.dll binaries as side-by-side assemblies to the same folder and have the app work - but that's not the point of this blog]. Oddly enough, running an installation repair immediately after the upgrade install completes does publish the assemblies globally. Furthermore, running the OurApp SP1 on a Vista [pre-SP1] system that does not already have OurApp FCS installed results in the assemblies being properly published. To make it even more interesting, installing Vista SP1 on the system before running the OurApp SP1 update results in the assemblies being properly published.

Now, I'm not a novice. I did a thorough comparison of MSI logs from all three scenarios described above and, except for the expected time stamps and property values related to the different system configurations, the log files are exactly the same.

BTW, I opened the MS support incident because I need to know whether or not the fixes to MSI 4.0 that were snuck into Vista SP1 can be applied independently of Vista SP1 (via a hotfix or redistributable) as it doesn't make sense for us to document that our customers should upgrade to Vista SP1 prior to applying our update. However, based on the fact that previous MSI updates have only been made available as redistributables, I'm not going to hold my breath. But I will update the blog once MS figures out the answer.

[UPDATE 5-16-08]
Evidently, this is caused by a known issue with the latest version of msvcr80.dll made available via merge modules (8.0.50727.762), see http://support.microsoft.com/?kbid=905238. However, MS isn't providing a means to incorporate the updated msvcr80.dll (8.0.50727.1434) into your installation via merge modules so the only way to get the update is to have your customer apply either the .NET Framework 2.0 SP1 or .NET Framework 3.0 hotfix (or Vista SP1, which applies the hotfix). The other way to resolve the problem, as mentioned in the KB article, is to move execution of the RemoveExistingProducts standard action so that it occurs after InstallFinalize.

I think it's a pretty lousy decision on the part of Microsoft to not update their damn merge modules. I didn't find this issue until the tail end of the QA process so I can't change the location of RemoveExistingProducts without potentially introducing additional issues and requiring my customers to apply the .NET Framework update when my application doesn't use the .NET Framework is ridiculous. I should be able to incorporate the updated files directly into my application since MS provided a means for me to incorporate the bad files in the first place!

Goodness or Badness?

I've noticed a disturbing trend that I've also been guilty of and I'd like my readers opinions.

As we all know, MSI databases are an open format. It's very easy to inspect them and transform them to fit the consumers needs. This is a good thing and yet it's a bad thing.

It seems that various Windows Installer Experts/Bloggers seem to think it's within their right to stand up on their soapbox, proclaim to be the all knowing expert of "Setup Goodness(TM)" and then proceed to mercilessly judge the authors of packages who don't meet their exacting standards. Typically the packages being reviewed are from companies that are competitors of Microsoft.


Granted, their are valid technical points, but I believe the message comes across in a very arrogant, vicious manner. So I'd like my readers opinion. Below are a few of examples for you to judge yourself. Afterwords, head over and vote on my new poll.



VirtualBox 1.6.0 setup another example of the second law of thermodynamics
Google Earth setup experience Google App Engine delivered to Windows by WiX.
Google Toolbar Beta for Enterprise a "Trojan horse" MSI package
ComponentID GUID Sloppiness Observation

Welcome Back SeBackupPrivilege

Back in October 2006 I reported about an undocumented security change in MSI 4.0/Vista that prevented Deferred NoImpersonate CA's from having the same security rights as they had previously had on downstream operating systems. The story was picked up by Microsoft's Vista Compatibility blog and justified as an attempt to tighten OS security. Maarten van de Bospoort of the MSFT AppCompat team said that it was `design decision made by the installer folks.`

The argument was naturally pointless since any elevated process ( including MSI CA's themselves or bootstrappers ) could easily tweak the registry and restart the MSI service to get around the restriction. As an aside, my discovery was also cited by Microsoft MVP Stefen Kruger.

Tonight I read on the Windows Installer team blog that after 1 1/2 years, Microsoft is finally correcting this issue an restoring SeBackupPrivilege.