A new decade means time for a fresh windows install at work.  I ran into some trouble with windows 7, visual studio 2008, and SQL 2008 Express.  Here’s how I resolved them.  Contrary to most things I found on the web, I’m not using betas or release candidates.

First off, installing SQL 2008 Express.  I only wanted the management tools, and this was a little hard to come by.  I downloaded various EXE files from MSDN, but none of them worked (they would error out, bring up an seemingly unrelated installer, or any other confusing behavior that may have led you here).  Here’s what worked for me:

1. Be sure any previous installation attempts have been purged via Add / Remove Programs
2. Go to the “other install options” page for SQL express: http://www.microsoft.com/express/Database/default.aspx#Installation_Options
3. Click the “Management Tools” install button (for me that’s: http://www.microsoft.com/web/gallery/install.aspx?appsxml=www.microsoft.com%2Fweb%2Fwebpi%2F2.0%2FWebProductList.xml%3Bwww.microsoft.com%2Fweb%2Fwebpi%2F2.0%2FWebProductList.xml&appid=134%3B135)
4. Install the “Microsoft Web Platform Installer” (MWPI) if it asks you to
5. Should be straightforward from here on

The funny thing here is the MWPI seems to download an installer that looks a lot like the one at Microsoft® SQL Server® 2008 Management Studio Express that didn’t work for me.

Next up, Visual Studio 2008 (VS2008).  My company has an MSDN subscription, so we downloaded an ISO (named en_visual_studio_2008_professional_x86_x64wow_dvd_X14-26326.iso) and I used freeware MagicISO to mount it, then ran “setup.exe”.  The install failed on the “Microsoft Visual Studio Web Authoring Component” (MVSWAC).  Here’s what worked for me:

1. IF YOU WANT SQL2008, DO THAT FIRST
2. Be sure any previous installation attempts have been purged via Add / Remove Programs
3. Download WebDesignerCore.EXE from microsoft
4. Run it
5. Install VS2008 from disc/iso as normal.

Digging into the ISO using 7zip, the problem is /WCU/WebDesignerCore/WebDesignerCore.EXE is corrupt.  To get VS2008 to install cleanly, first we need to install MVSWAC, at which point the VS2008 installer will happily skip past the corrupt file.  I ran across several blog/forum posts with horror stories about VS2008 installing SQL2005, and needing to uninstall half the planet to get things working right.

As always, be sure to hit up windows update, and change your update settings so you get fixes for VS2008 and SQL2008.

I’ve been doing some maintenance programming for a few days solid (rare for me to get to program that much), and I again find myself amazed that any software works at all.  I’ve only been programming seriously for about a decade (mostly web apps), but it feels like I’m building rickety crap on top of other people’s horrible hacks.

The bar for quality software seems so abysmally low.  When coding around some bizarre behavior I’m seeing out of the .NET framework, I know I’m introducing weird brittle bits.  It feels wrong, but I don’t see any other option.  And this is new code, written for the latest released version of a very popular system!  It seems like everyone else is doing the same thing in every programming environment I’ve seen.

My best guess is I’m working at maybe the 1000th layer of abstraction over the bare metal, and that sounds low.  That’s a lot of cruft, hacks, bugs, security holes, late-night fixes, bad compromises and coffee.

Maybe my sense of “clean code” is just OCD?  Sometimes I wonder if writing good code is just a waste of time.  Is shoddy copy/paste winning the evolutionary battle for the software base that will drive humanity for the next millennium?

I have a client who acts as a Shibboleth Service Provider (SP), and the corresponding Identity Provider (IdP) needed to update some of their information, so I had to spend a few hours debugging shibboleth again this morning.

The punchline: in the metadata for an IdP, there are TWO places you need to specify the scope of the IdP, once in the <IDPSSODescriptor> to cover authentication, and once later in the <AttributeAuthorityDescriptor> to cover any attributes.

Shibboleth is all about trust, and the lack thereof. The SP and IdP share a few keys, and then each maintain their own configuration files specifying how much information to trust. When a user logs in, they get bounced from SP->IdP to authenticate, then again from IdP -> SP with an auth token. From there, the SP can assume an authenticated user. If the SP wants any more information about the user (say, a username), it can make SOAP calls to the IdP requesting more attributes. The IdP checks it’s config files to see how many attributes to release to that particular SP, and sends back the attributes. The SP doesn’t trust the IdP on it’s word, so the SP checks it’s config files to validate that it can accept these attributes from this IdP. This is where I ran into a problem, getting an error like:

attribute (username) scope (foo.bar.edu) not accepted

This brings up another aspected of attributes, the scope.  An attribute’s scope seems somewhat akin to a namespace to me, it’s a way of grouping related attributes and prevent name conflicts between attributes.  In my case, my IdP was “bar.edu”, and my SP was configured to accept the username attribute from any site with any value (in my AAP.xml).  However, the IdP was returning the username scoped under a different domain, so the SP, being a paranoid creature, assumes the IdP is trying to falsify information and refused the attribute.  To convince the SP to accept the different scope, you have to change the metadata for the IdP, and specify that it is allowed to provide attributes in the new scope.  Unfortunately for me, there are 2 kinds of scopes you can change:

1. Scope of the initial authentication
2. Scope of the attributes

There are both children of <EntityDescriptor>.  This led to a frustrating morning of restarting IIS (because you need to restart everything under the sun for the configuration to get reloaded), and not seeing any change.  There are many ways to alter your attribute policy, and along to way to discovering the second place to specify scope I ran across another error message:

attribute (username) value (my-user) could not be validated by policy, rejecting it

This seemed to happen when my SP config tried to treat the username as an unscoped attribute.  You can’t just ignore the scope of an attribute if the IdP is sending one.

The end solution is to add another <shibmeta:Scope> specification to the attributes configuration section in the IdP’s metadata:

  <AttributeAuthorityDescriptor protocolSupportEnumeration="urn:oasis:names:tc:SAML:1.1:protocol">
    <Extensions>
      <shibmeta:Scope xmlns:shibmeta="urn:mace:shibboleth:metadata:1.0" 
                      regexp="false">bar.edu</shibmeta:Scope>
      <shibmeta:Scope xmlns:shibmeta="urn:mace:shibboleth:metadata:1.0" 
                      regexp="false">foo.bar.edu</shibmeta:Scope>
    </Extensions>
    ...

Problem: .NET framework classes don’t use interfaces enough

Specific example: DataSource / DataBind()are separately defined on Repeater and GridView, (and Control, and many, many others), and my abstract base class doesn’t care which option an implementor chooses, it just wants to bind the data however the base control wants it.

Possible Solutions:

• Define interface IDataBinding, class MyRepeater : Repeater, IDataBinding, class MyGridView : GridView, IDataBinding, etc.
• Define interface IDataBinding, upgrade to C# 3, use extension methods to add IDataBinding nope, extension methods can’t do this
• Copy/paste identical code from my base class into concrete classes
• Use reflection to set DataSource and call DataBind(), completely sidestepping the C# type system

---

Problem: .NET framework classes not designed for extension: “cannot override inherited member 'System.Collections.Generic.Dictionary<X, Y>.Add(X, Y)' because it is not marked virtual, abstract, or override”

Specific Example: Extending System.Collections.Generic.Dictionary to do some permit a list of values for one key, so adding the first item is stored as a single value, but adding a second value to the same key stores both values in a list. Error message is: “cannot override inherited member 'System.Collections.Generic.Dictionary<X, Y>.Add(X, Y)' because it is not marked virtual, abstract, or override”

Possible Solutions:

• Define wrapper classes that encapsulates the framework class and implements all the base interfaces, with the vast majority of the code being straight delegation to the framework class:

  class MyDict : IDictionary, [other interfaces...] {
  private Dictionary<X,Y> dict = new Dictionary<X,Y>;
  public bool Contains(X key){ return dict.Contains(key);}
  [... other simple wrappers...]
  }

• Find a less appropriate framework class that is designed for extension, duplicate behavior of the proper framework class manually, eg: add run-time type checks for Hashtable.Add(object, object) in place of the letting the compiler handle the types as in Dictionary<X,Y>.Add(X, Y)
• Extend Dictionary<X,Y>, define a method AddList(X,Y), and avoid using IDictionary in the rest of my code
• Upgrade the C# 3, use extension to add function AddList to IDictionary, be sure to include those extensions on every consumer of IDictionary

---

Problem: .NET framwork classes not designed for extension: members declared private/ internal / protected internal

Specific Example: Storing additional data in ViewState on 2 different controls that have different base classes. One is a user control, another extends RadioButtonList to provide different UI for the same data

Possible solutions:

• Upgrade to C# 3, use extension methods to add functions nope, extension methods can only see public members, ViewState is protected
• Use reflection to set ViewState
• Copy/paste code into each control
• Define interface IPublicViewstate, class MyRadioButtonList : RadioButtonList, IPublicViewState, class MyUserControl : UserControl, IPublicViewState, etc

---

As Nathan said, I’m stuck between a rock and IHardPlace.

An update trigger using COLUMNS_UPDATED() was failing, and I figured out the problem:

Some simple SQL:

SELECT CAST(0×0200 as bigint), 0×0200 | 0
SELECT CAST(0×02000 as bigint), 0×02000 | 0
SELECT CAST(0×020000 as bigint), 0×020000 | 0
SELECT CAST(0×0200000 as bigint), 0×0200000 | 0
SELECT CAST(0×02000000 as bigint), 0×02000000 | 0
SELECT CAST(0×020000000 as bigint), 0×020000000 | 0
SELECT CAST(0×0200000000 as bigint), 0×0200000000 | 0
I had “IF COLUMNS_UPDATED() | 0 > 0″ in my trigger, and on large values of COLUMNS_UPDATED(), bitwise ORing with zero equals zero.  Of course, “IF COLUMNS_UPDATED() | 0 > 0″ is a retarded way to say “IF COLUMNS_UPDATED() > 0″, but whatever codepath that large number triggers wigs out there to.  Try these out:

SELECT 1 WHERE 0×0200000000 > 0
SELECT 1 WHERE CAST(0×0200000000 as bigint) > 0

So, to fix my trigger, I have “IF CAST(COLUMNS_UPDATED() as bigint) > 0″, which still seems a little retarded, but at least my trigger works.

I use quicken to manage my finances, and find it overall does an excellent job, but every now and then it does something outrageous:

Seriously?   You still can’t handle that?  I guess that means its time to take another stab at gnucash.

Spent too much damn time debugging the database layer in rails again today. This was the error:

OLE error code:80004005 in Microsoft OLE DB Provider for SQL Server
Cannot create new connection because in manual or distributed transaction mode.

This was getting thrown after an insert, and the problem was very non-obvious.  I found one potential solution on Occasionally Useful Software‘s post about Ruby and SQL Server, but that seemed a little heavy-handed.

Somewhere in the bowels of ActiveRecord, dbi.rb, ADO.rb, and the SQLNCLI driver, something expected one result-set per SQL command. My insert was firing a database trigger, and so the ruby stack saw two result-sets, one saying “1 row affected” for the original insert, one saying “0 rows affected” for the trigger. I’m still a little confused, as I thought those “rows affected” messages were separate from proper result-sets (the results from a SELECT, for example), but apparently those little buggers count enough.

Adding a “SET NOCOUNT ON” to the top of the trigger fixed it.

Mucking about with Shibboleth again, and ran into some errors on the SP, specifically:

2007-07-10 19:49:42 DEBUG SAML.libcurl [79] sessionGet: SSL read: error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate, errno 0

This is the “catch-all” error message for Shibboleth OpenSSL Errors. After much research and testing, it was a problem with the IdP, their server was rejecting my client certificate.

How it should work (I think)

1. User requests protected content on the SP
2. SP redirects to the IdP for authentication
3. IdP authenticates user, sends an SSL (with client certificae) SOAP request to the SP with some info, and then redirects the user back
4. SP validates SOAP request by comparing client certifacte with a white-list in the shibboleth metadata
5. SP sends an SSL (with client certificate) SOAP request to IdP to get more information about the user (in my case, a username so I can identify them in my database)
6. IdP validates SSL cert of SP with a white-list in their shibboleth metadata, responds with whatever information was requested
7. SP uses that information to serve or deny access to the user from step 1

SSL cert problems can happen at steps 4 and 6. You can test these somewhat by using openssl on the command line. Openssl has about a brazillion options, but the one useful here is s_client(1).

Be sure the SP recognizes the IdP’s certs

First up, check that the SP has the IdP’s certs in order:

openssl s_client -connect HOST:443 -showcerts

That will give you back the certificate chain:

Certificate chain
0 s: SUBJECT FOR THE IDP
i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)05/CN=VeriSign Class 3 Secure Server CA
—–BEGIN CERTIFICATE—–
ASCII JUMBLE
—–END CERTIFICATE—–
1 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)05/CN=VeriSign Class 3 Secure Server CA
i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
—–BEGIN CERTIFICATE—–
ASCII JUMBLE
—–END CERTIFICATE—–
—
Server certificate
subject=SUBJECT FOR THE IDP
issuer=/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)05/CN=VeriSign Class 3 Secure Server CA
—
some other stuff…

The issuing certificate (in the example, cert 1) should be in your shibboleth metadata. All you need is the top-most issuer, and shibboleth will look down the certificat chain until it finds a certificate it trusts, and all will be well.

If that doesn’t solve it, then we see if the IdP has the SP cert straight.

Be sure the IdP recognizes the SP’s certs

This is harder to debug from the SP’s side, but you can try make an SSL connection using the SP cert and key:

openssl s_client -connect HOST:443 -showcerts -cert SP.crt -key SP.key

If that connects, then you know the server isn’t rejecting you outright, but it’s possible that Apache config is rejecting you elsewhere. The best way to check is to just ask your IdP if your config is still in their shibboleth white-list.

If that doesn’t solve it, check the apache config.

Be sure everyone’s apache config is OK

The possible failure here is that the certificate is being rejected before making it to shibboleth. You don’t want to tell your whole server to accept any client certificate, you want to just pass those through to shibboleth and let shibboleth decide. You’ll want to have SSLVerifyClient optional_no_ca in your Apache location blocks for shibboleth URLs.

Conclusion

Certificates are a good idea, but a pain in the ass. The shibboleth-users mailing list is a good source of information, and you can get prompt replies from there. And next time I have this problem, I’ll know where to look for debugging tips.

I was recently implementing some custom events, and found a couple of good (if old) articles describing how to do this efficiently using EventHandlerList:

• Manage many events easily with EventHandlerList
• Objects with dense events, but sparse usage can benefit from custom event storage.

Those articles go into why it’s nicer to deal with one EventHandlerList instead of many seperate EventHandlers, so read those for more information. For the lazy, here’s some code showing how you’re supposed to use these things:
[csharp]
public class MyClass {
private EventHandlerList Events = new EventHandlerList();

public event EventHandler MyEvent {
add { Events.AddHandler(“MyEvent”, value); }
remove { Events.RemoveHandler(“MyEvent”, value); }
}

public event EventHandler MyOtherEvent {
add { Events.AddHandler(“MyOtherEvent”, value); }
remove { Events.RemoveHandler(“MyOtherEvent”, value); }
}

protected void OnMyEvent(object sender, EventArgs e) {
EventHandler handler = (EventHandler) Events["MyEvent"];
if (handler != null) {
handler(sender, e);
}
}

protected void OnMyOtherEvent(object sender, EventArgs e) {
EventHandler handler = (EventHandler) Events["MyOtherEvent"];
if (handler != null) {
handler(sender, e);
}
}
}
[/csharp]

Pretty straightforward stuff. When you add an event handler to the list, you associate it with a key, and then when its time to trigger the events, you look for any handlers under the same key. The other day I was putting together something similar, and ran into some unexpected behavior with the keys. I had started by refactoring the magic strings into an enum:
[csharp]
protected enum MyEvents {
MyEvent,
MyOtherEvent
}
[/csharp]
and replaced all the strings with members of that enum. I figured this would work just fine, but the change caused my unit test to fail. Upon debugging, the EventHandlerList was always returning null in my On*Event calls. After some more testing, the pattern became apparent: value types don’t work as keys. This was somewhat unexpected, as I’ve used enums like this in Hashtables all over the place before. After doing a little Reflectoring, the actual search for the key comes down to traversing a linked list with a simple equality test, something like this:
[csharp]
while (head != null)
{
if (head.key == key)
{
return head;
}
head = head.next;
}
[/csharp]
The culprit ends up being C#’s auto-boxing. The key is stored as an object, so my value types are being boxed on the way in, and therefore == is comparing object identity, not the object values. If EventHandlerList used head.key.Equals(key), everything would have worked how I expected. The solution to rid myself of magic strings now becomes using static objects as my keys, so the object identities will match:
[csharp]
private static readonly object MyEventKey = new object();
private static readonly object MyOtherEventKey = new object();
[/csharp]
That pattern reminds me a lot of enums in Java before it got a enum keyword, which came on the heels of C#’s nice solution to the enumerated type problem. It’d be nice if I could use enums for their intended purpose, but cases like this make me a bit wary. Where else in the .NET framework am I going to find object identity equality where I expect to find object value equality? Is there some rational explanation for this, or is this just a bug?

I saw an announcement today that CodePlex, Microsoft’s version of Sourceforge, has released a source control client. From the release:

A common theme we’ve heard from our users is the desire to be able to work offline (in the “edit-merge-commit” style) when working on their CodePlex projects. Six months ago, we started working to write such a client that would integrate with our existing TFS server infrastructure, and today we’ve released our first beta of the client.

The CodePlex Client is a command line client for Windows, and requires .NET 2.0.

This infuriates me. This cool thing they spent six months (six!) writing is called Subversion, and it had a 1.0.0 release three years ago. Subversion had its first beta in late 2003, so the Codeplex folks are waaay behind the state of the art on this one.

As a whole, I think the state of software is abysmal. The only way to make it better is to stop writing new code. New code is always full of bugs, and its an expensive path to get from blank screen to stable program. We need to treat programming more like math, we need to build on our results. Development tools is a special market, as our needs are all very similar, and when we need a tool, we have the skills to make it.

The Codeplex staff stated they needed to write their own client in order to integrate with the TFS server infrastructure. According to an msdn article (Get All Your Devs In A Row With Visual Studio 2005 Team System), TFS seems to be a complicated tool to help manage your developers. Reading the description, TFS seems to be an issue tracker, unit tester, continuous integration, source control system, and visual studio plugin. So, basically a combination of Trac, NUnit, CruiseControl.NET, Subversion, and a visual studio plugin. Why not just write the visual studio plugin, and hook into the tools people are already using? All those tools have rich plugin-architectures that would probably support any sensible addition you’d want to make.

This problem is ingrained at Microsoft, which feels the need to brand everything, but it is in no way limited to them. A search on Sourceforge for “issue tracker” gives 585 results. Sifting through those to pick a winner is difficult.

It’s more fun to write new code than read old code, but this fun wears off. After a certain initial momentum creating your new tool, you will inevitably come to a realization “this is going to take me for-fucking-ever”. Unless your itch is particularly strong, you’ll probably quit, and the world will be cursed with a 586th buggy issue tracker. By writing a plugin, you can ride the new-code high usually from start to finish, since its a much smaller task.

Reading code seems more difficult, but I think that’s largely perception. Its just another puzzle to solve. Once you get over the idea that reading code is more difficult, it’s really not that bad. For most mature projects, it’s probably easier for you to read through someone else’s mound of debugged code than it is to write and debug your own mud-ball.

I think we need find and evolve extensible tools, and stop trying to write them over again. We can get the custom behavior we all need by writing and debugging plugins, which are going to be orders of magnitude faster and cheaper than writing the whole system from scratch. I see this happening already, with communities forming around different tools to share plugins.

Next time you need a development tool, don’t open a new code file. Do us all a favor, open up a browser, and just re-use previous results.