Exposing a .NET 2.0 Class Via COM: A Case Study

I needed to access a .NET 2.0 class method from a COM application — something I don’t have an every day need for.

Like a lot of things, it seems simple enough on the surface, but the devil is in the details. This rather lengthy post illustrates a typical forensic investigation for this kind of thing, leading from obstacle to obstacle, through to the final solution. Its value is not only to describe how specifically to expose a .NET class to a COM client, but how to figure such problems out for yourself.

First, according to one school of thought, you add a couple of attributes to the class:

[ClassInterface(ClassInterfaceType.AutoDual)]
[ProgId("myStuff.Common.Utility")]

Actually even these aren’t really mandatory, because if I understand correctly, the default interface type is AutoDual anyway, and the default ProgId is the fully qualified name of the class you’re decorating. If those defaults are acceptable, you don’t need the attributes — but I figure this way there is no question what I want, and if someone changes the defaults or the configuration of the project later, it won’t break things.

The common wisdom now is to go to the project properties, under the Build tab, and check “Register for COM interop”, then build the project and everything happens automagically. (You should also click the Assembly Information button on the Application tab and check “Make assembly COM-Visible” as well).

That is great for your development machine, but I needed to deploy this on a server where Visual Studio isn’t available. Besides, my COM client and its interactive test environment for accessing this Interop DLL only exists on that server. This is one of those cases where the magic is kind of useless and you have to figure out what’s going on under the hood so you can do the same thing on the target machine.

So, the next layer of the onion: at a Visual Studio command prompt, execute the following:

REGASM foo.bar.dll /tlb:foo.bar.tlb

The command creates a type library (.TLB file) and registers the assembly for COM interop. Note that the registry entry that’s created for your library actually points to a generic runtime DLL (system32\mscoree.dll) which in turn loads your actual assembly.

Next, there was the little matter of deployment. The library already exists and is used by a .NET 2.0 application that’s called by a Windows service. If I simply rolled out the decorated / recompiled DLL and registered it in place, there would be conflicts because the client’s IT team rolls out new releases of that service at least once a day, sometimes more often. If IT did that while the COM app was using the DLL, they would either be unable to copy the new DLL on top of the old one due to file locking, or they would crash the COM app if it happened to be processing.

Since the particular method I was using seldom (virtually never) changes, and the COM application is in the process of being sunsetted, I took the easy way out and elected to deploy a private copy of the .NET DLL in the COM app’s home folder, and register it there.

So, I copied the .NET DLL and the generated .TLB into the COM app’s home folder, and executed regasm as I did above on my dev machine.

The first problem I encountered when I tried to access the .NET method from the COM test environment was one of those (un)helpful COM error codes and its vaguely worded error message text that said something about some unspecified file not being found.

Googling around for something like this can be a hit-or-miss affair, but after about 15 minutes of noodling around I found someone suggesting that even though the regasm utility could find the DLL when you executed it, some sort of unwarranted assumption is made (by the .NET runtime?) about where your DLL is physically to be found when it’s called by the COM client, and you have to use the /codebase option of regasm to explicitly tell it that, at runtime, you want to use the DLL you’re registering:

REGASM foo.bar.dll /codebase /tlb:foo.bar.tlb

That took care of the initial problem, and revealed the next issue:

OLE error code 0x80131509: Unknown COM status code.

That’s when I stumbled upon this excellent, comprehensive listing of all COM and OS error codes, which had this entry for that error code:

COR_E_INVALIDOPERATION: An operation is not legal in the current state.

This not only gave me a constant name (COR_E_INVALIDOPERATION) to Google on in addition to the hex error code, but also, a slightly more meaningful error message both to search on and possibly to use as a clue to the true problem.

Further searching located a person who indicated they got this error when the .NET assembly references another assembly, and that’s when it hit me that this assembly has a reference to another DLL that I hadn’t placed in the COM application folder along with the .NET assembly. The referenced wrapper is not used by the method I’m calling, but just possibly the assembly is feeling a little lonely and refusing to work alone.

So I copied the referenced DLL into place, unregistered / reregistered the DLL, and still no joy … the same error. The only other intelligence I could dig up was an ominous post that said this error crops up in .NET2.0, breaking assemblies that would work fine under the .NET 1.1 runtime.

Figuring it was time for a different approach, I did something I probably should have done in the first place and pulled out my venerable copy of Adam Nathan’s .NET and COM: The Complete Interoperability Guide. I tend to forget that I still have paper books on the shelf.

Nathan takes a different approach, assuming you will always put your interop DLL into the Global Assembly Cache (GAC). This requires it to be strongly named. So, you begin by generating a key file:

sn -k KeyFile.snk

… and then add another attribute to the class to be exposed:

[assembly:AssemblyKeyFile("Keyfile.snk")]

Right away you run into the problem that VS2005 doesnt like “assembly:”, claiming that it’s not a valid attribution location. But if you leave it off, it complains that AssemblyKeyFile is only valid on ‘assembly’ declarations. Talk about circular problems!

Well, I said Nathan’s book was venerable, and that means it was published in 2002, and thus, the instructions would have been developed with VS2002 or at best, VS2003. Something is clearly different with VS2005.

It turns out that the AssemblyKeyFile attribute is no longer needed and in fact is now considered a security risk. Just leave it out. Instead, specify the strong name key file in the project properties under the Signing tab by checking “Sign the assembly” and pointing to the file, then building the project.

Alas, I still got the same COM error, whether or not I install this DLL in the GAC (gacutil -i foo.bar.dll to install, or gacutil -u foo.bar to uninstall — note the absence of the dll extension when uninstalling, as you just use the assembly name itself, not its file name).

Now it was clearly time to simplify, so I made a copy of the project, reduced it down to just the method I wanted to expose and its dependencies, and compiled that to a stand alone class library with no dependencies outside the .NET 2.0 assemblies normally installed in the GAC. I configured the new project to create and use the .snk file, and to expose itself to COM, and to register for COM Interop, as before. But I left out all the code attributes … trying to keep everything as simple and default as possible.

I copied the new dll / tlb files to the target server, regasm’d it, installed it in the GAC, and woo-hoo — it worked perfectly.

The best I can figure is that the obscure COM error was masking something that would have been simple to resolve in the original DLL, if only I had known what it was. Perhaps a reference to a missing config file.

Now if I were in the situation where I really needed to use the original code base and have it work, I would have to go back and trouble shoot my way out of that … probably by setting up some try / catch blocks in the class constructor and writing detailed exception info out to a text file in order to figure out what the real problem was. But this solution does the job for me, since it’s essentially a temporary situation while the COM app is being decommissioned.

The above represented a few unpleasant hours of hair-pulling. But I think it points out an important principle of trouble-shooting and problem-solving: when you run into a brick wall, don’t keep ramming your skull into it. Instead, move the wall, or attack something else. There obviously wasn’t enough perceivable information to continue down the first two paths I tried, and where a lot of people get tripped up is to get exasperated and get into an impotent raging funk that this can’t possibly be happening and makes no sense. The truth is, it is happening, and would make sense if you had clear visibility into the true problem, which you don’t.

The other trap is to think that your only option is to post to newsgroups like a lost soul. I’ve been known to do it, but generally don’t bother anymore because in my experience you often get no response, no timely response, and/or an inaccurate response. The more complex and obscure the issue, the worse the idea of getting help from newsgroups is, because you have to communicate a complicated issue and get someone to understand it.