When debugging BizTalk solutions you find yourself many times in a situation where you need to attach the Visual Studio debugger to the running BizTalk host instance. This is very easy to do. In Visual Studio you simply select ‘Debug’ then ‘Attach to Process’. From the dialog you select the ‘BTSNTSvc.exe’ process and finally click the ‘Attach’ button.

It gets a little more tricky when you have multiple host instances running on your development box. The dialog now shows all host instances and you need to pick the one running the artifact (pipeline component, orchestration, etc.) that you want to debug. The bad thing is that there is no readable name displayed and the only property that distinguishes between the instances is the PID (column: ID). So how do you know how to pick the correct one and go on debugging?

There are basically two approaches:

- You select all the ‘BTSNTSvc.exe’ process in order to attach the debugger to all of them.

- You use some tooling to find out the PID.

Although the first option works, it is not the best solution. Attaching to all processes takes more time and resources so it is better to just pick the right one.

Now this is nothing new and there have been a number of smart people that blogged about methods to find out the PID for a host instance. Samples are here and here.

In this post I want to show the PowerShell way of doing this.

When I use the ‘Get-ChildItems’ cmdlet on the host instances container PowerShell shows a list of ‘BTSHostInstance’ objects in the console:

Since the process id is not a property of the ‘BTSHostInstance’ object it is not shown in the list.

Fortunately the nice thing about PowerShell is that you can extend properties with extra members using the Add-Member cmdlet. In the script below I create a function that adds a a property containing the process id to every non isolated host.

function GetHostPID
{
    Get-ChildItem -Path 'Biztalk:\Platform Settings\Host Instances' | ForEach-Object {

        if ($_.HostType -ne 'Isolated')
        {
            [string]$a = (Get-WmiObject Win32_Process -filter "CommandLine Like '%$($_.HostName)%'").ProcessId
       
            $_ | Add-Member -MemberType NoteProperty -Name PID -Value $a
       
            Write-Output $_
              
        }
    } | Format-Table PID, Name, HostName, NTGroupName, RunningServer, HostType, ServiceState
}

When I execute the function I get a nice list similar to the list above but with the PID added to every row:

I can add this function to my function library so it loads automatically on startup and is always available in my PowerShell console.

Of course you can do this without the PowerShell provider for BizTalk and use WMI only. In that case it would be hard and require far more lines of code to get a nice formatted list like show above.

Every now and then you need to do some BizTalk management things in code. This is where the classes come in very handy.

Last week I wrote a small sample to show and explain the use of the classes to a co-BizTalker. He was excited about this but told me it would be a good thing if we provide some more sample code. I’m sure we (me and the great people I work with on this project) will come up with a lot more information when we release the final version. For now I will only post this small sample.

It shows that a lot of “BizTalk management work” can be done using only a couple lines of code. Also this is done in a uniform way (using a single API).

This is what the sample does:

• connect to a BizTalk group (management database).
• create an application.
• add a BizTalk assembly (resource) to the application.
• import a binding file.
• creates and exports an MSI file from the new application.
• Starts the application.
• Resets the BizTalk Host.

Here is the code:

As you can see below I only need 8 lines of code to accomplish it. I don’t know how much code this would take when it was done using the “regular” API’s but I’m very sure many more lines are needed.

If you like this please download the beta from codeplex and start using the classes.

From a developer perspective however there is a nasty side effect that you should be aware of. This is caused by the fact that there is no MS virtualization tool running on a client OS that supports 64 bit guests.

Most developers follow the best practice to do development in a virtual environment and use their host for e-mail, word processing, etc.

In my case the host was Windows 7. I used my server running hyper-v to initially build the a new virtual BizTalk 2009 development box running on Windows Server 2008 R2. After that was done I wanted to transfer the .vhd to my laptop and run it from there.

As far as I know the MS options for running the VM are:

• Microsoft Virtual PC 2007
• Microsoft Virtual Server 2005 R2
• Microsoft Windows Virtual PC (currently RC)

As none of these options support 64 bit guest meaning I can’t use a Microsoft Virtualization tool to run my new Windows Server 2008 R2 development environment. Hyper-V supports 64 bit guest OS but doesn’t run on Windows 7.

The “not so nice” solutions left for me were:

• Use a non MS virtualization tool (like sun virtual box or VMware) that support 64 bit guests.
• Replace Windows 7 on my host with windows server 2008 hyper-V.
• Rebuild the virtual machine and use another 32 bit OS.

I could easily fix this by repairing the BizTalk installation but I must say I do encounter more strange problems with BizTalk 2009 and Visual Studio 2008. It looks like fixes and patches to VS mess up the BizTalk 2009 functionality.

This is probably related to the problem (and solution) described in Matt Milners post. I didn’t try it but I think Matt’s registry solution would have also fixed the issue above.

In particular I wanted to examine the context properties of message stored in the spool table. The reason I wanted to do this is a little bit irrelevant for now and might eventually come back in a future post or article.

Anyway, the spool table has a very simple structure:

CREATE TABLE [dbo].[Spool](
    [uidMessageID ] [uniqueidentifier] NOT NULL,
    [UserName] [sysname] NOT NULL,
    [PublishingServer] [sysname] NOT NULL,
    [OriginatorSID] [sysname] NOT NULL,
    [OriginatorPID] [nvarchar](256) NOT NULL,
    [dtTimeStamp] [datetime] NOT NULL,
    [dtExpiration] [datetime] NULL,
    [nvcMessageType] [nvarchar](128) NULL,
    [nNumParts] [int] NOT NULL,
    [uidBodyPartID] [uniqueidentifier] NULL,
    [nvcBodyPartName] [nvarchar](256) NULL,
    [nCounter] [int] NOT NULL,
    [imgContext] [image] NULL )

The column I was interested in is called ‘imgContext’. For each message this column stores an encoded serialized value of the collection of context properties, something like this:

0xC4E0906C1849D311A24200C04F60A5330500000074000000680074007 etc, etc….

For my experiment I wanted to work in code with IBaseMessageContext interface much like the way we do when coding custom pipeline components.

So the question was how can I directly create a IBaseMessageContext instance from ‘0xC4E0906C1849D311A……’?

After clicking around for quite a while in Reflector I was able to create a very simple console application that gave me access to the context of the message. The only thing I needed to provide was the Message ID which can be fetched from the column ‘uidMessageID’ or the BizTalk Administration Console. Here is the code:

Running writes the context properties to the console:

I hope this code will help someone who, like me, also has the need to go beyond the BizTalk borders

A couple of notes:

• This code is only for ‘research’ purposes and should never be used in production environments or any other purpose.
• If you want to do this. Make sure the message stays in the spool table. If the message is processed by BizTalk and everything went fine the message will be deleted from the spool table. To prevent this you can set a breakpoint on the processing orchestration, create an error or disable the BizTalk Sql agent jobs.
• The code was developed using BizTalk 2009 but should also work in previous versions.

The new table:

As mentioned in the previous post this information will probably also appear in the the final BizTalk 2009 documentation.

After doing some research on the Internet I found out that it is possible and fairly simple to do. Actually it was already possible in BizTalk 2006. It was just something that I (and the co-workers that I asked) didn’t know. So please stop reading if you are already familiar with this

A typical  scenario is a pipeline that consists of a XmlDisassembler and this custom RIP pipeline component. The interchange is received in the pipeline and disassembled into separate body messages by the XmlDisassembler. After that, for each body stream some processing is performed by the custom pipeline component. If this processing fails the single body instance needs to be suspended instead of the whole interchange.

There are basically three things you must do in your pipeline component to enable this:

Write property “SuspendMessageOnRoutingFailure”

The property “SuspendMessageOnRoutingFailure” needs to be written to the context of the message in case the processing of the message in the pipeline components succeeds. The value of the property needs to be set to true.

This property tells the messaging engine that each body message needs to be suspended in case of a routing failure. If you don’t write this to the context you’ll get a routing failure report for each body message while there will only be one suspended message (the whole interchange).

Write property “MessageDestination”

This property needs to be written to the context of the message when the custom pipeline components has decided that the processing on this specific body message has failed.

The value needs to be ‘SuspendQueue’ so that the messaging engine knows this particular body message needs to be suspended.

Return a searchable stream

The messaging engine needs a searchable stream to be able to put the body message in the ‘SuspendQueue’. I don’t know the exact reason behind this but the engine probable needs to rewind the stream for proper error handling. Anyway we just have to make sure we return a searchable stream when the body instance needs to be suspended.

I wrote a simple test application to test this. The first I did was adding a property to enable or disable the RIP feature of my component. Much like the setting for the XmlDisassembler (and XmlValidator in BizTalk 2009)

The execute method of the pipeline looks like this:

First the incoming stream is wrapped a XpathMutatorStream object to be able to receive an event when a certain xpath is matched. I use this to check whether or not the message is wrong (and should be suspended). In this sample the body messages each contain an attribute to indicate whether they are ‘good’ or ‘wrong’.

Next I create a virtual stream (which is searchable) to meet the requirement of returning a searchable stream.

In the third step I create a new output message containing the searchable stream.
If the message is ‘good’ I don’t care about RIP being enabled or disabled and just return the message (using the method ‘HandleHappFlow’). The only thing I need to do is write the ‘SuspendMessageOnRoutingFailure’ context property to the message as mentioned above.

If the message is ‘wrong’ and RIP is enabled I call the method ‘HandleRecoverableError’. The code of this method:

It just returns the message with the searchable stream, set some error information and writes the property ‘MessageDestination’ to the context.

Finally if the message is ‘wrong’ and RIP is disabled I throw an error so that the whole interchange will be suspended by the messaging engine.

The sample source can be downloaded from here. It was developed using the beta release of BizTalk Server 2009. As said before the sample should also work for BizTalk 2006. The solution needs to be manually converted back to a Visual Studio 2005 solution however.

BizTalk Server 2009 brings developers a lot of new functionality. In presentations and blog posts most of those new functions have already been described and shown.

One of the new things that is not or only briefly mentioned are the enhancements to Recoverable Interchange Processing (RIP) in BizTalk 2009. Also the help file (beta) of BizTalk 2009 doesn’t provide a lot of information on this (yet) and there are no samples.

I had some time to play with RIP in BizTalk 2009. The diagram below provides an overview of the very simple BizTalk sample application that I used.

This is what happens:

1). First a ‘family’ envelope XML message (also called an interchange) is received by a file receive port.

2). The receive port contains custom pipeline which in turn contains two default pipeline components; XmlDisassembler and XmlValidator.

3). The XmlDisassembler disassembles the family envelope message into separate ‘person’ XML messages

4). The XmlValidator component validates the individual ‘person’ messages before they are delivered to the message box.

5.) The receive port has failed message routing turned on. So if anything fails in the steps above a send port with a subscription on the failed messages picks up the messages and writes them to an error folder (left send port in the diagram). If everything works ok the disassembled messages are saved in the out folder by the another send port (send port at the right in the diagram).

Another thing to note is that I set the recoverable interchange processing option to true in the properties of the XmlDisassembler.

RIP in BizTalk Server 2006 R2

First lets look on how this works in BizTalk Server 2006 R2.

When I post an invalid message (see message below; the second node has an invalid element named ‘Name’). The XmlValidator fails and the message is suspended, handled by failed message routing and finally written into the error folder.

The interesting thing here is that the message in in the error folder contains the full interchange (that is the original message that I provided to BizTalk). This happens because the RIP feature only applies to the XmlDisassembler component (see ‘A’ in the diagram) and not to the XmlValidator component (‘B’ in the diagram).

RIP in BizTalk Server 2009

Compared to BizTalk 2004 RIP was a big improvement in BizTalk 2006. In BizTalk 2009 the RIP feature has now been extended to other components. The XmlValidator pipeline component now also has the option to turn op RIP via the pipeline configuration.

The invalid message above is now handled in a different way. The disassembled valid person messages (1 and 3,4,5) are now put into the out folder while the invalid person message (2) is put in the error folder.

So now instead of the whole interchange only the failed person message is suspended while the valid messages continue their way to the message box.

The RIP setting in the XmlValidator has no relation with the RIP setting in the XmlDisassembler. So the above sample will produce the same result if the RIP setting in the XmlDisassembler is set to false. 

Internally this works because the XmlValidator pipeline component has been extended with code that will return a message stream even when there is a validation error. This stream is accompanied with special information on the context so that the message agent will put the message in the suspended queue.

RIP and maps

In BizTalk 2009 it is even possible to extend this behavior to maps that are used in the receive port. To enable this you have to write a special property called ‘SuspendMessageOnMappingFailure’ to the context of the message.

Setting the value of ‘SuspendMessageOnMappingFailure’ to true will only suspend the invalid message instead of the whole interchange. I personally think it is a bit cumbersome that you have to enable this in code. One has to write a custom pipeline component just to turn on a feature. It would be nice if this could be enabled by setting a property in the map configuration section of the port using the BizTalk administration console.

Anyway the BizTalk 2009 Recoverable Interchange Processing feature is a nice extension to handle interchanges in BizTalk server. It can save you from a lot of custom coding in pipeline components.

It occurred when I was installing Windows Sharepoint Services 3.0 with service pack 1, which is one the prerequisites for the SharePoint Services adapter.

I wanted to get on with the installation and didn’t want to spend much time on finding a solution. So to get around this I copied the .MSI back to my host system (in my case Windows Vista (Business edition) 64 bit). After that I used the command:

Sharepoint.exe /extract:c:tempwss

to extract the installation files from the downloaded .exe. Finally I copied the extracted files back to my virtual machine and ran ‘Setup.exe’. This time setup completed without a glitch.

I suspect Windows 2008 Server or one of the BizTalk 2009 prerequisites I already installed prior to WSS 3.0 to be the cause of the problem. Please comment or e-mail me if you’re not lazy like me, but took the time to find a more elegant and satisfying solution . Until that time I hope I can help someone with this workaround.

In BizTalk Server 2009 it looks like Microsoft has continued to move functionality from the Health and Activity (HAT) tool to the BizTalk Administration Console (MMC).

When we went from BTS2K4 to BTS2k6 we saw that some of the tasks we did with HAT (and other tools) on 2k4 now could be done with MMC on 2k6.

The biggest improvement for me personally was the inclusion of the subscription viewer function in the MMC. I still have nightmares about the user interface of the 2k4 subscription viewer tool .

In BizTalk Server 2009 Microsoft took it a step further and removed the HAT tool. This means the remaining functionality is now also included in MMC.

To be more precise the HAT is not completely removed you only don’t need to start it as a separate tool anymore. If you for example use things like ‘Message flow’ or the ‘Orchestration debugger’ you’ll still get a HAT style window. Also the task manager shows a process called ‘BTSHatApp.exe’ which I think I saw somewhere before . Of course I also couldn’t resist typing ‘BTSHatApp.exe’ from the start prompt, but that results in the following error message:

Anyway I think it is a good thing because we now really have one tool (or at least one entrance point) to do everything. I also never liked the HAT user interface very much.

There are (or were) basically three functions/queries that I use in HAT:

• Most recent 100 service instances
• Messages sent in the past day
• Messages received in the past day

The query expression pane in the 2k9 MMC now has values for ‘Tracked Message Events’ and ‘Tracked Service Instances’. This will allow me to do the same thing in MMC that I previously had to do in HAT.

Screen shot 1 shows the ‘Most recent 100 service instances’ in the 2k9 way. Screen shot 2 shows the ‘Messages received in the past day’.

Screen shot 1: Most recent 100 service instances.

Screen shot 2: Most recent 100 send messages.

I also promised to update the overview as soon as a new version of service pack is available. To keep the promise I added the recent beta of BTS2k9.

This information is now also provided by Microsoft in the BizTalk Server 2009 help. See help topic ‘What Version of BizTalk Server Do I Have?’.