http://maxime-labelle.spaces.live.com/Blog/cns!D8D9369449D177DA!236.entry

Maxime added support for deploying vocabularies and policies to the PowerShell provider for BizTalk. In our opinion this is the easiest way to deploy BRE artefacts.

For now it is only available when you grab and build the latest sources. It will be included in the final 1.0 release of course.

Great work Maxime!

Untyped policies work great when tested directly in the Business Rules Composer interface or when executed from .Net code. Unfortunately I stumbled across a nice issue when I wanted to call the rules from an orchestration using the call rules shape.

In my first design of the orchestration I received a message of type System.Xml.XmlDocument. After that I used a call rules shape to execute the policy. Because this is an untyped policy it will only accept a message of System.Xml.XmlDocument as input.

Easy! Well, not exactly. While testing this orchestration I did not encounter an exception but found out my rule also did not fire. I enabled rule tracking and saw that my message was not asserted as a fact into the BRE. The tracked information was ‘fact not recognized’.

I tried some things to fix this without success. I ended up viewing the generated C# code for the orchestration and noticed a difference in the code generated for the call rules shape when using untyped and typed policies. For typed policies a new instance of Microsoft.RuleEngine.TypedXmlDocument is created  as a fact wrapper around the orchestration message. This TypedXmlDocument is then passed on to the BRE. For untyped policies this is different. There is no TypedXmlDocument created and the XmlDocument message is passed directly on to the BRE.

So in pseudo C# code, for a typed policy:

typedXDoc = new Microsoft.RuleEngine.TypedXmlDocument("MessageType”, (System.Xml.XmlDocument)orchestrationMessage);
policy.Execute(typedXDoc);

for an untyped policy the code looks like this:

policy.Execute((System.Xml.XmlDocument)orchestrationMessage);

The obvious difference between the two is that the first uses a ‘TypedXmlDocument’ instance to wrap the message. I expected the XLANG code generator to do the same for the untyped version but that is not the case. So what does this mean? Does this mean untyped policies are not supported? Or at least not in orchestrations? Is the only option to use code in an expression shape or helper class to execute untyped policies from within an orchestration?

Because I was completely stuck here I decided to ask BRE (and BizTalk) guru Charles Young for help. 

It turned out that I had to use the special ‘Any’ schema to solve this. As Charles explained to me there are two ways of working with untyped messages in BizTalk. One is the famous XmlDocument approach, the other one is the (undocumented) ‘Any’ schema.

One of the differences between the XmlDocument and ‘Any’ schema is that the latter is treated as a schema type by BizTalk. This means the XLANG code generator will wrap it inside a TypedXmlDocument for a rules call. This exactly like the way it works for a typed policy.

The only two things I had to do was change were the message type from ‘XmlDocument’ to ‘Any’ and accordingly the policy.

This is a picture of the revised orchestration.

The changes needed in the policy are described in a rewritten version of the original post. You can find it here.

A demo solution around this can be downloaded from here. It contains two orchestrations. One which uses the XMLDocument approach without the rules getting fired. The other using the Any approch with the rules getting fired. Remember to change to paths in the binding file before deploying.

Full credits for this solution go to Charles Young. Charles thanks for helping me out.

I this post I will use the exact same sample as in the previous post. These are the schemas used:

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003"

xmlns="http://UntypedBRE.FirstSchema"
targetNamespace='http://UntypedBRE.FirstSchema'

xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="FirstSchema">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="FirstName" type="xs:string" />
        <xs:element name="LastName" type="xs:string" />
        <xs:element name="IsJohn" type="xs:string" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003"
xmlns="http://UntypedBRE.SecondSchema"
targetNamespace='http://UntypedBRE.SecondSchema'
xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="SecondSchema">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="FirstName" type="xs:string" />
        <xs:element name="LastName" type="xs:string" />
        <xs:element name="IsJohn" type="xs:string" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003"

xmlns="http://UntypedBRE.ThirdSchema"
targetNamespace='http://UntypedBRE.ThirdSchema'

xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="ThirdSchema">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="FirstName" type="xs:string" />
        <xs:element name="LastName" type="xs:string" />
        <xs:element name="IsJohn" type="xs:string" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

In my sample policy I want to check the ‘FirstName’ element. If the value is equal to ‘John’ I want to fill the ‘IsJohn’ element with value ‘yes’. The policy (and single rule) should work for all the above schemas.

The problem is that mentioned schemas belong to a different namespace and have a different rood node, hence in BizTalk terms have a different message type. Because XML schemas facts in the BRE are by default tightly coupled to a specific schema the consequence is that those facts can only operate on a single type of message.

In order to make this generic you have to do the following:

1. Add one of the schemas to the Facts Explorer in the BRE

2. Make the schema general

As you can see the Document Type resembles the type of the schema I added. To make this generic I change this value to ‘Microsoft.XLANGs.BaseTypes.Any’. This will make sure that if I use a fact from this schema in a rule it will not be typed to this schema but will be generic:

3. Create the rule

In this step I create the rule. The facts will be filled in later.

4. Modify the XML facts

In the rule I need to evaluate the ‘FirstName’ fact and optionally set the ‘IsJohn’ fact. Because I want this to work on all schemas I need to define the facts in a generic way. If I click on the ‘FirstName’ fact I can see the xpath statements that point to this fact in the property pane:

The ‘Xpath Field’ and ‘Xpath Selector’ properties are directly referring to ‘FirstSchema’ root node and namespace. I change the values to make them generic also:

Note that I’m using ‘self::node()’ here. I described this trick before here.

Now the XML fact is no longer pointing to a specific namespace or root node. It just points to a ‘FirstName’ node somewhere in Xml message.

There are of course other possible values for ‘Xpath selector’ and ‘Xpath Field’ to solve this. It all depends on the schemas. If for example the facts you need all have the same parent node you can make the ‘Xpath selector’ select the parent node and the ‘Xpath Field’ select the ‘FirstName’ element.

I do the same for the fact I want to update in the action of the rule:

5. Complete the rule by adding the facts

Finally I can drag the XML facts from the Facts Explorer to my rule to complete the condition and create a new action. Like this:

You can see that the both the condition and the action are not referring (anymore) to any specific schema  but instead to any schema that has ‘FirstName’  and ‘IsJohn’  elements.

Testing the rule with instances from two different schemas shows that this works:

One thing to note about this is that the way I changed the xpath statements for the Xml facts comes with a performance penalty. Using things like ‘//*…….’.  will make the engine go through the whole xml tree which is less efficient then using the original full xpath statement. So if performance is a strict requirement be careful using techniques like these.

Another thing is that I do not check for the existence of the nodes first. The policy will crash when an message is processed that does not contain on of the nodes used in the rule.

I wanted to do the same after the release of SP1 for BizTalk server 2006 R2. Unfortunately it appears that the version number is not updated after applying the service pack. I double checked with some other people but they faced the same issue.

I already reported this issue to Microsoft and blogged about it for the beta release of the SP but it seems they did not fix it for the final release. It think it is a bad thing as you cannot easily determine which version and sp level is running based on the version number.

Anyway the latest version number table looks like this:

I will report the issue to Microsoft again. Although I double checked with some people I might be wrong here. Please let me know if you see different behavior.

In order to still be able to determine the version number in an easy way I wrote a PowerShell script. I have a post about it here.

You cannot just look at the version number anymore because it seems SP1 of BizTalk Server 2006 R2 does not update the version number. This means the version numbers for BizTalk 2006 R2 without and with service pack 1 applied are the same. See also my blog post here.

The script will detect any BizTalk version and service pack from 2004 and later. I have tested it on different environments with different BizTalk versions. Please let me know if it is not working

Below is the source, but you can also download it from here.

# Initialization of helper variables
# BizTalk version numbers
$versionBTS2004 = "3.0.4902.0"
$versionBTS2004SP1 = "3.0.6070.0"
$versionBTS2004SP2 = "3.0.7405.0"
$versionBTS2006 = "3.5.1602.0"
$versionBTS2006R2 = "3.6.1404.0"
$versionBTS2009 = "3.8.368.0"

# BizTalk version description
$descriptionBTS2004 = "BizTalk Server 2004"
$descriptionBTS2004SP1 = "BizTalk Server 2004 with service pack 1"
$descriptionBTS2004SP2 = "BizTalk Server 2004 with service pack 2"
$descriptionBTS2006 = "BizTalk Server 2006"
$descriptionBTS2006R2 = "BizTalk Server 2006 R2"
$descriptionBTS2006R2SP1 = "BizTalk Server 2006 R2 with service pack 1"
$descriptionBTS2009 = "BizTalk Server 2009"

# Registry paths
$bizTalkRegistryPath = "HKLM:\SOFTWARE\Microsoft\BizTalk Server"
$biztalk2006SP1UninstallRegistryPath = 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\Microsoft Biztalk Server 2006 R2 Service Pack 1 `[KB 974563`]'

$installedVersion = $null

# Check if BizTalk is installed:
if ((Test-Path $bizTalkRegistryPath) -eq $true)
{

# Set location to BizTalk registry key
Set-Location $bizTalkRegistryPath
$key = Get-ChildItem

# Get version number
$productVersion = $key.GetValue("ProductVersion")

switch ($productVersion)
{

$versionBTS2004 { $installedVersion = $descriptionBTS2004 }
$versionBTS2004SP1 { $installedVersion = $descriptionBTS2004SP1 }
$versionBTS2004SP2 { $installedVersion = $descriptionBTS2004SP2 }
$versionBTS2006 { $installedVersion = $versionBTS2006 }
$versionBTS2006R2
{
if ((Test-Path $biztalk2006SP1UninstallRegistryPath) -eq $false)
{
$installedVersion = $descriptionBTS2006R2
}
else
{
$installedVersion = $descriptionBTS2006R2SP1
}
}
$versionBTS2009 { $installedVersion = $descriptionBTS2009 }
}
}

if ($installedVersion -eq $null)
{
Write-Host "BizTalk Server is not installed on this machine."
Exit
}

Write-Host "BizTalk Server installation found on this machine."
Write-Host "Product version number: $productVersion"
Write-Host "Installed version: $installedVersion"

Although we will have this fixed in the 1.0 final release I decided to provide a hotfix right now for those who want to play with the beta and have a remote SQL server. This is major show stopper.

If you want to receive the hotfix please drop me an e-mail (see about page of my blog) and I will send you the fix and instructions a.s.a.p.

As an alternative you can download the latest sources from codeplex and compile them yourself.

The hotfix will also fix other reported issues:

- #3042
- #3170
- #3280
- #3390
- #3391
- #3393

When debugging BizTalk solutions you find yourself many times in a situation where you need to attach the Visual Studio debugger to the running BizTalk host instance. This is very easy to do. In Visual Studio you simply select ‘Debug’ then ‘Attach to Process’. From the dialog you select the ‘BTSNTSvc.exe’ process and finally click the ‘Attach’ button.

It gets a little more tricky when you have multiple host instances running on your development box. The dialog now shows all host instances and you need to pick the one running the artifact (pipeline component, orchestration, etc.) that you want to debug. The bad thing is that there is no readable name displayed and the only property that distinguishes between the instances is the PID (column: ID). So how do you know how to pick the correct one and go on debugging?

There are basically two approaches:

- You select all the ‘BTSNTSvc.exe’ process in order to attach the debugger to all of them.

- You use some tooling to find out the PID.

Although the first option works, it is not the best solution. Attaching to all processes takes more time and resources so it is better to just pick the right one.

Now this is nothing new and there have been a number of smart people that blogged about methods to find out the PID for a host instance. Samples are here and here.

In this post I want to show the PowerShell way of doing this.

When I use the ‘Get-ChildItems’ cmdlet on the host instances container PowerShell shows a list of ‘BTSHostInstance’ objects in the console:

Since the process id is not a property of the ‘BTSHostInstance’ object it is not shown in the list.

Fortunately the nice thing about PowerShell is that you can extend properties with extra members using the Add-Member cmdlet. In the script below I create a function that adds a a property containing the process id to every non isolated host.

function GetHostPID
{
    Get-ChildItem -Path 'Biztalk:\Platform Settings\Host Instances' | ForEach-Object {

        if ($_.HostType -ne 'Isolated')
        {
            [string]$a = (Get-WmiObject Win32_Process -filter "CommandLine Like '%$($_.HostName)%'").ProcessId
       
            $_ | Add-Member -MemberType NoteProperty -Name PID -Value $a
       
            Write-Output $_
              
        }
    } | Format-Table PID, Name, HostName, NTGroupName, RunningServer, HostType, ServiceState
}

When I execute the function I get a nice list similar to the list above but with the PID added to every row:

I can add this function to my function library so it loads automatically on startup and is always available in my PowerShell console.

Of course you can do this without the PowerShell provider for BizTalk and use WMI only. In that case it would be hard and require far more lines of code to get a nice formatted list like show above.

I’m really honored and excited. I would like to thank the people at Microsoft, community members and other people involved. Special thanks to fellow MVP Jan Eliasen (IMHO: the book will be great ) and codeplex powershell team members: Maxime and Michel.

No MVP logo? Well, I’m waiting for access to the MVP logo kit

Another option is that a new development team member has just started on a fresh development box and you want him to use the same host configuration that you have. There are a lot of other scenarios where you want to copy the configuration of your hosts and host instances to another machine.

A lot of great BizTalk people have already showed how this can be done in an automated way. Some samples are:

- VBScript and WMI
- Custom tool 1
- Custom tool 2

In this post I want to show you how this can be done using PowerShell and the PowerShell provider for BizTalk. So this is actually the next post in my series on more advanced things you can do with the provider.

I use a slightly different approach compared to the existing tools. I use a PowerShell script that can be executed on the source server. This is the server that already has the host configuration set up, for example a development server. The output of running this script is actually another PowerShell script that you can run on the destination server. This output scripts contains all necessary statements to create the hosts, corresponding host instances and settings.

This is how it works:

- execute the script
- the script prompts for the name and path of the output file:

- after a short while the script is finished and a PowerShell script is created. You can run this script on the destination server or send it to an administrator who will run it on another server.

This is the created script:

#Determine BizTalk root and switch to host container.
$BizTalkRoot = (get-psdrive -PsProvider BizTalk).Root
Join-Path $BizTalkRoot 'Platform Settings\Hosts' | Set-Location

$RunningServer = $Env:ComputerName
#Create hosts
#Create host 'ReceiveHost' and set properties.
New-Item -Path 'ReceiveHost' -HostType 'InProcess' -NtGroupName 'BizTalk Application Users' -AuthTrusted:$False
Set-ItemProperty -Path 'ReceiveHost' -Name HostTracking -Value False
Set-ItemProperty -Path 'ReceiveHost' -Name Is32BitOnly -Value True

#Create host 'ProcessingHost' and set properties.
New-Item -Path 'ProcessingHost' -HostType 'InProcess' -NtGroupName 'BizTalk Application Users' -AuthTrusted:$False
Set-ItemProperty -Path 'ProcessingHost' -Name HostTracking -Value False
Set-ItemProperty -Path 'ProcessingHost' -Name Is32BitOnly -Value True

#Create host 'SendHost' and set properties.
New-Item -Path 'SendHost' -HostType 'InProcess' -NtGroupName 'BizTalk Application Users' -AuthTrusted:$False
Set-ItemProperty -Path 'SendHost' -Name HostTracking -Value False
Set-ItemProperty -Path 'SendHost' -Name Is32BitOnly -Value False

#Create host 'TrackingHost' and set properties.
New-Item -Path 'TrackingHost' -HostType 'InProcess' -NtGroupName 'BizTalk Application Users' -AuthTrusted:$False
Set-ItemProperty -Path 'TrackingHost' -Name HostTracking -Value True
Set-ItemProperty -Path 'TrackingHost' -Name Is32BitOnly -Value False

#Switch to host instances container.
Join-Path $BizTalkRoot 'Platform Settings\Host Instances' | Set-Location

#Create host instances
#Create host instance 'Microsoft BizTalk Server ReceiveHost BTS2K9-DEV' and set properties.
$Credential = $Host.UI.PromptForCredential('Host Instance user credentials', 'Please enter credentials for host instance ''ReceiveHost''', '', '')
New-Item -Path 'hostinstance' -HostName 'ReceiveHost' -RunningServer $RunningServer -Credentials $Credential
Set-ItemProperty -Path 'Microsoft BizTalk Server ReceiveHost BTS2K9-DEV' -Name IsDisabled -Value False

#Create host instance 'Microsoft BizTalk Server ProcessingHost BTS2K9-DEV' and set properties.
$Credential = $Host.UI.PromptForCredential('Host Instance user credentials', 'Please enter credentials for host instance ''ProcessingHost''', '', '')
New-Item -Path 'hostinstance' -HostName 'ProcessingHost' -RunningServer $RunningServer -Credentials $Credential
Set-ItemProperty -Path 'Microsoft BizTalk Server ProcessingHost BTS2K9-DEV' -Name IsDisabled -Value False

#Create host instance 'Microsoft BizTalk Server SendHost BTS2K9-DEV' and set properties.
$Credential = $Host.UI.PromptForCredential('Host Instance user credentials', 'Please enter credentials for host instance ''SendHost''', '', '')
New-Item -Path 'hostinstance' -HostName 'SendHost' -RunningServer $RunningServer -Credentials $Credential
Set-ItemProperty -Path 'Microsoft BizTalk Server SendHost BTS2K9-DEV' -Name IsDisabled -Value False

#Create host instance 'Microsoft BizTalk Server TrackingHost BTS2K9-DEV' and set properties.
$Credential = $Host.UI.PromptForCredential('Host Instance user credentials', 'Please enter credentials for host instance ''TrackingHost''', '', '')
New-Item -Path 'hostinstance' -HostName 'TrackingHost' -RunningServer $RunningServer -Credentials $Credential
Set-ItemProperty -Path 'Microsoft BizTalk Server TrackingHost BTS2K9-DEV' -Name IsDisabled -Value False

There are a couple of things to mention about the generated script:

- This is by no means a script applicable for all situations and environments. The purpose is just to show the output script as a result of running the source script. The source script will probably need some modification to make it apply to your environment. For example the script does not take into account multi server configurations, etc.
- The output script will prompt for credentials for each host instance it will create. There are also ways to script this from source to destination. See for example here. In most cases hosts in different environments will run under different accounts which is why I choose to show a prompt each time.
- Also in this script the windows group for the host is kept the same for the destination environment. This might not be the case in your particular situation.

After running this script on the destination server I have a exact copy of my host environment on the source server.

The source script looks like this:

$ScriptFile = Read-Host 'Enter full path of script file:'</pre>
'#Determine BizTalk root and switch to host container.' | Out-File $ScriptFile
'$BizTalkRoot = (get-psdrive -PsProvider BizTalk).Root' | Out-File $ScriptFile -Append
'Join-Path $BizTalkRoot ''Platform Settings\Hosts'' | Set-Location' | Out-File $ScriptFile -Append
'' | Out-File $ScriptFile -Append

'$RunningServer = $Env:ComputerName' | Out-File $ScriptFile -Append

$BizTalkRoot = (get-psdrive -PsProvider BizTalk).Root
Join-Path $BizTalkRoot 'Platform Settings\Hosts' | Set-Location

'#Create hosts' | Out-File $ScriptFile -Append
$HostArray = @()
Get-ChildItem -Path . |  Foreach-Object {
    if (!$_.IsDefault -and $_.HostType -ne 'Isolated') {
   
        $HostArray += $_.Name
   
        "#Create host '$($_.Name)' and set properties." | Out-File $ScriptFile -Append
        "New-Item -Path '$($_.Name)' -HostType '$($_.HostType)' -NtGroupName '$($_.NtGroupName)' -AuthTrusted:`$$($_.AuthTrusted)" | Out-File $ScriptFile -Append
        "Set-ItemProperty -Path '$($_.Name)' -Name HostTracking -Value $($_.HostTracking)" | Out-File $ScriptFile -Append
        "Set-ItemProperty -Path '$($_.Name)' -Name Is32BitOnly -Value $($_.Is32BitOnly)" | Out-File $ScriptFile -Append
        '' | Out-File $ScriptFile -Append
                   
    }
}

'#Switch to host instances container.' | Out-File $ScriptFile -Append
'Join-Path $BizTalkRoot ''Platform Settings\Host Instances'' | Set-Location' | Out-File $ScriptFile -Append
'' | Out-File $ScriptFile -Append

Join-Path $BizTalkRoot 'Platform Settings\Host Instances' | Set-Location

'#Create host instances' | Out-File $ScriptFile -Append
Get-ChildItem -Path . |  Foreach-Object {
    if ($HostArray -contains $_.HostName) {
      
        "#Create host instance '$($_.Name)' and set properties." | Out-File $ScriptFile -Append
        "`$Credential = `$Host.UI.PromptForCredential('Host Instance user credentials', 'Please enter credentials for host instance ''$($_.HostName)''', '', '')" | Out-File $ScriptFile -Append
        "New-Item -Path 'hostinstance' -HostName '$($_.HostName)' -RunningServer `$RunningServer -Credentials `$Credential" | Out-File $ScriptFile -Append
        "Set-ItemProperty -Path '$($_.Name)' -Name IsDisabled -Value $($_.IsDisabled)" | Out-File $ScriptFile -Append
         '' | Out-File $ScriptFile -Append
       
    }
}

In the script I skip the default and isolated host. As with the generated output script this is just a sample and might not apply to your specific situation. The source script should be modified to confirm to your requirements and environment.

You can download the source script from here.

The other day one of my BizTalk buddies asked me if I knew a way to process different messages by the same policy in the Business Rules Engine (BRE). In other words is it possible to create some sort of generic policy that can process different types of messages?

In his scenario the schemas had some fields in common and he needed those fields to be evaluated and optionally set in a single BRE policy.

Of course you can make a policy that depends on different schemas and add rules for each schema. The bad thing about that is that you will get multiple copies of almost the same rule which might lead to a maintenance nightmare.

I created a small sample to show how this can be achieved in a single policy and single rule. Below are three very basic schemas I used in this sample:

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="<a href="http://schemas.microsoft.com/BizTalk/2003">http://schemas.microsoft.com/BizTalk/2003</a>" xmlns="<a href="http://UntypedBRE.FirstSchema">http://UntypedBRE.FirstSchema</a>" targetNamespace="<a href="http://UntypedBRE.FirstSchema">http://UntypedBRE.FirstSchema</a>" xmlns:xs="<a href="http://www.w3.org/2001/XMLSchema">http://www.w3.org/2001/XMLSchema</a>">
  <xs:element name="FirstSchema">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="FirstName" type="xs:string" />
        <xs:element name="LastName" type="xs:string" />
        <xs:element name="IsJohn" type="xs:string" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="<a href="http://schemas.microsoft.com/BizTalk/2003">http://schemas.microsoft.com/BizTalk/2003</a>" xmlns="<a href="http://UntypedBRE.SecondSchema">http://UntypedBRE.SecondSchema</a>" targetNamespace="<a href="http://UntypedBRE.SecondSchema">http://UntypedBRE.SecondSchema</a>" xmlns:xs="<a href="http://www.w3.org/2001/XMLSchema">http://www.w3.org/2001/XMLSchema</a>">
  <xs:element name="SecondSchema">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="FirstName" type="xs:string" />
        <xs:element name="LastName" type="xs:string" />
        <xs:element name="IsJohn" type="xs:string" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="<a href="http://schemas.microsoft.com/BizTalk/2003">http://schemas.microsoft.com/BizTalk/2003</a>" xmlns="<a href="http://UntypedBRE.ThirdSchema">http://UntypedBRE.ThirdSchema</a>" targetNamespace="<a href="http://UntypedBRE.ThirdSchema">http://UntypedBRE.ThirdSchema</a>" xmlns:xs="<a href="http://www.w3.org/2001/XMLSchema">http://www.w3.org/2001/XMLSchema</a>">
  <xs:element name="ThirdSchema">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="FirstName" type="xs:string" />
        <xs:element name="LastName" type="xs:string" />
        <xs:element name="IsJohn" type="xs:string" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

I my sample policy I want to check the ‘FirstName’ element. If the value is equal to ‘John’ I want to fill the ‘IsJohn’ element with value ‘yes’. The policy (and single rule) should work for all of the above schemas.

The problem is that mentioned schemas belong to a different namespace and have a different root node, hence in BizTalk terms have a different message type. Because XML schema facts in the BRE are by default tightly coupled to a specific schema the consequence is that those facts can only operate on single type of message.

In order to make this generic you have to do the following:

1. Add one of the schemas to the Facts Explorer in the BRE.

2. Make the schema ‘general’

As you can see the Document Type resembles the type of the schema I added. To make this generic I change this value to ‘System.Xml.XmlDocument’. This will make sure that if I use a fact from this schema in a rule it will not be typed to this schema but will be generic.

3. Create the rule

In this step I create rule. The facts will be filled in later.

4. Modify the XML facts.

In the rule I need to evaluate the ‘FirstName’ fact and optionally set the ‘IsJohn’ fact. Because I want this to work on all schemas I need to define the facts in a generic way. If I click on the ‘FirstName’ fact I can see the xpath statements that point to this fact in the property pane:

The ‘Xpath Field’ and ‘Xpath Selector’ properties are directly referring to ‘FirstSchema’ root node and namespace. I change the values to make them generic also:

Note that I’m using ‘self::node()’ here. I described this trick before here.

Now the XML fact is no longer pointing to a specific namespace or root node. It just points to a ‘FirstName’ node somewhere in a Xml document.

There are of course other possible values for ‘XPath Selector’ and ‘XPath Field’ to solve this. It all depends on the schemas. If for example the facts you need all have the same parent node you can make the ‘XPath selector’ select the parent node and the ‘XPath Field’ select the ‘FirstName’ element.

I do the same for the fact I want to update in the action of the rule:

5. Complete the rule by adding the facts.

Finally I can drag the XML facts from the Facts Explorer to my rule to complete the condition and create a new action. Like this:

You can see that the both the condition and the action are not referring (anymore) to any specific schema  but instead to any schema that has ‘FirstName’  and ‘IsJohn’  elements.

Testing the rule with instances from two different schemas shows that this works:

One thing to note about this is that the way I changed the xpath statements for the Xml facts comes with a performance penalty. Using things like ‘//*…….’.  will make the engine go through the whole xml tree which is less efficient then using the original full xpath statement. So if performance is a strict requirement be careful using techniques like these.

Another thing is that I do not check for the existence of the nodes first. The policy will crash when an message is processed that does not contain on of the nodes used in the rule.

The best way to for any application to support management and automation is to implement such functionality on top of Windows PowerShell. Windows PowerShell contains a comprehensive API which enables the implementation of Cmdlets or a Cmdlet provider and the implementation of custom hosts.

Cmdlets or a cmdlet provider
PowerShell has uniform syntax for working with commands working and container hierarchies. This is a great advantage for administrators as they only need to learn a single way of working to manage it all. By implementing cmdlets we can “translate” this PowerShell way of working in calls to an applications native management API.

This is exactly what we have done in the provider for BizTalk. Internally we call BizTalk management APIs like ExplorerOM to perform things like starting applications, exporting MSIs, etc.

A custom host
Besides implementing custom cmdlets you can also implement a custom PowerShell host. The host can be any sort of application as long as it implements the PowerShell host interfaces. It can be a Win32 app, a web app or a text console for example.

This is exactly what Microsoft has done in the most recent versions of Exchange server (2007 & 2010). Management is done using PowerShell. The graphical management console is a PowerShell host. Exchange can be managed using this GUI or for example using scripts or commands running in the command shell console that comes out of the box with PowerShell.

The diagram below shows this architecture:

Things get even more interesting since we have PowerShell 2.0. It has a new feature called remoting which allows for remote administration and automation. Remoting uses WinRm (Microsofts implementation of the WS-Management standard). Goods news is that WinRM is firewall friendly. It only uses http(s) for communication. Now the diagram looks like this:

Is this secure? Yes it is, both PowerShell and WinRM have been designed with focus on security. Both come out of the box with security enabled. Of course you still need to pay attention especially when exposing things outside the enterprise.

Back to BizTalk
For BizTalk we have a couple of options to remotely administer a BizTalk server. We can choose between: Remote Desktop, client installation of Admin console, custom web services for management tasks, etc. Of course all have their pros and cons. With PowerShell remoting we have a new additional option.

Below I’ll show a couple of screens to see how this works.

I used two virtual machines in this demo. I installed the following software on those machines:

Virtual machine 1 (name: BTS2K9-dev):

- Windows Server 2008 R2 (I know not supported for hosting BizTalk)
- SQL Server 2008
- BizTalk Server 2009
- PowerShell 2.0
- PowerShell provider for BizTalk 1.0 (beta)

Virtual machine 2 (name: Client):

- Windows Server 2008 R2
- PowerShell 2.0
- PowerShell provider for BizTalk 1.0 (beta)

The obvious but important thing to note here is that the only thing that I have installed on virtual machine 2 that is related to BizTalk is the provider. In other words I did NOT NEED ANYTHING from the BizTalk installation CD on this machine.

The first thing I needed to was to enable remoting in PowerShell. This very simple and only requires the execution of a single command ‘Enable-PSRemoting’.

Second thing I needed to do was to add the client to the list of trusted computers on the BizTalk box. This mainly needs to be done because I just wanted to communicate over http instead of https for this demo.

On the BizTalk Box the management console is shown. Note that the default host instance is stopped:

Back to the PowerShell console on the Client to do the following:

- Create a new remote session on the BizTalk Box.
- Add the PowerShell provider snap in to the session.
- Switch to the default BizTalk drive.
- Switch the location to the host instances.
- Start the default host instance.

Finally a check on the other box to see if the host instance is running to verify this has worked (need to refresh the admin console first):

As you can see in the screen shots this is really easy to do. In my opinion it is a great new, location transparent, firewall friendly, client only way of managing BizTalk Server.

The only thing left to be done is to rebuild the BizTalk Administration Console on top of Windows PowerShell.

I usually do a lot of tryouts and proof of concepts on my development box. In my urge to get things working I never spend too much time on descriptive application names. I also never clean up things after I’m done.

This will in inevitably lead to the following mess after a couple of weeks:

From time to time I feel the need to clean up this mess. I used to do this with the BizTalk Administration Console. I start on top of the list right click the application to first stop and then delete it. Then I go to the second, etc.

The problems start to arise when there are references between applications. If I delete an application that is referenced from another application I get the following warning:

I can still delete the application but the warning is annoying and takes time. The last thing I want is to lose too much time cleaning up

Things get even worse when there are assembly resources inside the application which are referred to by assembly resources in other applications. This time it is not a warning but a serious error preventing me from deleting the application.

This means I need to take this reference dependency tree into account and delete the “child” applications before their “parents”.

This all makes cleaning up my development box very annoying and time consuming which is probably why I keep postponing it in the first place.

You can nicely automate this procedure using a generic application removal script that uses the PowerShell provider for BizTalk. In the script I take advantage of the exception handling possibilities of PowerShell. I sort of do a “on error resume next”.

This is how the script works:

• - Build a list of all the applications and put them in a array variable.
• - Loop through the list and try to delete the application on top of the list.
• - When the deletion fails because of dependencies we catch, in PowerShell terms “trap”, the exception and shuffle the array a little bit so that the application that failed is moved to the bottom of the list.
• - We then continue with the application that is now on top of the list.
• - The loop will end when all the applications are removed.

This is the script:

Some details about the script:

In the fragment below we use the provider to get a list of all applications. We make sure exclude the system and default application because they cannot be deleted.

The second thing to show in detail is the error handling routine. The provider (or actually the BizTalk Management Classes library) throws an exception of type BtsException since the application is referenced and cannot be deleted. We catch this exception by using a trap statement. In the exception handler we do two things. First we make sure to set a variable in the main scope so that we know what to do with the array later. Either move the top item to the bottom or remove the top item. Second we use “continue;” to have the main routine continue with the next statement.

After running the script this is what the admin console shows:

Nice!!!!

The script can be downloaded from here

The setup (and source) can be downloaded from codeplex.

I hope BizTalk developers and administrators will find this a useful addition to their toolbox.

In the coming weeks we will provide more information on usage and show some tips.

Every time Microsoft releases a new version of BizTalk we are all very disappointed that they did not make the expression window resizable or at least bigger.

There are rumors that this is done to discourage developers to write a lot of code in those windows and move to external assemblies with helper classes instead.

Although I agree with that I still think it would be very nice to have a bigger window occasionally.

Like the “Orchestration Variable Retriever Functoid” this was on my “experimental things to do list” for a long time. I never had or took the chance to try it until BizTalk king Stephen W. Thomas announced his competition for the best BizTalk tip or trick.

Of course I want to join this competition so I started coding. Now that it is finished I must admit it has become quite a hack but it seems to work .

Anyway the “ExpressionMaximizer” will be my entry for the competition.

A quick walkthrough:

I have an orchestration with an expression shape containing a lot of code:

I double click the expression shape to open it (in normal mode):

With expression window still open I press CTRL+SHIFT+ALT+M (nice key combination ) and ……:

I have a big expression window where I can type many long lines of code!!!!

Everything is back to normal when you close the window and open it again.

Note that you first have to open the expression shape before pressing the key combination.

Want to try it yourself?
(description below is based on Visual Studio 2008)

- Download this zip file and extract it somewhere.
- Click ‘Tools’ –> ‘External Tools…’
- Press the ‘Add’ button
- Type a title, e.g. ‘Expression Window Maximizer’
- Fill in the command text box by browsing to the extracted ‘ExpressionMaximizer.exe’

- Press the ‘OK’ button
- Select ‘Tools’ –> ‘Customize…’
- Press the ‘Keyboard…’ button
- In the ‘Show commands containing:’ text box type ‘Tools.External’
- Select the correct ‘Tools.ExternalCommandX’ from the list. (You need to find out the correct entry by counting the external tools in the Tools menu).
- Click in the text box ‘Press shortcut keys:’ to set the focus to it.
- Press CTRL+ALT+SHIFT+M
- Click the ‘Assign’ button.

- Press the ‘Ok’ button.
- Press the ‘Close’ button.

I tried this on ‘Windows Server 2008 R2/VS2008′ and ‘Windows Server 2003 R2/VS2005’ without problems but I still take no responsibility for malformed windows or other disasters After all it is a big hack!

If you’re interested you can find the (VS2008) source code here.

There is also a link to a great presentation that they (Michel and Maxime) did at the French BizTalk User Group. The presentation is in the French language. For me that means I had to use the translation dictionary a lot .

Since then I received a lot of emails and comments from people that where unable to get the sample up and running. I must admit there are quite a lot manual steps to get from the downloadable source to a running BizTalk application.

This is why I updated the article. It has links to three downloads now:

• - the sources
• - installable MSI file
• - detailed setup instructions for the MSI file.

I hope this will make life easier for people who want to try it out.

To bad as I use the version number to quickly lookup what version of BizTalk is installed on a certain (virtual)box. I mainly use the following powershell command to get the version info from the registry:

get-itemproperty -path ‘HKLM:\SOFTWARE\Microsoft\BizTalk Server\3.0′ -name ProductVersion

I have published a version number overview table in the past each time a new version or SP was released:

BizTalk Server version number overview (updated for BTS2K9)
BizTalk Server version number overview (updated)
BizTalk Server version number overview

The table ‘BizTalkDBVersion’ in the BizTalk management database also holds version information for the product and the database. This table is also not affected by the SP installation.

I’m sure this will be fixed in the final version. For now my table looks like this:

Every now and then you need to do some BizTalk management things in code. This is where the classes come in very handy.

Last week I wrote a small sample to show and explain the use of the classes to a co-BizTalker. He was excited about this but told me it would be a good thing if we provide some more sample code. I’m sure we (me and the great people I work with on this project) will come up with a lot more information when we release the final version. For now I will only post this small sample.

It shows that a lot of “BizTalk management work” can be done using only a couple lines of code. Also this is done in a uniform way (using a single API).

This is what the sample does:

• connect to a BizTalk group (management database).
• create an application.
• add a BizTalk assembly (resource) to the application.
• import a binding file.
• creates and exports an MSI file from the new application.
• Starts the application.
• Resets the BizTalk Host.

Here is the code:

As you can see below I only need 8 lines of code to accomplish it. I don’t know how much code this would take when it was done using the “regular” API’s but I’m very sure many more lines are needed.

If you like this please download the beta from codeplex and start using the classes.

We´re currently very close to releasing the first beta of this new version.

The provider now uses a separate object layer for BizTalk management tasks. The layer is called ‘BizTalk Factory Management Automation’. This new architecture allows us to focus on provider functionality in the provider codebase and delegate all BizTalk related stuff to the object layer.

Nice things:

• The object layer hides us from all the different technologies and API’s (WMI, OMExplorer, etc.) that you can come across when doing BizTalk management from code. The object layer is a wrapper around all this stuff and provides a uniform way to do BizTalk management from .Net code.
• It is not solely aimed and meant for the provider but can be used stand alone or from your own projects.

The object layer is already in Beta and can be downloaded from here. Samples and instructions on how to use this will be provided later.

A big compliment to Maxime who is mainly responsible for the object layer and did almost all of the coding!

From a developer perspective however there is a nasty side effect that you should be aware of. This is caused by the fact that there is no MS virtualization tool running on a client OS that supports 64 bit guests.

Most developers follow the best practice to do development in a virtual environment and use their host for e-mail, word processing, etc.

In my case the host was Windows 7. I used my server running hyper-v to initially build the a new virtual BizTalk 2009 development box running on Windows Server 2008 R2. After that was done I wanted to transfer the .vhd to my laptop and run it from there.

As far as I know the MS options for running the VM are:

• Microsoft Virtual PC 2007
• Microsoft Virtual Server 2005 R2
• Microsoft Windows Virtual PC (currently RC)

As none of these options support 64 bit guest meaning I can’t use a Microsoft Virtualization tool to run my new Windows Server 2008 R2 development environment. Hyper-V supports 64 bit guest OS but doesn’t run on Windows 7.

The “not so nice” solutions left for me were:

• Use a non MS virtualization tool (like sun virtual box or VMware) that support 64 bit guests.
• Replace Windows 7 on my host with windows server 2008 hyper-V.
• Rebuild the virtual machine and use another 32 bit OS.

I could easily fix this by repairing the BizTalk installation but I must say I do encounter more strange problems with BizTalk 2009 and Visual Studio 2008. It looks like fixes and patches to VS mess up the BizTalk 2009 functionality.

This is probably related to the problem (and solution) described in Matt Milners post. I didn’t try it but I think Matt’s registry solution would have also fixed the issue above.

First some obvious commands:

net stop “BizTalk Service BizTalk Group : BizTalkServerApplication”

net start “BizTalk Service BizTalk Group : BizTalkServerApplication”

iisreset /stop

iisreset /start

Use these commands to respectively:

• stop host instance
• start host instance
• stop IIS
• start IIS

PSService

PSService.exe makes it easy to start and stop services on remote systems. If you for example want to stop or start the SQL Server agent on a remote SQL server hosting the BizTalk databases you can use:

psservice.exe \\MySqlServer stop SQLServerAgent

psservice.exe \\MySqlServer start SQLServerAgent

PSService is part of the SysInternals toolkit and can be downloaded from here

Clear event logs

I also wanted to start my tests with clear and clean eventlogs. I found the WMI ‘Clear event log’ script by Mark Fairpo for this purpose. To clear event logs I used this command:

ClearEvt.wsf Application Security System

Performance monitoring

To be able to monitor and see test results after the has finished we need to define some counter logs in performance monitor. Those logs store performance counter information during the test. Examples are memory usage, network usage, BizTalk spool table, etc, etc. You can start and stop those logs from the command line using the logman.exe tool.

The commands are:

Logman.exe Stop “BizTalk Performance Counters”

Logman.exe Start “BizTalk Performance Counters”

This will stop and start the counter log named “BizTalk Performance Counters”

Clear Messagebox and Tracking databases

Finally it is very important to start each test with a clean messagebox and tracking database. You easily achieve this by executing the following query:

USE BizTalkMsgBoxDb
GO

EXEC dbo.bts_CleanupMsgbox
GO

USE BizTalkDTADb
GO

exec dbo.dtasp_CleanHMData
GO

Be aware that the first mentioned stored procedure doesn’t do anything by default (it is an empty stored procedure). You need to load it the first time from the file ‘msgbox_cleanup_logic.sql’ located in the ‘Schema’ sub folder of the BizTalk program folder. Also be careful with the parameters you provide to this stored procedure. If you use the wrong parameters all subscriptions will be wiped and you’ll have redeploy all BizTalk applications.

If you put the above SQL statements in a script you use it from the command line, like:

sqlcmd.exe -S MySqlServer -i d:\testscripts\init_bts_db.sql

I hope these commands are useful to other people too.

When I want to mimic the characteristics of the BizTalk production environment in a test lab as close as possible, LoadGen is the way to go. The load, in terms of number of messages per time unit, can be easily set and tuned by changing values in the LoadGen configuration file.

As said before I want to get as close as possible to the production environment. This also means that I want to use messages that are close to (or better equal to) the production environment.

LoadGen comes with template functionality that you can use to change the message content before passing the message on to BizTalk. This functionality is a bit limited to my opinion.

What I wanted to do is copy a batch of messages from production and use them in my test. To be able to do this I had to write a simple ‘MessageCreator’ component for LoadGen.

A ‘MessageCreator’ is a component that implements the ‘IMessageCreator’ interface found in LoadGen.dll. The ‘MessageCreator’ is called whenever the LoadGen run-time needs a new message. Its task is to create the message and pass it back to the run-time. After that the run-time passes the message to the configured transport components. (Actually the ‘MessageCreator’ fills an internal queue from which other threads dequeue messages).

The default ‘MessageCreator’ that comes with LoadGen is only capable of using a single file for input so I created a new one that:

• - builds a list of files in a specific folder
• - random picks a file from the list
• - uses this file to create the new message

The code is straightforward. The ‘CreateNewMessage’ is the main method responsible for creating the message:

It calls two helper methods ‘CreateFileList’ and ‘GetFileFromList’:

I used the following LoadGen configuration file for my test:

The ‘MessageCreator’ is put in synchronous mode to make sure it create unique messages for all the client threads.

The ‘SrcFilePath’ path contains a pointer to a file where I actually only need the folder. Unfortunately LoadGen fails when there is no file at the end of the path. Internally I only use the path. The same applies to ‘TemplateFilePath’. I don’t need that here but the run-time for this setting.

In this sample I skipped the implementation of the ‘OptimizeFileSizeLimit’ property to keep things simple. I also did not need in in my test. The ‘OptimizeFileSizeLimit’ can be configured in the configuration file and basically tells the LoadGen run-time whether or not it is dealing with a large messages. If this is the case the creation of the message might be performed in another, more optimized way. The samples provided with LoadGen use this to skip file transport component and directly stream from the message component.

I have a very simple schema and corresponding instance:

Above message travels through BizTalk using the following path:

receive port -> orchestration -> send port

I defined the ‘Firstname’ element as promoted property to have it promoted by the out-of-the-box XmlReceive pipeline. After that the message is sent to the message box and picked up by the orchestration to do some processing. Next the orchestration will send the message to the message box again using a send shape. Finally the send port picks up the message and sends it to some URI using an adapter.

In the send port I used a custom pipeline containing a pipeline component with this code:

 After the message has been picked up by the send port the messaging engine will alter the context of the message. The ‘Firstname’ element that was promoted before is not promoted anymore now. To prove this I call the IsPromoted method for the specific property (see code above).

So what if you want to know if the property was promoted before? This is where the ContextPropertyType object comes in. Together with the ‘GetPropertyType’ method on the context object I’m able to find the status of the property.

Running this example shows the following output in DebugView:

 As you can see the PropertyType has a value equal to ContextPropertyType.WasPromoted. In the above sample I used an orchestration in the middle but the same will also apply to messaging only scenarios.

Although I recently made use of this API while implementing a very simple repair and resubmit feature, there might not be so many real world scenarios where one would use this. At least I couldn’t think of many. (Suggestion? Ping me, please.).  Also be aware that is an undocumented feature for which the BizTalk help states it is for internal use only. This basically means Microsoft will not guarantee any backward compatibility for this when new versions or SPs are released.

 ”If the BizTalk Server computer is running ASP.NET 2.0 or later then you can set autoConfig=truein the processModel section of the Machine.config file to automatically configure the following settings to achieve optimal performance based on the machine configuration”

This totally contradicts the following statement in the best BizTalk book around:

“ASP.NET 2.0 introduced a self-tuning feature to counter this issue whereby it will set the maxConnection value to a sensible value. This only applies to HTTP or Web Service requests created within ASP.NET 2.0 not BizTalk, so the above still applies to BizTalk solutions.”

I have tested this on my development box and it turned out that the MSDN content is not correct. The ’autoConfig’ setting does not apply to BizTalk.

I didn’t take  much time for us to decide to join forces. We are currently in the process of merging the two source bases and planning for the next release. 

I’m very happy to have the chance to work together with Maxime and Michel on this project. I also think the merge brings benefits like higher (code) quality and  higher release frequency.

This  next release will be the one and only PowerShell provider for BizTalk (I hope) and will be available on codeplex. My version will be downloadable from here until the first joined effort release is available.

In particular I wanted to examine the context properties of message stored in the spool table. The reason I wanted to do this is a little bit irrelevant for now and might eventually come back in a future post or article.

Anyway, the spool table has a very simple structure:

CREATE TABLE [dbo].[Spool](
    [uidMessageID ] [uniqueidentifier] NOT NULL,
    [UserName] [sysname] NOT NULL,
    [PublishingServer] [sysname] NOT NULL,
    [OriginatorSID] [sysname] NOT NULL,
    [OriginatorPID] [nvarchar](256) NOT NULL,
    [dtTimeStamp] [datetime] NOT NULL,
    [dtExpiration] [datetime] NULL,
    [nvcMessageType] [nvarchar](128) NULL,
    [nNumParts] [int] NOT NULL,
    [uidBodyPartID] [uniqueidentifier] NULL,
    [nvcBodyPartName] [nvarchar](256) NULL,
    [nCounter] [int] NOT NULL,
    [imgContext] [image] NULL )

The column I was interested in is called ‘imgContext’. For each message this column stores an encoded serialized value of the collection of context properties, something like this:

0xC4E0906C1849D311A24200C04F60A5330500000074000000680074007 etc, etc….

For my experiment I wanted to work in code with IBaseMessageContext interface much like the way we do when coding custom pipeline components.

So the question was how can I directly create a IBaseMessageContext instance from ‘0xC4E0906C1849D311A……’?

After clicking around for quite a while in Reflector I was able to create a very simple console application that gave me access to the context of the message. The only thing I needed to provide was the Message ID which can be fetched from the column ‘uidMessageID’ or the BizTalk Administration Console. Here is the code:

Running writes the context properties to the console:

I hope this code will help someone who, like me, also has the need to go beyond the BizTalk borders

A couple of notes:

• This code is only for ‘research’ purposes and should never be used in production environments or any other purpose.
• If you want to do this. Make sure the message stays in the spool table. If the message is processed by BizTalk and everything went fine the message will be deleted from the spool table. To prevent this you can set a breakpoint on the processing orchestration, create an error or disable the BizTalk Sql agent jobs.
• The code was developed using BizTalk 2009 but should also work in previous versions.

The new table:

As mentioned in the previous post this information will probably also appear in the the final BizTalk 2009 documentation.

Usage

This is the declaration of a string variable ‘lastName’ in a very simple test orchestration:

This is the expression shape where the value of that variable is set to my last name:

This is the map that is executed using a transform shape right after the expression shape above. The map contains the variable retriever functoid. It has one parameter that takes the name of the variable to fetch.

Please pay special attention to the icon because that bloody thing took me 50% of the development time. The result shows why I try to stay away from UI development as much as possible.

Finally this is the Xml message returned from the orchestration via the file adapter.

Disadvantages

At first I was a little excited that I got this working. I did some testing with different orchestrations and it seems to work OK. After a while (and thinking this over) my excitement was tempered because I think the functoid has three big disadvantages:

1. Although questions related to this popup regularly in the BizTalk newsgroups I could not think of any real world examples. The sample above could also be implemented by using a message assignment shape after the map. In the message assignment shape the value of the variable can be assigned using xpath, properties or distinguished fields. The only way the functoid can be useful is when you need an orchestration variable value in a map to do some processing while the actual value is not mapped to the destination schema. But then again there are other ways to implement that. (Using a helper message and a multi message map). 
2.  The functoid code contains a considerable amount of reflection code. I didn’t do any performance tests but it is obvious that reflection comes with a cost. So in terms of performance it will probably be much better  to use alternative methods.
3.  This is probably not supported by MS. Mainly because it uses XLANG code which is normally hidden from the developers. 

These disadvantages make me conclude that this functoid is not very useful in real world scenarios. I really want to know what others think about this. So whether you agree or don’t agree please share your thoughts on this!

The other way around

Now that I figured out a way to access a variable it is a small step to take this a little further and build a functoid that WRITES the value of a variable in an orchestration. I didn’t implement such a functoid because of above mentioned points. I also think writing, as opposed to, reading is very tricky because you need to take things like serialization and locking into account.

If your still not convinced that you should not use this you can download the functoid “dll” from here.

Installation instructions:

• copy the .dll to the ‘Mapper Extensions’ folder which resides in the BizTalk installation folder.
• put the .dll in the gac.
• Open a map in Visual Studio, click right in the toolbox area and choose the functoids tab.
• Browse the the functoid dll in the ‘Mapper Extensions’ folder to add it to the toolbox.

The source is also available here. It is build using BizTalk 2006 R2.

After doing some research on the Internet I found out that it is possible and fairly simple to do. Actually it was already possible in BizTalk 2006. It was just something that I (and the co-workers that I asked) didn’t know. So please stop reading if you are already familiar with this

A typical  scenario is a pipeline that consists of a XmlDisassembler and this custom RIP pipeline component. The interchange is received in the pipeline and disassembled into separate body messages by the XmlDisassembler. After that, for each body stream some processing is performed by the custom pipeline component. If this processing fails the single body instance needs to be suspended instead of the whole interchange.

There are basically three things you must do in your pipeline component to enable this:

Write property “SuspendMessageOnRoutingFailure”

The property “SuspendMessageOnRoutingFailure” needs to be written to the context of the message in case the processing of the message in the pipeline components succeeds. The value of the property needs to be set to true.

This property tells the messaging engine that each body message needs to be suspended in case of a routing failure. If you don’t write this to the context you’ll get a routing failure report for each body message while there will only be one suspended message (the whole interchange).

Write property “MessageDestination”

This property needs to be written to the context of the message when the custom pipeline components has decided that the processing on this specific body message has failed.

The value needs to be ‘SuspendQueue’ so that the messaging engine knows this particular body message needs to be suspended.

Return a searchable stream

The messaging engine needs a searchable stream to be able to put the body message in the ‘SuspendQueue’. I don’t know the exact reason behind this but the engine probable needs to rewind the stream for proper error handling. Anyway we just have to make sure we return a searchable stream when the body instance needs to be suspended.

I wrote a simple test application to test this. The first I did was adding a property to enable or disable the RIP feature of my component. Much like the setting for the XmlDisassembler (and XmlValidator in BizTalk 2009)

The execute method of the pipeline looks like this:

First the incoming stream is wrapped a XpathMutatorStream object to be able to receive an event when a certain xpath is matched. I use this to check whether or not the message is wrong (and should be suspended). In this sample the body messages each contain an attribute to indicate whether they are ‘good’ or ‘wrong’.

Next I create a virtual stream (which is searchable) to meet the requirement of returning a searchable stream.

In the third step I create a new output message containing the searchable stream.
If the message is ‘good’ I don’t care about RIP being enabled or disabled and just return the message (using the method ‘HandleHappFlow’). The only thing I need to do is write the ‘SuspendMessageOnRoutingFailure’ context property to the message as mentioned above.

If the message is ‘wrong’ and RIP is enabled I call the method ‘HandleRecoverableError’. The code of this method:

It just returns the message with the searchable stream, set some error information and writes the property ‘MessageDestination’ to the context.

Finally if the message is ‘wrong’ and RIP is disabled I throw an error so that the whole interchange will be suspended by the messaging engine.

The sample source can be downloaded from here. It was developed using the beta release of BizTalk Server 2009. As said before the sample should also work for BizTalk 2006. The solution needs to be manually converted back to a Visual Studio 2005 solution however.

BizTalk Server 2009 brings developers a lot of new functionality. In presentations and blog posts most of those new functions have already been described and shown.

One of the new things that is not or only briefly mentioned are the enhancements to Recoverable Interchange Processing (RIP) in BizTalk 2009. Also the help file (beta) of BizTalk 2009 doesn’t provide a lot of information on this (yet) and there are no samples.

I had some time to play with RIP in BizTalk 2009. The diagram below provides an overview of the very simple BizTalk sample application that I used.

This is what happens:

1). First a ‘family’ envelope XML message (also called an interchange) is received by a file receive port.

2). The receive port contains custom pipeline which in turn contains two default pipeline components; XmlDisassembler and XmlValidator.

3). The XmlDisassembler disassembles the family envelope message into separate ‘person’ XML messages

4). The XmlValidator component validates the individual ‘person’ messages before they are delivered to the message box.

5.) The receive port has failed message routing turned on. So if anything fails in the steps above a send port with a subscription on the failed messages picks up the messages and writes them to an error folder (left send port in the diagram). If everything works ok the disassembled messages are saved in the out folder by the another send port (send port at the right in the diagram).

Another thing to note is that I set the recoverable interchange processing option to true in the properties of the XmlDisassembler.

RIP in BizTalk Server 2006 R2

First lets look on how this works in BizTalk Server 2006 R2.

When I post an invalid message (see message below; the second node has an invalid element named ‘Name’). The XmlValidator fails and the message is suspended, handled by failed message routing and finally written into the error folder.

The interesting thing here is that the message in in the error folder contains the full interchange (that is the original message that I provided to BizTalk). This happens because the RIP feature only applies to the XmlDisassembler component (see ‘A’ in the diagram) and not to the XmlValidator component (‘B’ in the diagram).

RIP in BizTalk Server 2009

Compared to BizTalk 2004 RIP was a big improvement in BizTalk 2006. In BizTalk 2009 the RIP feature has now been extended to other components. The XmlValidator pipeline component now also has the option to turn op RIP via the pipeline configuration.

The invalid message above is now handled in a different way. The disassembled valid person messages (1 and 3,4,5) are now put into the out folder while the invalid person message (2) is put in the error folder.

So now instead of the whole interchange only the failed person message is suspended while the valid messages continue their way to the message box.

The RIP setting in the XmlValidator has no relation with the RIP setting in the XmlDisassembler. So the above sample will produce the same result if the RIP setting in the XmlDisassembler is set to false. 

Internally this works because the XmlValidator pipeline component has been extended with code that will return a message stream even when there is a validation error. This stream is accompanied with special information on the context so that the message agent will put the message in the suspended queue.

RIP and maps

In BizTalk 2009 it is even possible to extend this behavior to maps that are used in the receive port. To enable this you have to write a special property called ‘SuspendMessageOnMappingFailure’ to the context of the message.

Setting the value of ‘SuspendMessageOnMappingFailure’ to true will only suspend the invalid message instead of the whole interchange. I personally think it is a bit cumbersome that you have to enable this in code. One has to write a custom pipeline component just to turn on a feature. It would be nice if this could be enabled by setting a property in the map configuration section of the port using the BizTalk administration console.

Anyway the BizTalk 2009 Recoverable Interchange Processing feature is a nice extension to handle interchanges in BizTalk server. It can save you from a lot of custom coding in pipeline components.

There is Non Disclosure Agreement (which I didn’t read) presented to the candidates before the exam starts so I am probably not allowed to post a brain dump here

Anyway I think the exam is not so difficult if you, like me, work day to day with the product. Like all MS exams you should study on the features that are new compared to the previous version in this case BizTalk 2006 R1). These feaures are: RFID, WCF adapters and EDI/AS2.

It occurred when I was installing Windows Sharepoint Services 3.0 with service pack 1, which is one the prerequisites for the SharePoint Services adapter.

I wanted to get on with the installation and didn’t want to spend much time on finding a solution. So to get around this I copied the .MSI back to my host system (in my case Windows Vista (Business edition) 64 bit). After that I used the command:

Sharepoint.exe /extract:c:tempwss

to extract the installation files from the downloaded .exe. Finally I copied the extracted files back to my virtual machine and ran ‘Setup.exe’. This time setup completed without a glitch.

I suspect Windows 2008 Server or one of the BizTalk 2009 prerequisites I already installed prior to WSS 3.0 to be the cause of the problem. Please comment or e-mail me if you’re not lazy like me, but took the time to find a more elegant and satisfying solution . Until that time I hope I can help someone with this workaround.

Lets take the following schema and corresponding message as example:

In this sample the value of the ‘FamilyNumber’ element from the envelope has to be promoted to the context of each body message. Ok, very easy…… we thought.

This can be done using build in BizTalk functionality. Create a property schema, promote the element from the envelope, use the default XmlReceive pipeline to promote the value from the envelope to each body message and done.

After the first test run We were very surprised to see that none of the context messages of the disassembled bodies contained the promoted value. It took an embarrassing long time before we found out that the element to promote needs to be before the body messages instead of after the body messages.

It makes sense because the XmlDisassembler component in the XmlReceive pipeline works in a streaming forward only manner. Each time a complete body message is read from the stream it is delivered to the message box (or next pipeline component). So when the stream reaches the element to promote the bodies have already been delivered to the message box. This is why their message context does not contain the promoted value.

Changing the schema to:

will do the trick.

The PowerShell Provider for BizTalk lets you browse BizTalk as a file system. The primary goal for writing this provider is to aid me in my day to day BizTalk development work. With this in mind the first thing I wanted to be able to do is stop, start and restart host instances in an easy manner. I also wanted to replace my host manager tool with something with more flexibility. You can read more on host instances and PowerShell here.

Download link can be found here

Some features:

- Browseable BizTalk Adminstration Console like hierarchy and structure of artifacts. Read more here

- Tab completion saves you from a lot of typing

- Custom cmdlets aimed at BizTalk.

Installation instructions:

prerequisites: PowerShell 1.0 installed, BizTalk Server 2006 R2 (haven’t tried R1 yet) installed and configured.

steps:

1. run ‘PowerShell.BizTalk.msi’ and follow instructions.

2. Start a new PowerShell shell.

3. Type the command:

Add-PSSnapin PowerShell.BizTalk

4. Optionally verify that the snap-in is loaded by executing the command:

Get-PSSnapin

You should see the snap-in in the list of loaded snap-ins (see example below).

5. Add a connection to the local BizTalk installation by executing the command:

New-PSDrive LocalBTS BizTalk LocalBTS: -scope global -server <ServerName> -database <DatabaseName>

Replace <ServerName> and <DatabaseName> with the name of the server and database with the name of the management database.

For example:  New-PSDrive LocalBTS BizTalk LocalBTS: -scope global -server BizTalk01 -database BizTalkMgmtDb

This will create a new PowerShell drive object pointing to the BizTalk management databases.

Automatically load the snap-in:

Step 3,4 & 5 need to be executed every time you start a new PowerShell and want to use the BizTalk provider. To prevent you from typing the commands every time you can of course create a personal profile script. This will be loaded every time a new PowerShell shell is started. In order to do this:

1. Create a new folder in your ‘My Documents’ folder. Name this folder ‘WindowsPowerShell’.

2. Create a new text file in the folder created in step 1. Name the file ‘Microsoft.PowerShell_profile.ps1′.

3. Set the contents of the text file like below and save it.

When opening a new shell the snap-in should be loaded automatically.

Supported artifacts:

Compared to BizTalk Adminstration Console this release of the provider has a limited set of artifacts that are browseable. Also each artifact has less properties than you’re used to in the console.

Limitations in this version:

- The provider makes heavily use of ExplorerOM model. As far as I know this is only supported on 32 bit platforms. So be aware when using the provider on a 64 bit platform.

- The provider does not (yet) (fully) support connections to remote BizTalk installations. Some things like e.g. browse applications work, while other browse host instances will not work.

- Make sure you have enough rights (BizTalk administrator) before using the provider. It does not check role membership or authorizations before executing code so insufficient rights may lead to nasty errors.

- Many other……

Please send me an e-mail or post a comment if you have questions, remarks, bugs, etc…

In BizTalk Server 2009 it looks like Microsoft has continued to move functionality from the Health and Activity (HAT) tool to the BizTalk Administration Console (MMC).

When we went from BTS2K4 to BTS2k6 we saw that some of the tasks we did with HAT (and other tools) on 2k4 now could be done with MMC on 2k6.

The biggest improvement for me personally was the inclusion of the subscription viewer function in the MMC. I still have nightmares about the user interface of the 2k4 subscription viewer tool .

In BizTalk Server 2009 Microsoft took it a step further and removed the HAT tool. This means the remaining functionality is now also included in MMC.

To be more precise the HAT is not completely removed you only don’t need to start it as a separate tool anymore. If you for example use things like ‘Message flow’ or the ‘Orchestration debugger’ you’ll still get a HAT style window. Also the task manager shows a process called ‘BTSHatApp.exe’ which I think I saw somewhere before . Of course I also couldn’t resist typing ‘BTSHatApp.exe’ from the start prompt, but that results in the following error message:

Anyway I think it is a good thing because we now really have one tool (or at least one entrance point) to do everything. I also never liked the HAT user interface very much.

There are (or were) basically three functions/queries that I use in HAT:

• Most recent 100 service instances
• Messages sent in the past day
• Messages received in the past day

The query expression pane in the 2k9 MMC now has values for ‘Tracked Message Events’ and ‘Tracked Service Instances’. This will allow me to do the same thing in MMC that I previously had to do in HAT.

Screen shot 1 shows the ‘Most recent 100 service instances’ in the 2k9 way. Screen shot 2 shows the ‘Messages received in the past day’.

Screen shot 1: Most recent 100 service instances.

Screen shot 2: Most recent 100 send messages.

I also promised to update the overview as soon as a new version of service pack is available. To keep the promise I added the recent beta of BTS2k9.

This information is now also provided by Microsoft in the BizTalk Server 2009 help. See help topic ‘What Version of BizTalk Server Do I Have?’.

You can find it here.

Hope you find this useful

I would be great if BizTalk developers or administrators:

• just could walk through the biztalk artifacts in the same way we do with file folders.
• have a (rich) set of powershell cmdlets aimed at BizTalk.

In other words we need a: Powershell provider for BizTalk. Searching the internet resulted in nothing. I found one initiative but that did not seem to have started and there were also no downloadable bits.

Anyway I decided to code a BizTalk provider myself. I have copied the artifact hierarchy tree used in the BizTalk Administration Console as much as possible (see screenshots below). The provider also supports tab completion which makes it very easy to navigate through the BizTalk artifacts.

The screenshot below contains an example of a BizTalk cmdlet to stop a specific host.

Although I’ve been very, very busy with this in the last couple of weeks there is still a lot of work to be done. So why am I telling you about this when there are no downloadable bits yet. There are two reasons:

• I hope there are BizTalkers out there that use Powershell and want to test my provider eventually. (If so drop a comment or e-mail).
• I want to make a list of first release cmdlets that should be included. (Suggestions? Let me know!)

I shared this interest with him an decided to spend some time on a small test solution to measure the performance. Because I wanted to be sure about my results (and also because this was his blog subject ) I asked the real expert (Yossi) to validate my results.

With a (little delay ) Yossi posted about the results.

Thanks Yossi.

Although it was written for BizTalk Server 2004 (.Net v1.1) it also works for BizTalk Server 2006 R1 and R2.

Apparently some people still use it because now and than I receive questions from people asking where it can be downloaded. Because I am not sure if it is still available from the old location, I’ll put it on my skydrive here.

I’m currently working on a new version of the tool. I hope to have it finished soon.

While googling for a solution to this problem I found a number of people having exactly the same problem. Besides re-installing the entire machine there is no real solution mentioned. Because I didn’t want to reinstall my machine and because I think there are different causes for this problem I decided to take the hard way. Below is a description of the steps I took to save myself a lot of time:

I started by looking for the SQL Adapter binaries. No rocket science is needed to find out that those are located inside the ‘Microsoft.BizTalk.Adapter.SQLAdmin.dll’ assembly. This assembly can be found in the BizTalk installation folder.
The next step is to use Reflector together with Reflector FileDisassembler AddIn to ‘extract’ the source code of the wizard.
After fixing some references and other minor things I was able to recompile the .dll.
Next thing is to tell Visual Studio debugger to break on all exceptions instead of having the code handle them.  (You can do this in the dialog which appears by pressing CTRL-ALT-E in Visual Studio).
By starting a new Visual Studio instance and attaching the other one (with the extraced source) to it I was able to debug the wizard.
When I reached the point where the wizard was disappearing before I now got an exception telling me exactly what was wrong (see picture below). 

SQL Adapter FileNotFoundException

It turned out the installation of MSXML on my box was wrong/corrupted. I got a FileNotFoundException which basically told me that I had to (re)install MSXML. After reinstalling MSXML and throwing away the decompiled source (to keep MS people happy ) the wizard was working OK.

I was very happy that I reduced completely reinstalling the BTS box to just reinstalling MSXML.

One option is to use the undocumented BizTalk Document Specification (DocumentSpec) API. Although this way of creating a message has some obvious disadvantages (see also comments to Yossi’s blogpost) it can be the best option in certain specific situations.

On of the things you need to do to create a message instance is to create a new instance of the DocSpec class. The constructor of the DocumentSpec class takes the .Net/BizTalk schema type name and the assembly display name that contains the schema as parameters. The code should look something like this:

This code works fine for single root schemas but you’ll get an error of you try this for a multiroot schema (an xsd that has multiple root nodes):

So how can I make this code work for multi root schemas? It took me a while to find out but the solution is simple. Just add the name of the root node of the message you want to create preceeded by a ‘+’ sign:

This will provide the DocumentSpec class with all the information needed to create a message instance. 

• Visual Studio is not always right. In rare cases you’re right and Visual Studio is not!
• You don’t have to use an atomic transaction scope anymore when you want to use the “Call Rules” shape.

Besides a paper version, a digital version is now also made available by Microsoft NL. It can be downloaded from here.

Since .Net Magazine is local the article is only available in dutch.

I have read about people with similar problems in the BizTalk newsgroups. I recently encountered this problem too when I wanted to write a little helper class that determines which BizTalk version was installed. I knew that there is a BizTalk version number stored in the registry. The ‘ProductVersion’ key located in ‘HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\BizTalk Server\3.0′ contains this information. The real problem was however how to match this number to a BizTalk version or Service Pack. After searching the net for a long time I decided to make a list myself. I was lucky that I had been to lazy to clean up my harddisk and delete old Biztalk 2004 vpc’s .

Here is the list: 

I hope this will save some people a lot of time when they are in the same situation I was. Also there might be better ways that I don’t know (maybe ExplorerOM or WMI) to retrieve version information. Please drop a comment in that case.

There is also a version number stored in the BizTalk SQL Server databases. Most of the BizTalk databases contain a table ‘BizTalkDBVersion’ which contains besides the database version number also the product version number. The strange thing is that this number was not altered by the installation of the service packs (SP1 and SP2) on BizTalk 2004. After the upgrade the table still showed ‘3.0.4902.0′ as the product version number while the registry version number was changed.

I’ll try to keep this list updated when new versions or service packs are released.

There is an updated post on this topic. You can find it here.

Oops, this must be the longest and worst blog post title you have ever seen. Let’s quickly make clear what I mean:

Recently I needed to update multiple elements using a single rule in the Business Rule Engine (BRE). To most BizTalkers this is nothing special. The thing that complicated this particular scenario was that the elements to update where in different hierarchical levels within the xml instance and thus had different parent nodes.

See the following XML and corresponding XSD instance for an example of this scenario:

]]> http://biztalkmessages.vansplunteren.net/2008/06/11/updating-multiple-nodes-with-different-parents-and-hierarchical-levels-using-the-biztalk-bre/feed/ 5 biztalkmessages