One of the technology backbones of my company moving forward is the use of BizTalk
to “fan out” messages from our ERP system (SAP). Instead of having hundreds of direct
interfaces to SAP by downstream systems who need real-time data, we can use BizTalk
to receive a single message from SAP (XI), and distribute it to all interested parties.
One issue brought up recently was “ordered delivery” and ensuring that if a downstream
system needs messages delivered in the order sent by SAP, that BizTalk honor it. So,
I set out to test a variety of scenarios using BizTalk’s built-in ordered delivery
capabilities.
Setup: I exposed a BizTalk schema as a web service to simulate our publishing
receiver from SAP. A simple Windows Form is sending messages to this web service,
and stamping each one with a sequence number. I then built a couple of web services
to act as “subscribers” of these SAP data events. These web services publish the data
to a database table, thus letting me see the exact order of messages delivered. Each
SOAP send port has a subscription based on message type.
Scenario #1: Send port, no ordered delivery
When a series of messages are published to the receiving web service and routed to
the subscriber without ordered delivery, the result may look like this:
As you can see, even on my single development machine, the order gets mixed up. This
is due to batch processing and the multi-threaded nature of BizTalk message distribution.
Scenario #2: Send port, ordered delivery turned on
To turn ordered delivery on, all we need to do is check a box on the send port (this
assumes that your inbound transport can deliver messages in order. Examples include
MSMQ, SOAP and HTTP).
The result in the database looks like this:
So you can see that all the messages are in sequential order.
Scenario #3: One send port no ordered delivery, one send port WITH ordered delivery
I wanted to prove that ordered delivery only impacted the corresponding port. When
sending the message into BizTalk with two send ports subscribing, the data looked
like this:
“Subscriber #1″ kept everything in order, but “Subscriber #2″ delivered messages a
bit more haphazardly.
Scenario #4: Non-ordered delivery send port, no retries, and error in service
So what if the service raises an error? When there are no retries, and the send port
doesn’t have an ordered delivery requirement, the result looks like this:
The bad message (#12) is simply skipped (because it is now suspended).
Scenario #5: Ordered delivery send port, retries enabled, and error in service
How about for an ordered delivery send port? What if there’s an error in the service
call, and retries are turned on for the port? Also, the “cancel if error” is turned
off. Check this out:
The messages queue up waiting for the first one to retry, and hopefully succeed. We’ll
see more of this in a moment.
Scenario #6: Ordered delivery send port, NO retries, and error in service
This time, there’s an error in the service, and no retries. Also, the “cancel if error”
flag is still turned off. The result is:
So the messages still get delivered AFTER the bad message has been encountered. That’s
expected, since we told BizTalk to keep going, even for the ordered delivery port.
Scenario #7: Ordered delivery send port, NO retries, “cancel if error” turned on,
and error in service
What if we reproduce the previous scenario, BUT, turn “cancel if error” on? This flag
can be set on the send port here:
What you’re telling BizTalk is that if any message fails for this ordered delivery
port, stop processing until this error can be corrected. This is useful if you’re
concerned that an “insert new contact” message failed, but you expect a “modify existing
customer” to be following. Clearly the “modify” message will fail unless the “insert”
gets figured out. The result of this?
Processing stopped after message #11 because message #12 failed. The send port is
still “started”, but you’ll find a suspended message. If you open it up, you’ll see
that all following messages are queued up until the offender gets resolved.
Scenario #8: Send port stopped, and restarted for a non-ordered delivery send port
What if we KNOW that a downstream system is unavailable (e.g. maintenance) and want
to prevent the inevitable failure of delivery? Maybe we want to shut off the send
port, queue up the messages, and once the unavailable system is back online, open
the distribution pipeline again. For a non-ordered delivery port, after restarting
a stopped send port, the output looked like this:
The messages are in a crazy order, as you can see.
Scenario #9: Send port stopped, and restarted for an ordered delivery send port
Final scenario. What if we stop an ordered delivery port, and then start it back up
later? Do we retain the correct ordering? When I sent five messages in, I got a suspended
instance with five messages:
If I send in three MORE messages, they actually get added to this suspended instance:
Once I finally turn the subscriber service back on, all messages were sent in the
same order received.
So there you go. Ordered delivery is a fairly powerful concept, and not particularly
hard to do with BizTalk. It greatly impacts performance because the send port won’t
send subsequent messages until a delivery confirmation is received, but, in many cases
that performance impact is outweighed by business requirements.