Continuing the series on testing in BizTalk projects this article is about testing orchestrations.
From projects I have seen or through speaking to other developers I have come across the following techniques for testing orchestrations:
1. Testing by debugging them in HAT
Im amazed how many times this comes up when I have interviewed people. Ok first thing Debugging != Testing. Secondly the problem here is the "tests" are manual and there for not easily repeatable and prone to error.
2. Manual Testing
I have come across projects where the team had written documented test cases and then taken manual steps such as dropping a file in a folder and seeing what has happened. I think this approach is poor because if you write the test cases in XML using the BizUnit format rather than in a word document you suddenly have a test case which is just as readable but can now be automated. This means your tests are much less expensive to run, maintain and repeat
3. Tested as part of the larger process
Ochestrations are difficut (if not potentially impossible) to test in isolation because then depend on the message box and need to be deployed to BizTalk in order to be executed. For this reason a common approach is to test at a higher level of testing the overall process in BizTalk which will include the orchestration and any ports, rules etc. These tests would hopefully be done with BizUnit and because the tests were executed and successful it is assumed that the orchestrations within the process are tested correctly too.
4. Tested with specific bindings for tests
This approach is less common and is very similar to approach 3, however the key difference is that the tests are executed with a set of specific bindings which are aimed to simplify testing and allow the tests to focus on quality test coverage of the orchestration.
A seperate set of tests with more realistic bindings would then be tested later to validate the overall process through BizTalk.
This approach can be useful expecially if you have a complex orchestration which would be difficult to test fully with the normal bindings. The main difficulty of this approach is the additional cost and maintenance to perform these extra tests, although it will probably pay off long term if done in the appropriate places.
Recommendations of approach
My preference on testing for orchestrations is to combine approach 3 and 4. Only using 4 in special cases. I believe this forms a pragmatic approach which meets our normal testing aims of being automated, repeatable, etc, etc
Im not going to go into detail of a sample for testing orchestrations as i intend to do a future post about testing the overall process through BizTalk. However one absolutely key thing with the testing of orchestrations is that you should be using the Microsoft SDC Orchestration Profiler tool. This tool will give you coverage statistics for the orchestrations and identify any missing test cases. This tool is available from the following location: