I did this article ages ago and have just found where I had misplaced it.
Anyway the intention is to provide some introductory information about BAM (Business Activity Monitoring) for those who are relatively new to BizTalk and to help people to decide if it is suitable for their project. I will also provide some links to useful resources I have used before when working with BAM.
What is BAM?
Business Activity Monitoring (BAM) is a feature of BizTalk which allows you to obtain information and metrics around your executing processes. To do this you define an observation model of your process which indicates which metrics you are interested in. This information is then captured during the execution of the process and can be queried by business users. BAM has a number of tools which lets different types of users interact with this view of the process.
What do they do
Orchestration Designer for Business Analysts
These tools help the Business Analst to define the data to capture and to define how the data will be aggregated
Tracking Profile Editor
BAM Web Services
These tools/components allow the developer to map the observation model to the physical implementation of the process. An example of this would be indicating that a particular shape in an orchestration maps to a mile stone defined in the observation model
The BAM Web Services allow a developer to integrate custom applications with BAM so they can query inform from BAM
These tools allow a business user to access BAM information in tools which they are used to such as Office and Sharepoint
For a newbie some of the common questions which might be asked are:
What's the difference between BAM and HAT?
BAM is intended for business users to monitor business processes. HAT is intended for administrators to monitor and track the health of the system.
What's the difference between BAM and BAS?
· BAS provides information workers with features in tools such as Office/Sharepoint which allow them to interact with and manage processes involving Business Partners.
· BAM allows business users to access information about processes which have executed or are executing and aggregations of this information
Why use BAM?
The following are some of reasons why a client would like BAM:
Unlike application development, BizTalk development often does not have anything tangible which can be shown to a client. BAM will provide them with important KPI related information about their business processes and can help them to see the benefits of the solution and help them confirm they are getting a satisfactory ROI.
A business user would be able to see real-time information about their business processes enabling them to be agile and responsive to ensure they are getting the most from their business. An example of this might be seeing that we are starting to build up a back log of orders in the warehouse so they could draft in more resources to help clear this temporary bottleneck.
BAM alerts can also provide the trigger to automate some of these decisions.
BAM is a built-in feature of BizTalk which a company who invests in BizTalk will already have made. The cost of mining information about processes with BAM would be significantly cheaper than using a third party tool. Although I’m sure it would be possible to capture the information in BAM and then export it to third party tools if required.
BAM as a reporting solution provides a familiar environment to business users by taking advantage of Office and Sharepoint
BAM can be implemented with no code in lots of cases
One of the cool things about BAM is that often a project chooses not to implement it as a core requirement, but an organisation can always come back to their established BizTalk solution and look to add BAM as an additional layer to the solution where they could make the information about processes available without having to deploy a new version of their BizTalk components which are already running. You would simply deploy the BAM artefacts to the existing BizTalk group.
How do I spot an opportunity for BAM on my project
1. If a client is considering BizTalk against a potential other product a demonstration of how easily BAM can be implemented in the solution and the valuable information that it can return can be a good way to promote the benefits of BizTalk.
2. If a client is considering a BizTalk solution but is also considering a custom approach which is a common case due to the cost of BizTalk, then it can be useful to discuss how the custom solution would be monitored and how the data would be made available in a consistent manner. This kind of thing would quickly increase the complexity of the custom solution and would make the fact that BAM is available out of the box and easy to implement into your processes an attractive proposition.
3. Although it is recommended to begin thinking about BAM from the start of your project BAM can be added to existing BizTalk solution later. This provides the opportunity for additional work on a project where BAM was not originally in scope. When you hear a client wanting to have a better understanding about their business processes or when you hear them wanting to have volumetric type information relating to their processes this is an opportunity for us to talk to them about adding BAM to their existing solution.
4. If you find that people struggle to understand the role BizTalk is playing in a solution or thing that BizTalk is a magic box which makes things happen and people just generally don’t understand what it does then BAM can be a good way to give people visibility to what BizTalk does in a way they might understand.
5. If you are a consultancy then BAM is a fantastic opportunity for you to sell another project into your existing customers if BAM was not an original requirement of the other projects. You will already know their processes and if you can demonstrate the value of the information you can get your client then you have another potential project here.
Sometimes I think the best way to approach this is to take an existing process which you thing would benefit from being BAM enabled and then just do it as a POC. Showing the business the value of the information and how easily it can be accessed combined with the low cost involved in getting it is probably the best way to get people interested in BAM. I think it is common in these kinds of situations that when a business user sees something like this that will help them they will often come back and ask for more.
Components of BAM
This section will provide some terminology and basic information about some of the background components which make up BAM:
The BAM Portal is an ASP.net web application which allows business users to view the information held in BAM. In addition to this users can also create alerts so that they are informed when a particular event happens.
BAM Observation Model
The BAM Observation Model is a high level view of the business process. The BAM Observation model is normally defined by a business analyst using the add-in in Microsoft Excel. The Observation Model defines the milestones and data items to collect, information about any aggregations and how they should be presented to users. The Observation Model contains the following items:
BAM Activities - See below for more info
Aggregations - An aggregation can be either real time or scheduled, see below for more info
o Aliases - There will be business terms defined for common data items to allow business users to understand what value a process may be presenting them. This is intended to ensure that developer naming conventions which may not have meaning to a business user are not displayed.
o Duration - The period of time over which the activity is monitored
o Milestone Groups - Sets of milestones which start or end the process
Aggregations are defined by the business analyst as pivot tables in Excel when they are creating the BAM Observation Model. The aggregation(s) define pre-calculated summaries of data which will significantly improve query response time compared to having to calculate thousands of rows of data.
Aggregations are implemented in the database as OLAP cubes.
Real-time aggregations (RTA) allow a business user to see information on business processes that will change through out the day (for example if you refresh your screen the data may have changed). This is because you are seeing data based on the activity tables which contain currently executing activities. RTA data is available via dynamically created views in the BAM Primary Import Database.
This can be very useful to a business but there are some things you need to be aware of:
· Real-time aggregations significantly affect the write speed of BAM to the activity tables therefore you should only consider RTA for the most important aggregations.
Scheduled Aggregations will periodically take a snap shot of the currently active activities and also the completed ones and then save this information to allow it to be queried later (For example a user may see a snapshot of this mornings order). Scheduled Aggregation data is available via dynamically created OLAP cubes in the BAM Analysis Database.
An activity represents a unit of work within a business process, for example a loan application. The activity shows two key sets of things.
· Milestones - These are date based events which indicate the history of the activity such as "order completed".
· Data items - These are context type information about the unit of work such as an application number.
BAM Management Utility (BM.exe)
BM.exe is the tool which is used to deploy a BAM Observation Model to BizTalk.
Tracking Profile Editor
A tracking profile is a map between the BAM Activity and the BizTalk implementation of the activity. The Tracking Profile Editor (TPE) is a tool that allows a developer to map parts of the activity onto BizTalk artefacts such as Orchestrations or Ports.
The BAM Inteceptor is used to instrument your application to capture data of interest. There are also interceptors available for WF and WCF.
Tracking Profiles Management Utility (bttdeploy.exe)
This is a command line tool used to deploy a Tracking Profile to BizTalk.
There are a number of databases which BAM will install. The details of these databases can be found at the below links:
BAM Implementation Workflow
Now we are aware of some of the components of BAM we now need to know how it all fits together. This section will identify who uses what and when in the process of creating a common end-to-end BAM solution.
The following diagram shows how this fits together.
(Image courtesy of BizTalk documentation)
1. To begin with the Business analyst will use Excel to create the BAM Observation Model by using the add-in in Excel to create activities and aggregations associated with the business process.
2. BM.exe is then used to deploy the BAM Observation Model. This will do the following things:
o In the BAM Primary Import it will create the Stored Procedures, Triggers and SSIS packages
o The BAM Archive Database is defined but the objects are not created until the SSIS packages are executed;
o If it is a Scheduled Aggregation then objects are created in the BAM Star Schema database.
o If it is a Scheduled Aggregation then the appropriate OLAP cubes are created
3. The developer then uses this observation model to create the tracking profile using the Tracking Profile Editor
4. The tracking profile is deployed to the BizTalk Management Database
5. When the business process is executed the information relating to the process is now stored in the various databases depending on how the Observation Model was defined.
6. A business user can use the BAM portal or Excel to view data held in the database or in OLAP cubes to review and monitor the business process
7. If required the Business User can choose to create alerts related to data held about the business process
The above describes the most common way in which BAM is implemented. There are some alternatives on this which are described briefly below:
· The developer may choose to report data to BAM using the BAM API rather than creating a Tracking Profile.
· A developer may choose to use the BAM web services and then display BAM data in their own application.
· A business analyst can create visual diagram of the business process using the Orchestration Designer for Business Analysts (ODBA) which is the BAM Definition which can be imported into Excel to help complete the creation of the BAM Observation Model.
Barriers to using BAM
Although BAM is a really useful feature for BizTalk projects I think there are a few things which I would consider to be barriers which contribute to the fact that BAM is not implemented on more projects. These include:
Although there is no cost for BAM in terms of licensing, I find that most projects focus solely on the core requirements to implement a process or integration. They try to do this while minimizing cost. Based on this any consideration of a value added feature such as BAM is often descoped from projects.
The key way this kind of barrier can be handled is to get buying from the project sponsor. If this stakeholder can understand the benefits and justify the extra work then BAM could be made one of the deliverables or it could be used to measure success of the project.
If you have a KPI or SLA success measure for your project like we want to be able to process n orders per day then this is something where BAM can help you.
Although the user groups described above indicate that there are tools for Business Analysts a lot of projects don’t really have analysts who do the roles/tasks described in this article
I think if you are an organisation who is committed to an IT strategy involving BizTalk then it is worth training one or more of your business analysts in how to use the tools to implement BAM. Unfortunately I think this is something Microsoft have missed a trick because I’m not really aware of any training for Business Analysts on how to use BAM.
Build and Deployment Process
I think in terms of Development process there aren’t many samples which discuss using BAM in the build and deployment process for a BizTalk project.
I think I might look to wrapper some of the BAM deployment stuff up in MsBuild tasks and see if I can add this to the BizTalk build generator and do another post with some samples to help with this.
Sources of Information
This section contains a list of resources which may be useful in increasing your knowledge in BAM.