IBM Integration Bus sizing and performance

It's not the first time that I am been in a situation where I have been asked to evaluate how many cores would be necessary to run a load on IBM Integration Bus (IIB).

In this post I will provide you a way to make this evaluation based on public IIB performance reports.

Required information  

Before starting your evaluation, you first need to identify a set of flows that corresponds to ESB patterns that you would like to achieve: transformation, aggregation, protocols, logging, ...

You would like as well to define on what operating system the run time will run.

For these flows you will need to define the following information

• Kind of messages: XML, non XML
• Average size of messages
• Load peak expressed in transaction/sec. I usually use the peak since this is the throughput that you would like to sustain. 
• Operating system where the IIB will run

Note that the CPU type has also it's importance. Normally you would have to add a corrector factor to compensate the fact that you have a more efficient processor.

Performance reports

Once this information is available, you can start your performance evaluation computation.
For this we will use performance report that is publicly available. For IBM Integration Bus, go to the link message throughput. (good link to bookmark is the performance topic). For older version, you can find performance reports at the support pack link.
On the message throughput link select your operating system.

These performance reports have been measure using a processor: IBM xSeries x3690 X5 with 2 x Deca-Core Intel(R) Xeon(R) E7-2860.
For those that would like to know the weight associated to this processor, please have a look at the PVU information page. You will find that this processor corresponds to a 70 PVU processor (2 sockets with 10 cores).

You will find different type of patterns that have been tested: aggregation, coordination request/reply, message routing, ...
For each of these tests there is a table providing the message rate, the CPU busy and the CPU ms/msg.

The information that we will keep are the 

• message rate
• the CPU ms/msg

Performance evaluation

How can we use these information?

Let's take an example!

Imagine that you would like to deploy a flow doing aggregation with an average non-persistent message size of 2 kb.
This flow will process 460000 messages a day but there is a peak! During one hour in the day the flow has to sustain 720000 messages.

We would like to size the bus such a way that he will sustain the peak. This corresponds to a throughput of 200 trs/sec.
Lets have a look to the tables available on the performance report page. There is one for the aggregation:

	Non Persistent			Full Persistent
Msg Size	Message Rate	% CPU Busy	CPU ms/msg	Message Rate	% CPU Busy	CPU ms/msg
256b	4020.1	99.3	4.941	476.9	12.4	5.215
2kB	3430.6	98.3	5.732	707.8	19.7	5.559

For 2kb non persistent message, the processing of one message takes 5.732 ms of CPU.

The processing of 200 transactions per seconds would take 200 (msg/sec) *5.732 (CPU ms/msg) = 1146.4 (msg/sec)*(CPU sec/1000 * 1/msg) = 1.1464 CPU. 

So you would need to at least 2 processors to process this load. The total load of the machine with 2 CPU would be 57.32%.

It is not recommended to have a processor loaded at more than 70 % but here the 2 CPU will do the deal.

Of course you could have more complex situation such MQ, JMS, database, routing....

It is possible to approach more complex situation by adding the CPU load for each scenarios.
If my integration needs to be exposed as web service using SOAP messages, performs routing and transforms messages, I would approximate my CPU load by adding the corresponding load factor: 2.633 (SOAP) + 0.488 (routing) + 0.896(transformation).

This is not ideal since each scenarios include the protocol processing like MQ for routing.
In the previous reports that are available in the support pack page, more tests have been done and it is possible to build a spreadsheets to isolate the processing of each mediations.
For example there is a test made for MQInput-MQOutput (x ms/msg), MQI-Transformation-MQO (y ms/msg) and JMSIn-JMSout (z ms/msg). For a JMSI-Transformation-JMSO integration flow, you could therefore approximate the CPU load using: y ms/msg - x ms/msg + z ms/msg.
The performance in IIB V9 is higher than for V8, so using the old performance reports would provide you a good estimation with a security factor.

References

message throughput information about IIB in function of the operating system.
performance information, tips and hints.
support pack old performance reports.
PVU information page to find the PVU for a defined CPU.