Using AMQP for high volume integration with Hybris Commerce Solution

Introduction

Handling large amounts of data is always a challenge to any integration system. Our customer has a very specific requirement for Hybris integration with SAP ECC through their SAP PI system. The process should be able to handle thousands of orders per second from Hybris at peak business periods.
This blog describes an architecture in which AMQP broker (e.g., RabbitMQ) is used as an intermediate buffer system to absorb peak transaction volume and the Advantco AMQP adapter for SAP PI/PO integration.

Hybris integration options.

Hybris provides these three main options for integration with external systems:

• SOAP/RESTful web services
• Java Message Service (JMS)
• Spring Integration

Because the SOAP/RESTful web services and the JMS options are standard support by SAP PI/PO, these options seem to be good choices, however, they have their own limitations.

SOAP/RESTful web services option:Hybris allows us to expose a RESTful endpoint for exporting business data in JSON format. Our customer used the Advantco REST adapter to expose a RESTful service on the SAP PI system to receive incoming requests from Hybris. Although they noticed significant performance improvements comparing to the classic SOAP/XML services, their SAP PI system was not able to handle the thousands of requests per second due to the exceptional high number of threads that must be available to process these incoming requests. This exercise did provide a conclusion that Hybris REST/JSON is the correct way to export data as it is very fast, however, it’s not capable of handling thousands of concurrent requests at the same time.

Java Message Service (JMS) option:SAP PI does provide a standard JMS adapter but it does not have a native JSON/XML converter. In addition, JMS is not a perfect solution for cross-platform interoperability as most implementations relied on proprietary protocols.

The chosen architecture option.

To prevent flooding their SAP PI/PO with RESTful requests, the customer decides for a buffer system to absorb the high-volume incoming requests and have their SAP PI/PO system pull the data from the buffer system. RabbitMQ was the ideal solution as it is a very lightweight broker which can handle huge volumes of data and supports native JSON format.

To pull data from RabbitMQ, the customer uses the Advantco AMQP adapter which is fully compatible with RabbitMQ and has support for JSON/XML conversion. The number of application threads on the SAP PI system determines how fast data can be fetched from RabbitMQ. This scenario is different from the RESTful scenario in that the SAP PI system has control on how much data it wants to process at any time, independent of the data volume coming from Hybris.
This architecture setup satisfies the requirements to handle thousands of orders from Hybris while preventing their SAP PI from crashing because of the sheer amount of data.

Reference:

RabbitMQ : http://www.rabbitmq.com/

Hybris : https://www.hybris.com/en/

AMQP adapter : https://www.advantco.com/product/adapter/amqp

AMQP : https://www.amqp.org/

Please reach out to our sales team at sales@advantco.com if you have any questions.