Near real time Data Replication from Salesforce to SAP HANA

Salesforce provides APIs that enable external applications to receive events in near real-time mode. In this blog, we provide detailed configuration steps for two of these options: Platform Event and Change Data Capture (CDC). Platform Event is an excellent option when data from multiple objects are required as in the case of combining data from Opportunity with details from the Account object. Change Data Capture (CDC) provides a quick and configurable option to publish data of a specific object like Account or Contact.

The solution we provide here is based on SAP Smart Data Integration (SDI) tools which provides features to support data replication from external resources to SAP HANA. The two most important components in this context are the Data Provisioning Server, which is a native SAP HANA process, and the Data Provisioning Agent, which is a container running outside the HANA environment. The Advantco Salesforce adapter is deployed on the Data Provisioning Agent host and can be configured from the HANA environment duration a remote source is created. The remote source is the connection from SAP HANA to the Salesforce instance.

Preparations on Salesforce side.

Enable Change Data Capture (CDC) for the Account object.

Enabling Change Data Capture is a very easy and quick process.

In Salesforce, go to Setup -> Change Data Capture to activate CDC for the Account object.

Create and configure Platform Event for the Opportunity object.

In Salesforce, define a Platform Event object to hold the data that will be sent when an Opportunity is updated. This event object can hold data from any objects, in this case, data from Opportunity and Account.

Once the Platform Event object is defined, we write an APEX trigger code to retrieve the data from the Opportunity and Account.

The Salesforce configurations are ready. Next step is to configure the SAP SDI part.

Near real time data replication with Salesforce Platform Event or and Change Data Capture (CDC).

Installing the Advantco Salesforce Adapter on the Data Provisioning Agent host.

For a detailed installation and configuration instructions, please see this excellent blog from our NTT DATA colleague.

https://blogs.sap.com/2021/06/21/access-salesforce-custom-objects-from-hana-cloud-via-sdi/

Creating Remote Source to Salesforce.

Once the Salesforce adapter is deployed and registered to the SAP HANA server, it is now available to be used to create a Remote Source to Salesforce.

There are multiple ways to create a Remote Source, we use the Web-based Development Workbench user interface option.

Provide the Salesforce endpoint, the user credentials and in case of near real time replication, select the Streaming option.

Once the Remote source is configured and tested, one would see the Salesforce objects, Platform events and Event Channels.

Creating replication task to replicate Salesforce Change Data Capture events into HANA tables.

To capture Change Data Capture events into a HANA real table, we define a task and select the remote source created in the step above. In this example, the replication task replicate events from the Salesforce Account object using the standard Event channel.

The system will automatically create the HANA table structure for the Account Change Event object.

Once the replication is saved and started, the task can be monitored with the standard monitoring tool in HANA.

The virtual and real table were created automatically when the replication task was saved.

A query of the real table displays the events of the different accounts. The fields in Salesforce that was modified will have the updated information.

Creating replication task to replicate Salesforce Platform Event events into HANA tables.

To capture Platform events into a HANA real table, we use the same Streaming remote source as we used for CDC replication. We select the Platform Event object from the available Salesforce objects.

The system will create the table structure based on the Platform Event object structure that was created in Salesforce.

Save and activate the replication to start receiving Platform event from the Opportunity object into the HANA real table.

Each Platform event is replication in near real time into the HANA real table.

Summary

Receiving events in near real time from Salesforce to SAP HANA can be achieved using the Salesforce adapter for SAP SDI. Change Data Capture (CDC) provides an immediate update of any changes on Objects level in Salesforce. Platform Event enables events that can contain data from different Objects on Salesforce side.

Please reach out to our sales team at sales@advantco.com if you have any questions.