Use Case to S/4HANA Smart Data Integration (SDI)

First things first, why there was a need to develop SDI?

Every new technology or platform is developed as a need to address the challenges faced by older technologies, cater to the unmet needs of the consumer or to facilitate a change by making the processes easier and/or economical. Smart Data integration is no exception. 

A few years ago, if I wanted to load a flat file into HANA, I had to install Data Services, an ETL tool to replicate, transform and load the data from source to the target system. In order to use the real-time functionalities, I additionally installed SLT/SRS and in case I was not copying data to the source, I used SDA to take advantage of the virtual tables. So, in total I was installing 3 different tools with their own connectors, in-fact I had little choice until Smart Data Integration was developed.

The best of all world, combining all the features of ETL, real-time replication, and virtual tables: Smart data integration acts as a bridge between the source and HANA, reads the source data and translates the values into HANA datatype value. It Batch load and/or replicate changed data in real-time from source to SAP HANA tables providing immediate visibility into conditions located in either SAP HANA or source.

Use Cases:

This use case is focused on how SDI facilitates real-time replication of data from Salesforce to SAP HANA. The data can be either pulled on demand when any query executes or when data in Salesforce changes or gets updated, it gets automatically pushed into SAP HANA.

Advantco International LLC (Advantco) offers SFDC adapter and Microsoft CRM adapter for SAP HANA Smart Data Integration (SDI), its purpose is to batch load or to replicate changed data in real time from Salesforce.com or Microsoft CRM to the SAP HANA tables.

Why SDI is the preferred tool?

• Easy Configuration & Setup

All that needs to be done is to install the SAP SDI Agent and deploy the SFDC adapter on it. The SDI adapter is hosted on an S/HANA data provisioning agent and acts as a bridge between Salesforce and HANA, reads the source data and translates the values into HANA datatype value.

• Authentication with the Source system

The SFDC Adapter for SDI supports the basic authentication. All you need to do is provide your credentials (Username, Password, Security Token (Optional)) in a remote source configuration.

•  Session Reuse Mechanism

The SFDC Adapter for SDI supports Salesforce session reuse in order to reduce the number of login call to Salesforce.com. Salesforce session reuse will automatically renew the session if the current one is expired.

•  Adjusts Communication Method Dynamically Based on Data Volume

The Adapter provides a “Maximum Expected Number of Record” parameter which is designed to switch automatically between Salesforce APIs. SOAP API is being used when the actual number of records in source data is less than or equal to the value of the maximum expected number of records parameter. Otherwise, BULK API is being used.

• QueryLocator Results

The QueryLocator is used when querying a large number of records. The QueryLocator represents the server-side cursor that tracks the current processing location in the query result set. The Adapter will call the query repeatedly until all records in the result set have been processed.

• Virtual Table as a Source

As the SQL query runs directly in the source system, using the SDI Adapter you can virtually access the data without the need to copy the data to S/4HANA.

• Support batch and real-time for all sources

SDI has the capability to do both near-real-time data replication and an option to execute even in batch mode.

• Allow transformations on batch and real-time data

SDI as the capability to integrate and cleanse data. The SAP HANA SDI provide SAP HANA Web-based Development Workbench to create flow graphs to transform data. The flow graph enables ETL based batch and Real-time data flows. There are a wide variety of options available to meet the transformation data such as filtering, joining, aggregating, cleansing and more.

• Change Data Capture (CDC)

Replicating entire source data to the target every day puts an additional load on the non-critical tasks, affecting the performance of the whole system. CDC tracks the change data and transfers the latest changes across source and target, thereby enhancing efficiency and performance.

• Data Quality

Using SDI, you can rest assured of both the data quality and data reliability which enhances your experiences in the intended uses in operations, decision making, and planning.

• Predictive Analytics

Along with integration, SDI can be used to draw meaning insights based on predictive patterns and using machine learning and other algorithms.

• Provide one connectivity that supports all

SDI provides lots of flexibility to connect to both RDBMS and unstructured data sources. Furthermore, it also supports Big data and other new systems.

•  Provide one UI that supports all

The SAP HANA SDI provide SAP HANA Web-based Development Workbench which allows you to develop entire applications in a Web browser without having to install any development tools. It provides an intuitive UI and simplifies development by providing many convenient functions.

How it works:

1.How to replicate Account data from Salesforce.com in HANA tables

The SFDC Adapter for SDI acts as a bridge. It opens a connection to the Salesforce and read the source data and translate the values into the Hana datatype value.

Account Retrieval Use Case

The goal is to make this data of Account on Salesforce available in SAP HANA by Querying the remote Account via the Virtual table (data is not physically moved to the cloud, but remains in its original source.)

Create a Remote Source

Save and click Test connection:

We can browse the metadata tables which are provided by the adapter, each table corresponds to one sObject.

2. Grant privileges for _SYS_REPO

Execute the SQL statement below:

GRANT CREATE VIRTUAL TABLE, CREATE REMOTE SUBSCRIPTION, DROP ON REMOTE SOURCE “Salesforce” TO _SYS_REPO;

The result as below:

  3. Create a Virtual Table

After creating a Salesforce remote source, we can create a virtual table to retrieve data for an account as below:

Input virtual table name and select “ADVANTCO_SFDC” for schema

Go to “ADVANTCO_SFDC” schema and refresh “Tables” node:

4. Query on Virtual table

Open the SQL console, input and run the SQL statement with the result as below:

5. Account Replication Use Case

The goal is to make this data of Account on Salesforce available in SAP HANA by using replication task feature

5.1. Create a Replication Task

Create the Replication task as below:

Input the name:

Select the remote source, the target schema:

Select “Account” remote table and choose “Initial load only” for the replication behavior.

5.2. Run replication task

After saving the replication task, click run task with the result as below:

5.3. View result

After the replication task done, go to the “ADVANTCO_SFDC” schema to open content of “RT_Account” Table with the result as below:

Conclusion:

The SFDC Adapter for SDI and Microsoft Adapter for SDI provides business with the tools to acquire, service, and retain new customers. By delivering comprehensive data integration and customer data management tool that takes advantage of the best of SAP and SFDC/ Microsoft worlds, businesses can ensure that their Finance, Marketing, Sales, Customer Service, and Fulfillment departments collaborate with each other and get the right information at the right time.