---
title: Meeting of Worlds – Kafka Adapter
description: The meeting of two worlds – 3 Vs (volume, variety and velocity) of bigdata and the back office has brought in tremendous advantages to the business world.
image: https://resources.advantco.com/hubfs/Imported_Blog_Media/Online-Sale-orders-to-backend-SAP-using-Kafka.png
---

# Meeting of Worlds – Kafka Adapter

[ Sridhar Boyina ](https://resources.advantco.com/knowledge-base/author/sridhar-boyina)  Sep 8, 2019 4:00:00 PM

The meeting of two worlds – 3 Vs (volume, variety, and velocity) of big data and the back office has brought in tremendous advantages to the business world and has technical challenges to be dealt with.

Companies use data collection systems for nearly everything from business analytics to near-real-time operations, to executive reporting. As organizations are moving towards integrating IoTs to the regular operations, the data integration to the back office and operations in real-time (or near-real-time) is more in demand now. Apache Kafka is a leader in streaming the data and can also manage the 3 Vs very well.

While streaming the data from IoT devices, data generating software with the CRM or ERP systems using Kafka and lot of other out of the norm use cases, where customers want a little different or the business process needs a little more than the regular/normal usage, it is a challenge or a gap that is filled by [Advantco’s](https://www.advantco.com/) adapter.

Internet of Things to backend SAP system

A large advertising company uses Apache Kafka to stream data collected from the advertisement in the mobile apps. But then the data needs to be transmitted to the backend processes in SAP ERP system for further processing and usage.

 

Data collection software to backend SAP system

A retailer with online and in-store presence use Apache Kafka to collect order details from a Point of Sale (PoS) software but then needs to transmit to the SAP ERP system in a real-time or near-real-time frequency.

![Meeting of Worlds – Kafka Adapter](https://resources.advantco.com/hubfs/Imported_Blog_Media/Online-Sale-orders-to-backend-SAP-using-Kafka.png)

Time-bound Cleanup process

For a specific regulatory purpose, a manufacturing organization needs to delete customer data that has been placed in the Kafka cloud system after a specified amount of time.

![Meeting of Worlds – Kafka Adapter](https://resources.advantco.com/hubfs/Imported_Blog_Media/Time-bound-data-cleansing.png)

[Advantco adapter](https://www.advantco.com/product/adapter/apache-kafka-adapter-for-sap-netweaver-scpi) is being used for all these and more use cases. It enables to exchange data between non-SAP systems and the Integration Server or the PCK by means of Kafka APIs.

Benefits of the [Advantco Adapter for Kafka](https://www.advantco.com/product/adapter/apache-kafka-adapter-for-sap-netweaver-scpi)

The key advantage of [Advantco adapter](https://www.advantco.com/products/sap-pi-po-adapters) is that it is fully integrated with the SAP Adapter Framework, which means the Kafka connections can be configured, managed and monitored using the standard SAP PI/PO & CPI tools. With SAP scalability and stability, Advantco Kafka Adapter ensures the highest level of availability. It is also fully integrated with the SAP logging/tracing framework.

- Integration with SAP NetWeaver®: The adapter is built on the SAP NetWeaver® Adapter Framework
- Easy and quick installation by importing an SWCV transport (.tpz) into the Integration Repository/Enterprise Service Builder and deploying the J2EE software archive (SDA or SCA) via the J2EE SDM/JSPM.
- Integration with SAP NetWeaver® Runtime Workbench and SAP NetWeaver® J2EE Logging/Tracing. Supporting SAP NetWeaver PI/PO 7.5, PI 7.4, & SCPI.
- Configuration for the target topics and how to read streams of data
- Configuration for polling
- Adapter Specific Message Attributes

**Key features of the Advantco Adapter for Kafka**

Sender Adapter

The Sender Kakfa adapter must be configured as sender channels in the Integration Builder or the PCK. In a multi-cluster environment, the Sender Kafka adapter implicitly guarantees that there is only one session active among all server nodes at a time.

![Meeting of Worlds – Kafka Adapter](https://resources.advantco.com/hubfs/Imported_Blog_Media/Sender-Adapter-Advantco-Kafka.png)

A sender Kafka adapter is used to consume data from Kafka brokers then it sends data to the Integration Server or the PCK.

![Meeting of Worlds – Kafka Adapter](https://resources.advantco.com/hubfs/Imported_Blog_Media/Connection-Parameter-Advantco-Kafka-Adapter.png)

Receiver Adapter

The Receiver Kafka adapter must be configured as receiver channels in the Integration Builder or the PCK. Receiver Kafka channel sends message payloads received from the Integration Server or the PCK to Kafka Server.

Content Transformation

Content transformation or conversion parameters are for converting main message payload between Plain, JSON and XML formats.

![Meeting of Worlds – Kafka Adapter](https://resources.advantco.com/hubfs/Imported_Blog_Media/Content-Concersion-Advantco-Kafka-Adapter.png)

Producer API

Producers are used to publish messages to one or more Kafka topics. Producers send data to Kafka brokers. Every time a producer publishes a message to a broker, the broker simply appends the message to the last segment file. Actually, the message will be appended to a partition. Producer can also send messages to partition of their choice.

Variable Substitution

To enable the use of message descriptor apart from the payload for enhanced processes, the variable substitution is available varied use cases.

![Meeting of Worlds – Kafka Adapter](https://resources.advantco.com/hubfs/Imported_Blog_Media/Variable-Substitution-Advantco-Kafka-Adapter.png)

Advance Kafka Configuration

If the basic parameters provided do not fulfill the requirements, advance Kafka configuration can be enabled using the checkbox to specify the supported configurations for Kafka producers. The configuration key is the same as supported by the native Kafka documentation.

Logging and Tracing Configuration

The adapter uses standard *SAP Logging API* to write log and trace messages to standard log and trace files.

- Log messages help users to get information about adapter execution, including processing errors
- Trace messages are for Advantco’s developers to look deeply into adapter execution, which is helpful for debugging purposes.

**Conclusion**

[The Advantco adapter for Kafka](https://www.advantco.com/product/adapter/apache-kafka-adapter-for-sap-netweaver-scpi) enables exchange streams of data with the integration server or PCK by means of Kafka APIs.

Other benefits include

- Supports a variety of security options for encryption and authentication
- Supports both free-structure payload and Avro-structure payload
- Fully compatible with 0.11.0 and 1.0.0 brokers
- It is supported by a dedicated team with 24X7 support and comes with maintenance and upgrades etc.

Please reach out to our sales team at [sales@advantco.com](mailto:sales@advantco.com) if you have any questions.

[SAP Integration](https://resources.advantco.com/knowledge-base/tag/sap-integration), [Kafka](https://resources.advantco.com/knowledge-base/tag/kafka), [Kafka-PO-CPI](https://resources.advantco.com/knowledge-base/tag/kafka-po-cpi)

## Read On

[![](https://blogs.sap.com/wp-content/uploads/2018/07/SDI-Adapter-Image.png) ](https://resources.advantco.com/knowledge-base/use-case-to-s/4hana-smart-data-integration-sdi)

### [Use Case to S/4HANA Smart Data Integration (SDI)](https://resources.advantco.com/knowledge-base/use-case-to-s/4hana-smart-data-integration-sdi)

**First things first, why there was a need to develop SDI?**

[![](https://resources.advantco.com/hs-fs/hubfs/Imported_Blog_Media/Architecture-1.png?width=352&name=Architecture-1.png) ](https://resources.advantco.com/knowledge-base/content/advantco-kafka-adapter-for-sap-hana-smart-data-integration-sdi)

### [Advantco KAFKA adapter for SAP HANA Smart Data Integration (SDI)](https://resources.advantco.com/knowledge-base/content/advantco-kafka-adapter-for-sap-hana-smart-data-integration-sdi)

SAP HANA smart data integration and SAP HANA smart data quality load data, in...

[![](https://resources.advantco.com/hs-fs/hubfs/Advantco%20Kafka%20adapter%20LL_Pic1.png?width=352&name=Advantco%20Kafka%20adapter%20LL_Pic1.png) ](https://resources.advantco.com/knowledge-base/lessons-learned-from-kafka-integrations-using-the-advantco-kafka-adapter-for-sap-integration-suite)

### [Lessons learned from KAFKA Integrations using the Advantco KAFKA Adapter for SAP Integration Suite](https://resources.advantco.com/knowledge-base/lessons-learned-from-kafka-integrations-using-the-advantco-kafka-adapter-for-sap-integration-suite)

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Sridhar Boyina",
    "url" : "https://resources.advantco.com/knowledge-base/author/sridhar-boyina"
  },
  "dateModified" : "2022-06-28T13:47:03.259Z",
  "datePublished" : "2019-09-08T20:00:00.000Z",
  "headline" : "Meeting of Worlds – Kafka Adapter ",
  "image" : [ "https://resources.advantco.com/hubfs/Imported_Blog_Media/Online-Sale-orders-to-backend-SAP-using-Kafka.png" ],
  "mainEntityOfPage" : {
    "@id" : "https://resources.advantco.com/knowledge-base/content/meeting-of-worlds-kafka-adapter",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://resources.advantco.com/hubfs/Advantco%20logo%20AAC%20V1%20Ai%20file%201.png"
    },
    "name" : "Advantco International"
  }
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "NewsArticle",
  "author" : [ {
    "@type" : "Person",
    "name" : "Sridhar Boyina",
    "url" : "https://resources.advantco.com/knowledge-base/author/sridhar-boyina"
  } ],
  "dateModified" : "2022-06-28T13:47:03",
  "datePublished" : "2019-09-08T20:00:00",
  "headline" : "Meeting of Worlds – Kafka Adapter ",
  "image" : [ "https://f.hubspotusercontent20.net/hubfs/6260070/Imported_Blog_Media/Online-Sale-orders-to-backend-SAP-using-Kafka.png" ]
}
```

```json
{
  "@context" : "http://schema.org",
  "@type" : "Organization",
  "address" : {
    "@type" : "PostalAddress",
    "addressCountry" : "United States",
    "addressLocality" : "Charlotte",
    "addressRegion" : "NC",
    "postalCode" : "28203",
    "streetAddress" : "1235 East Blvd, Suite E #382 "
  },
  "alternateName" : "advantco",
  "areaServed" : {
    "@type" : "GeoCircle",
    "geoMidpoint" : {
      "@type" : "GeoCoordinates",
      "latitude" : "40.75996223497667",
      "longitude" : "-73.98761007334504"
    },
    "geoRadius" : "150 km"
  },
  "description" : "Feature rich Advantco adapters integrates SAP & Oracle system with top enterprise platforms like Kafka, Salesforce, Azure, and more.",
  "email" : "webadmin@advantco.com",
  "image" : "https://www.advantco.com/hs-fs/hubfs/unnamed%20(2).webp?width=450&height=60&name=unnamed%20(2).webp",
  "logo" : "https://6260070.fs1.hubspotusercontent-na1.net/hubfs/6260070/Advantco%20logo%20AAC%20V1%20Ai%20file%201.png",
  "mainEntityOfPage" : {
    "@id" : "https://resources.advantco.com/knowledge-base/content/meeting-of-worlds-kafka-adapter",
    "@type" : "WebPage",
    "description" : "The meeting of two worlds – 3 Vs (volume, variety and velocity) of bigdata and the back office has brought in tremendous advantages to the business world."
  },
  "naics" : "513210",
  "name" : "Advantco International",
  "sameAs" : [ "https://www.linkedin.com/company/advantco-international-llc" ],
  "telephone" : "",
  "url" : "https://www.advantco.com/"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "WebPage",
  "about" : [ {
    "@id" : "https://resources.advantco.com/knowledge-base/content/advantco-pre-packaged-content-solution",
    "@type" : "WebPage"
  }, {
    "@id" : "https://www.advantco.com/sap-hana-sdi-integration-adapters/sap-hana-sdi-salesforce-integration",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/salesforce-integration-common-requirements-and-use-cases",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/the-top-5-advantages-of-sap-and-salesforce-integration",
    "@type" : "WebPage"
  }, {
    "@id" : "https://www.advantco.com/sap-integration-adapters/sap-salesforce-integration",
    "@type" : "WebPage"
  } ],
  "mainEntity" : {
    "@type" : "WebPage"
  },
  "url" : "https://www.advantco.com/sap-integration-adapters/sap-salesforce-integration"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "WebPage",
  "about" : [ {
    "@id" : "https://resources.advantco.com/knowledge-base/receiving-events-from-dynamics-365-crm-using-webhooks",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/microsoft-dynamics-crm-adapter-for-sap-process-orchestration-and-sap-integration-suite-now-available-on-sap-store",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/adapters-for-sap-hci-integration-with-ms-dynamics-crm-and-rabbitmq",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/sap-cloud-platform-integration-enhancing-the-third-party-application-connectivity-with-new-offerings",
    "@type" : "WebPage"
  } ],
  "mainEntity" : {
    "@type" : "WebPage"
  },
  "url" : "https://www.advantco.com/sap-integration-adapters/sap-microsoft-dynamics-integration"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "WebPage",
  "about" : [ {
    "@id" : "https://resources.advantco.com/knowledge-base/content/sap-cloud-platform-integration-enhancing-the-third-party-application-connectivity-with-new-offerings",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/using-advantco-aws-adapter-to-integrate-sap-and-aws-business-environments",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/integrating-sap-and-aws-business-environments-with-the-advantco-aws-adapter",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/product/adapter/aws",
    "@type" : "WebPage"
  } ],
  "mainEntity" : {
    "@type" : "WebPage"
  },
  "url" : "https://www.advantco.com/sap-integration-adapters/sap-aws-integration"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "WebPage",
  "about" : [ {
    "@id" : "https://resources.advantco.com/knowledge-base/content/google-cloud-platform-adapter-from-advantco-international-now-available-on-sap-store",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/product/adapter/google-cloud-platform-adapter-for-sap-cpi",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/advantco-google-cloud-platform-adapter-integrating-sap-with-gcp",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/advantco-google-cloud-platform-adapter-integrating-sap-with-gcp",
    "@type" : "WebPage"
  } ],
  "mainEntity" : {
    "@type" : "WebPage"
  },
  "url" : "https://www.advantco.com/sap-integration-adapters/sap-google-cloud-integration"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "WebPage",
  "about" : [ {
    "@id" : "https://resources.advantco.com/knowledge-base/consumer-electronics-retailer-turns-to-advantco-amqp-azure-adapters",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/adapters-for-sap-hci-integration-with-ms-dynamics-crm-and-rabbitmq",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/integration-with-microsoft-azure-cloud",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/using-rabbitmqamqp-broker-for-extreme-high-volume-integration-with-sap-hybris-commerce-solution-with-sap-pipo",
    "@type" : "WebPage"
  } ],
  "mainEntity" : {
    "@type" : "WebPage"
  },
  "url" : "https://www.advantco.com/sap-integration-adapters/sap-azure-integration"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "WebPage",
  "about" : [ {
    "@id" : "https://resources.advantco.com/knowledge-base/lessons-learned-from-kafka-integrations-using-the-advantco-kafka-adapter-for-sap-integration-suite",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/kafka-adapter-for-sap-process-orchestration-and-sap-integration-suite-now-available-on-sap-store",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/advantco-kafka-adapter-enables-automobile-leader-to-use-big-data",
    "@type" : "WebPage"
  }, {
    "@id" : "https://resources.advantco.com/knowledge-base/content/kafka-adapter-with-avro-serialization-and-schema-registry",
    "@type" : "WebPage"
  } ],
  "mainEntity" : {
    "@type" : "WebPage"
  },
  "url" : "https://www.advantco.com/sap-integration-adapters/sap-kafka-integration"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "WebPage",
  "about" : [ {
    "@id" : "https://www.advantco.com/sap-integration-adapters/sap-salesforce-integration",
    "@type" : "WebPage"
  }, {
    "@id" : "https://www.advantco.com/sap-integration-adapters/sap-microsoft-dynamics-integration",
    "@type" : "WebPage"
  }, {
    "@id" : "https://www.advantco.com/sap-integration-adapters/sap-aws-integration",
    "@type" : "WebPage"
  }, {
    "@id" : "https://www.advantco.com/sap-integration-adapters/sap-google-cloud-integration",
    "@type" : "WebPage"
  }, {
    "@id" : "https://www.advantco.com/sap-integration-adapters/sap-azure-integration",
    "@type" : "WebPage"
  }, {
    "@id" : "https://www.advantco.com/sap-integration-adapters/sap-kafka-integration",
    "@type" : "WebPage"
  } ],
  "mainEntity" : {
    "@type" : "WebPage"
  },
  "url" : "https://www.advantco.com/"
}
```

```json
{
  "@context" : "https://schema.org/",
  "@type" : "BreadcrumbList",
  "itemListElement" : [ {
    "@type" : "ListItem",
    "item" : "https://advantco.com/",
    "name" : "Home",
    "position" : 1
  } ]
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "ProfessionalService",
  "areaServed" : {
    "@type" : "GeoShape",
    "address" : {
      "@type" : "PostalAddress",
      "addressCountry" : "US"
    }
  },
  "description" : "Advantco helps organizations update and transform their integration landscapes by migrating to SAP BTP Integration Suite, enabling secure, scalable, cloud-native connectivity across SAP and non-SAP systems.",
  "image" : "https://www.advantco.com/hs-fs/hubfs/1-2.png?width=2000&height=1015&name=1-2.png",
  "logo" : "https://www.advantco.com/hs-fs/hubfs/unnamed%20(2).webp?width=300&height=40&name=unnamed%20(2).webp",
  "name" : "SAP BTP Integration Suite Migration Services",
  "provider" : {
    "@type" : "Organization",
    "contactPoint" : {
      "@type" : "ContactPoint",
      "availableLanguage" : [ "English" ],
      "contactType" : "customer support",
      "url" : "https://www.advantco.com/contact-us"
    },
    "logo" : "https://www.advantco.com/hs-fs/hubfs/unnamed%20(2).webp?width=300&height=40&name=unnamed%20(2).webp",
    "name" : "Advantco",
    "sameAs" : [ "https://www.linkedin.com/company/advantco-international-llc/", "https://x.com/AdvantcoAdapter" ],
    "url" : "https://www.advantco.com"
  },
  "serviceType" : [ "SAP BTP Integration Suite Migration", "Legacy Middleware Migration", "Clean Core & Customization Rationalization", "ABAP & API Development" ],
  "url" : "https://www.advantco.com/sap-btp-integration-suite-migration"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "ProfessionalService",
  "description" : "Advantco offers secure, scalable, and cost-optimized AWS cloud services — from landing zone setup and migration to managed operations and AI/ML solutions.",
  "image" : "https://www.advantco.com/hs-fs/hubfs/4%20(1)-1.png?width=316&height=300&name=4%20(1)-1.png",
  "logo" : "https://www.advantco.com/hs-fs/hubfs/unnamed%20(2).webp?width=300&height=40&name=unnamed%20(2).webp",
  "name" : "AWS Cloud Services",
  "provider" : {
    "@type" : "Organization",
    "contactPoint" : {
      "@type" : "ContactPoint",
      "availableLanguage" : [ "English" ],
      "contactType" : "customer support",
      "url" : "https://www.advantco.com/contact-us"
    },
    "logo" : "https://www.advantco.com/hs-fs/hubfs/unnamed%20(2).webp?width=300&height=40&name=unnamed%20(2).webp",
    "name" : "Advantco",
    "sameAs" : [ "https://www.linkedin.com/company/advantco-international-llc/", "https://x.com/AdvantcoAdapter" ],
    "url" : "https://www.advantco.com"
  },
  "serviceType" : [ "AWS Landing Zone Setup", "AWS Application Migration", "Managed AWS Cloud Operations", "AWS AI/ML Solutions" ],
  "url" : "https://www.advantco.com/aws-cloud-services"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "ProfessionalService",
  "areaServed" : {
    "@type" : "Country",
    "name" : "Worldwide"
  },
  "description" : "Advantco helps organizations harness the full power of Google Cloud by designing, migrating, and managing secure, scalable, and cost-optimized workloads end-to-end.",
  "image" : "https://www.advantco.com/hs-fs/hubfs/4%20(1)-1.png?width=316&height=300&name=4%20(1)-1.png",
  "logo" : "https://www.advantco.com/hs-fs/hubfs/unnamed%20(2).webp?width=300&height=40&name=unnamed%20(2).webp",
  "name" : "Google Cloud Services",
  "provider" : {
    "@type" : "Organization",
    "contactPoint" : {
      "@type" : "ContactPoint",
      "availableLanguage" : [ "English" ],
      "contactType" : "customer support",
      "url" : "https://www.advantco.com/contact-us"
    },
    "logo" : "https://www.advantco.com/hs-fs/hubfs/unnamed%20(2).webp?width=300&height=40&name=unnamed%20(2).webp",
    "name" : "Advantco",
    "sameAs" : [ "https://www.linkedin.com/company/advantco-international-llc/", "https://x.com/AdvantcoAdapter" ],
    "url" : "https://www.advantco.com"
  },
  "serviceType" : [ "Google Cloud Foundation Setup", "Application Migration & Modernization", "Managed Google Cloud Operations", "Data Analytics & AI/ML Solutions" ],
  "url" : "https://www.advantco.com/google-cloud-services"
}
```