• All Files

 

Change Data Capture using Spark SQL

 

Open solution

 

Category	Talend specific
Prerequisites	Talend Data Integration basics, Talend Big Data Basics, Talend Big Data - Spark Batch, Context management common framework
Third-party software	Spark SQL, Hadoop cluster
Description	The Spark SQL CDC template Job uses Spark SQL to compare two datasets and identify records that need to be inserted, updated, and deleted after comparing with the last Job execution.