Example: Define and Apache Flink job for basic data filtering
In this example we’ll define an Apache Flink streaming job, named
JobExample, performing a basic filtering of the data available in the Apache Flink table named
KCpuIn, and insert the results into the
When defining Apache Flink jobs, the input and output objects are Apache Flink table, enabling to decouple the data pipeline definition from the source or sink technologies. If a change in the backend technology is needed, it can be handled by redefining the Apache Flink table, without needing to change the job.
We can define the Apache Flink job named
The image below shows the Aiven console page with the filled details.
The result of the data pipeline is the target table
KAlert being populated with data exceeding the
Depending on the Apache Flink table definition the data could either be written to an Apache Kafka topic or a PostgreSQL table where the data pipeline results can be verified.