tubeshowcase●1,674,138 videos♡ My collection 0
← Learn to Code

Learn to Code35:57

Apache Kafka Meets Apache Iceberg: Real-Time Data Streaming • Kasun Indrasiri • GOTO 2024

GOTO Conferences ↗

From the creator

This presentation was recorded at GOTO Chicago 2024. #GOTOcon #GOTOchgo https://gotochgo.com Kasun Indrasiri - Author of "Microservices for the Enterprise" @kasunindrasiri ORIGINAL TALK TITLE Kafka Meets Iceberg: Real-Time Data Streaming into Modern Data Lakes & Warehouses RESOURCES https://twitter.com/kasunindrasiri https://medium.com/@kasunindrasiri https://www.linkedin.com/in/kasun-indrasiri ABSTRACT In this talk, we'll explore how Kafka serves as a powerful platform for capturing real-time streaming data and how organizations are increasingly adopting Apache Iceberg table format to store data in data lakes and data warehouses. We'll discuss the key benefits of using Apache Iceberg tables in your data lake such as schema evolution, ACID transactions, hidden partitioning, time traveling and efficient querying. Next, we'll dive into how to efficiently stream data from Kafka into Iceberg-based data lakes. Confluent Tableflow will be introduced as a potential solution for streamlining the ingestion of Kafka streams into Iceberg tables within your data lake. A live demo will showcase the seamless integration of Kafka with Iceberg, equipping participants with practical knowledge to enhance their data architectures for powerful real-time analytics. • The role of Kafka in real-time data streaming • Why Apache Iceberg is essential for data lakes and data warehouses • Iceberg fundamentals: Core concepts and key features • Streaming data from Kafka to Iceberg tables in data la

Published 2025-02-12 · Watch on YouTube ↗
Availability and playback restrictions are controlled by the original channel.

Keep exploring Learn to Code