this post was submitted on 13 May 2024
20 points (95.5% liked)

Data Engineering

387 readers
1 users here now

A community for discussion about data engineering

Icon base by Delapouite under CC BY 3.0 with modifications to add a gradient

founded 2 years ago
MODERATORS
ericjmorey
20
Building a real-time data pipeline - Technical article and GitHub repo (medium.com)
submitted 7 months ago by [email protected] to c/data_engineering
0 comments fedilink hide all child comments
 

If you're a Data Engineer, before long you'll be asked to build a real-time pipeline.

In my latest article, I build a real-time pipeline using Kafka, Polars and Delta tables to demonstrate how these can work together. Everything is available to try yourself in the associated GitHub repo. So if you're curious, take a moment to check out this technical post.

no comments (yet)
sorted by: hot top controversial new old
there doesn't seem to be anything here