this post was submitted on 19 Oct 2023
1765 points (98.8% liked)
Programmer Humor
32356 readers
1219 users here now
Post funny things about programming here! (Or just rant about your favourite programming language.)
Rules:
- Posts must be relevant to programming, programmers, or computer science.
- No NSFW content.
- Jokes must be in good taste. No hate speech, bigotry, etc.
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Your knowledge of data engineering may be limited. SQL is predominant in data processing nowadays. FOSS tools such as DBT allows to write efficient data processing pipelines with SQL and some YAML config without the need for a general purpose coding language.
Why would anyone want that? Because SQL has the interesting property of describing the result you want rather than describing how to compute it. So you can put inside the database, a query engine with decades of optimizations, that will make a much better job at finding the best execution plan than the average developer.
It also means it's easier to train people for data processing nowadays.
Learning DBT was pretty easy for me as a data analyst. Now I’m contributing to my company’s data warehouse instead of just pulling existing data.