Isn’t Apache Arrow a table format rather than a batch processing tool?
You could add SQLMesh as an alternative to dbt.
News and discussion on Data Engineering topics
Isn’t Apache Arrow a table format rather than a batch processing tool?
You could add SQLMesh as an alternative to dbt.
thanks for suggesting sqlmesh. regarding arrow, have you read this - https://arrow.apache.org/docs/format/Columnar.html
btw, you've spelt Engineering wrong in the sidebar.
thanks for pointing out. fixed.