this post was submitted on 04 Aug 2024
84 points (97.7% liked)

Programming

17528 readers
233 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities [email protected]



founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 13 points 3 months ago (2 children)

I worked on a system whose database setup looked kind of like this. Interesting to see, that they weren't the only ones using that approach.

We had an online database for online transactions, all tables used a sequence generator table (basically a key/value with the key being table name and the value being the last id) l, every few minutes all the new transactions where dumped into a second instance for research and monitoring. Every night a job started, that first dropped all the transactions older than a few weeks from the online db and then exploded every single transaction into a bunch of id, key, value tuples. These were then push/pulled (very weird construct) into the datawarehouse. Since each new value was its own table, we had something like 20 tables, mostly being nulls. You might had columns like serviceA_call1_customer3_adress_streetname. Absolutely bonkers and only one man understood that thing.

[–] sukhmel 7 points 3 months ago

only one man understood that thing

that one man

coloured picture of Albert Einstein with an unlit smoking pipe

[–] [email protected] 2 points 3 months ago

But denormalized databases are not a new thing. There are engines that build on it on purpose in order to be more efficient, like Cassandra. Most data warehousing engines use this "trick". And of course you can do it with a regular RDBMS too.