No Stupid Questions (Developer Edition)

923 readers

1 users here now

This is a place where you can ask any programming / topic related to the instance questions you want!

For a more general version of this concept check out [email protected]

Icon base by Lorc under CC BY 3.0 with modifications to add a gradient

founded 1 year ago

MODERATORS

LinearArray

Is preloading/caching data before the actual method call an (anti)pattern? (self.no_stupid_questions)

submitted 11 months ago by Cyno to c/no_stupid_questions

10 comments fedilink hide all child comments

cross-posted from: https://programming.dev/post/6513133

Short explanation of the title: imagine you have a legacy mudball codebase in which most service methods are usually querying the database (through EF), modifying some data and then saving it in at the end of the method.

This code is hard to debug, impossible to write unit tests for and generally performs badly because developers often make unoptimized or redundant db hits in these methods.

What I've started doing is to often make all the data loads before the method call, put it in a generic cache class (it's mostly dictionaries internally), and then use that as a parameter or a member variable for the method - everything in the method then gets or saves the data to that cache, its not allowed to do db hits on its own anymore.

I can now also unit test this code as long as I manually fill the cache with test data beforehand. I just need to make sure that i actually preload everything in advance (which is not always possible) so I have it ready when I need it in the method.

Is this good practice? Is there a name for it, whether it's a pattern or an anti-pattern? I'm tempted to say that this is just a janky repository pattern but it seems different since it's more about how you time and cache data loads for that method individually, rather than overall implementation of data access across the app.

In either case, I'd like to learn either how to improve it, or how to replace it.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 3 points 11 months ago (1 children)

It sounds like you're more-or-less describing memoization? A method caches its input and output, often in some sort of dictionary, and utilizes that cache to return if it receives the same input multiple times.

[–] Cyno 2 points 11 months ago (1 children)

I'm not caching or reusing method results however, and even the inputs are not necessarily cached for multiple uses. I'm just preparing all potentially required input data before the method is actually called so I don't have to do any loads within the method itself, so the method is just pure code logic and no db interaction.

For example, imagine you have a method that scores the performance of an athlete. The common "pattern" in this legacy code base is to just go through the logic and make a database load whenever you need something, so maybe at the beginning you load the athlete, then you load his tournament records, then few dozen lines later you load his medical records, then his amateur league matches, etc.

What I do is I just load all of this into a cache before the actual method call, and then send it into the method as a data source. The method will only use the cache and do all the calculations in-memory, and when it's done the result would be in the cache as well. Then outside of the method I can just trigger a save or abandon it to persist the result. If I want to unit test it, I can easily just manually fill a cache with my data and use it as the data source (usually you'd have to mock custom response from the repository or something like that, inject an in-memory repository with the same data anyway or just resign to using an integrated test).

It's like I'm "containerizing" the method in a way? It's a pretty simple concept but I'm having trouble googling for it since I don't know how to call it.

[–] pohart 2 points 11 months ago (1 children)

What language are you using? Is a good idea to limit db calls, but maybe we can help with specific techniques idiomatic to your language

[–] Cyno 2 points 11 months ago* (last edited 11 months ago) (1 children)

Ah sorry, forgot to mention it here because I originally posted it on csharp and then crossposted. I'm specifically thinking about c#, EF and .net core for web dev.

[–] pohart 2 points 11 months ago* (last edited 11 months ago)

I don't know .net and sometimes quite some janky code, but I think in this case I would preload everything I definitely needed, locking the records I'm modifying. Then use ConcurrentDictionary.GetOrAdd(Tkey,Func<...>) to load values I might need only when they're needed.