Hehe pp
noli
joined 1 year ago
JS was a mistake.
Username checks out :p
Actually I'm not so sure. That honestly sounds like it could lead to several significant issues, such a
It's a joy to do async in go IMO
That's actually a really nice application, in this case to reduce bandwidth constraints as opposed to the usual use case of memory constraints!
The typical arguments for a dynamic typed language are that it takes less time to write something in it.
The benefits of static typed languages are that your development environment can be a lot smarter (ironically enough leading to faster development speed) and several classes of bugs being unable to happen. In a statically typed language, the IDE can detect if you're trying to call a function that takes a number but you're actually providing a string. In this case the IDE will let you know and you can immediately fix silly mistakes like that.
Cool, so in this case your filter is basically a classifier ML model. How would you set the hash functions then though?
The classical example of scanning a k:v space in a memory efficient manner is the best application I've seen, but it still doesn't really work if you need to know a key is definitely in that space. You still have to perform the lookup.
In the google BigTable paper they use it like this. The reason it makes sense in that context is that there is that bloom filters are tiny and can be easily held in memory for a large dataset, whereas performing the actual lookup requires a slow disk access/network request. This way you can reduce the amount of expensive lookups massively at the cost of a small amount of memory.
Also since the underlying data format is immutable the cost of recalculating upon removal doesn't matter since it doesn't need to happen
it turned out that even with absurdly high lookup rates, adding an initial check for negative presence in a record set, we didn't benefit at all.
Compared to which alternative? I also don't understand exactly what the filter's purpose was in this case, did you have all DNS records in a bloom filter to quickly check for misses? Or was it some kind of allowlist/blocklist of clients?
Finally, what metrics were you using to decide it did not benefit at all?
Interesting. Do I understand it correctly if I say it's a bloom filter where instead of setting a bit to 1 for each of the hashes, you increment a counter for that hash?
How do you infer the count then, take the minimum of all matching hashes? Because intuitively it seems to me like you would need a lot more space to avoid counts being too high
I am dumb. The more things I need to think about when reading code that is not the logic of the code, the worse it is. Any time I have to spend thinking about the peculiarities of the way the language handles something is time wasted.
I'll give a very simple example, think like you're trying to find a bug. Assume we're in a dynamic language that allows implicit conversion like this. We can write our code very "cleanly" as follows:
if(!someVar) doSomething();
-> ok, now we gotta check where someVar's value is last set to know what type of data this is. Then I need to remember or look up how those specific types are coerced into a bool.
When trying the same code in a statically typed language that doesn't do implicit coercion that code will fail to run/compile so probably you'll have something like this:
if(someVar.length() == 0) doSomething();
-> this time I can just look at the type of someVar to see it's a string and it's clear what the condition actually means.
The second option is both easier to read and less bug prone even without the type system. It takes maybe 3 seconds longer to type, but if your productivity in coding is that limited by typing speed then I envy you
That's actually a great idea