Programming

17517 readers

336 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities [email protected]

founded 1 year ago

MODERATORS

snowe

Ategon

[email protected]

How important is the option for dynamic linking (vs static linking) in the modern day? (lemmy.ml)

submitted 1 year ago* (last edited 1 year ago) by [email protected] to c/programming

62 comments fedilink hide all child comments

There was a time where this debate was bigger. It seems the world has shifted towards architectures and tooling that does not allow dynamic linking or makes it harder. This compromise makes it easier for the maintainers of the tools / languages, but does take away choice from the user / developer. But maybe that's not important? What are your thoughts?

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 6 points 1 year ago (1 children)

The main problem is that dynamic linking is hard. It is not just easier for the maintainers of the languages to ignore it, it removes an entire class of problems.

Dynamic linking does not even reliably work with C++, an "old" language with decades of tooling and experience on the matter. You get into all kind of UB when interacting with a separate DSO, especially since there are minimal verification of the ABI compatibility when loading a dynamic library. So you have to wait for a crash to be certain you got it wrong. Unless you control the compilation of your dependencies, it's fairly hard to be certain you won't encounter dynamic linking related issues. At which point you realize that, if the license allows it, you're better off static linking everything, including the C++ library itself: it makes it much more predictable, you're not forcing an additional dependency on your users and most UB are now gone (especially the one about raising exception across DSO boundaries, which can happen behind your back, unless you control the compilation of all your dependencies...).

That's especially true if you are releasing a library where you do not know it's runtime: it might be dynamically loaded via dlopen by a C++ binary that will load its own C++ library first, but some of your users use the version that is stuck on C++14 and your codebase is in C++23. This can be solved, by playing with LD_LIBRARY_PATH, but the application is already making use of it to load the C++ library it comes with instead of the one provided by the system (which only provides C++11 runtime), and it completely ignores the initial state of the environment variable (how could it do otherwise? It would have to guess the path to the libstdc++ is for a newer version and not the older one provided by the system). Now imagine the same issue with your own transitive dependency on top of that: it's a nightmare.

So dynamic linking never really worked, except maybe for C when you expect a single level of dependency, all provided by the system. And even then that's mostly thanks to C simpler ABI and runtime.

So I expect that is the main reason newer languages do not bother with dynamic linking: it introduces way too many issues. Look at your average rust program and how many version of a same dependency it loads, transitively. How would you solve that problem as to be able to load different versions when it matters but try first and foremost to load only one if possible? How would you be able to make the right call? By using semver? If nobody made any mistake why not, but you will rather be required to provide escape hatches that, much like LD_LIBRARY_PATH and LD_PRELOAD, will be misused. And by then, you only "solved" the simplest problem.

Nowadays, based on how applications are delivered on Windows and OSX, and with the advent of docker, flatpack/snap and appimage, I do not see a way back to dynamic linking anytime soon. It's just too complicated of a problem, especially as the number of dependency grows.

[–] lysdexic 4 points 1 year ago* (last edited 1 year ago) (1 children)

The main problem is that dynamic linking is hard.

That is not a problem. That is a challenge for those who develop implementations, but it is hardly a problem. Doing hard things is the job description of any engineer.

Dynamic linking does not even reliably work with C++, an “old” language with decades of tooling and experience on the matter.

This is not true at all. Basically all major operating systems rely on dynamic linking, and all of them support C++ extensively. If I recall correctly, macOS even supports multiple types of dynamic linking. On Windows, DLLs are use extensively by system and userland applications. There are no problems other than versioning and version conflicts, and even that is a solved problem.

You get into all kind of UB when interacting with a separate DSO, especially since there are minimal verification of the ABI compatibility when loading a dynamic library.

This statement makes no sense at all. Undefined behavior is just behavior that the C++ standard intentionally did not imposed restrictions upon by leaving the behavior without a definition. Implementations can and do fill in the blanks.

ABI compatibility is also a silly thing to bring up in terms of dynamic linking because it also breaks for static linking.

So dynamic linking never really worked,

This statement is patently and blatantly false. There was no major operating system in use, not a single one, where dynamic linking is/was not used extensively. This has been the case for decades.

[–] robinm 2 points 1 year ago

I think you don't understand what @CasualTee said. Of course dynamic linking works, but only when properly used. And in practice dynamic linking in a few order of magnitude more complex to use than static linking. Of course you still have ABI issue when you statically link pre-compiled libraries but in practice in statically linked workflow you are usually building the library yourself removing all ABI issues. Of course if a library is using a global and you statically linked it two times (with 2 differents versions) you will have an issue, but at least you can easily check that a single version is linked.

There are no problems other than versioning and version conflicts, and even that is a solved problem.

If it was solved, “DLL hell” wouldn't be a common expression and docker would have never been invented.

You get into all kind of UB when interacting with a separate DSO, especially since there are minimal verification of the ABI compatibility when loading a dynamic library.

This statement makes no sense at all. Undefined behavior is just behavior that the C++ standard intentionally did not imposed restrictions upon by leaving the behavior without a definition. Implementations can and do fill in the blanks.

@CasualTree was talking specically of UB related to dynamic linking and whitch simply do not exists when statically linking.

Yes dynamic linking work in theory, but in practice it's hell to make it work properly. And what advantage does it have compare to static linking?

Less RAM usage? That not even guaranteel because static linking allow aggressive inlining, constant propagation, LTO and other fun optimisation
Easier dependencies upgrade? That's mostly true for C, assuming you have perfect backward ABI compatibility. And nothing proves you that your binary is really compatible with newer versions of its libraries. And staticdependencies ungrade are an issue only because most Linux distribution don't have a workflow in witch updating a dependancy triggers the rebuil of all dependant binaries. If it was done it would then just be a question of download speed. Given the popularity of tools like docker who effectively tranforms dynamic linking into the equivalent of statically linking since all dependencies' versions are known, I would say that a lot of people prefer the confort of static linking.

To sum-up, are all the complications introduced specifically introduced by dynamic linking compared to static linking worth it for a non-guaranteed gain in RAM, a change in the tools of Linux maintainors and extra download time?