this post was submitted on 31 May 2024
34 points (92.5% liked)

Linux

48306 readers
738 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
34
Building and distributing binaries (self.linux)
submitted 5 months ago by lazyneet to c/[email protected]
28 comments fedilink hide all child comments
 

It's been a long day and I'm probably not in the best state of mind to be asking this question, but have you guys solved packaging yet?

I want to ship an executable with supporting files in a compressed archive, much like the Windows exe-in-a-zip pattern. I can cross-compile a Win32 C program using MinGW that will always use baseline Win32 functionality, but if I try to build for Linux I run into the whole dependency versioning situation, specifically glibc fixing its symbol version to whichever Linux I happen to be building from at the time. But if I try to static link with musl, the expectation is that everything is static linked, including system libraries that really shouldn't be.

AppImage is in the ballpark of what I'm looking for, and I've heard that Zig works as a compatibility-enhancing frontend if you're compiling C. I'd just like something simple that runs 99% of the time for non-technical end users and isn't bloated with dependencies I can't keep track of. (No containers.) Is this easily achievable?

top 28 comments
sorted by: hot top controversial new old
[–] [email protected] 30 points 5 months ago* (last edited 5 months ago)

I think you are confusing "windows like" with "user-friendly". A "bespoke archive, that you find on some developer's website, that you extract and somewhere it contains an executable and assets, that you move where you want to keep them, and then the user remembers to manually update it sometimes somehow" is not how you usually do stuff on Linux and is not even user-friendly.

Distributions come with programs like "gnome software" or "kde discover" that allows the user to graphically install programs from the distro's package manager, or from flatpak or snap. It will also help them to keep them updated and to manage dependencies. That is user-friendly.

I suggest using flatpak. It will work on almost all distros out of the box and will be easy to install and maintain for the user. If flatpak is too "bloated" for you because it uses containers, then you need to package it for every distro manually, but that's a lot of work. If it's something that just needs to be used once and never again, consider an appimage or a script, because they don't need to be installed.

Distros are different operating systems, it's not gonna be easy to package for all of them without compromises.

Also, if you really really really need to use your bespoke archive, you can do like native steam games do, and put every library you link in the archive, and link with relative paths instead of system wide paths, or with a launch script that loads your provided libraries. But that's not a great user experience. Steam gets away with it because it's the launcher that manages the whole thing.

[–] [email protected] 9 points 5 months ago

If you want something simple most people can run, use flatpak. AppImage often fails for me because I'm missing some file.so

[–] [email protected] 5 points 5 months ago* (last edited 5 months ago)

Glibc preserves backward compatibility, so if you build against the oldest version you want to support, the resulting binary will work with newer ones.

However that's definitely not what I recommend to do. Better learn packaging and build native packages for distros you are going to support. OBS can make this a bit easier (if your software is FOSS), but any modern CI will also do the job.

[–] [email protected] 4 points 5 months ago (2 children)

A solution I've used for the glibc problem, is to build on an older distribution in a chroot. There is also this project which might be of use to pick a specific version of glibc. The project README also explain how to do it manually.

As for distribution, I prefer something like makeself.sh, that installs to either ~/.local/ or if it is to be installed system-wide to /usr/local or /opt. The concept is just a small shell script appended with a compressed archive, it is easy to modify and even create by hand using standard tools like cat. This is a method widely used by native Linux games.

[–] [email protected] 3 points 5 months ago (1 children)

I've built a Dockerfile that does a hybrid of solution 1 and AppImage building.

It compiles the software with an older Debian release, then packages the software in a Python AppImage with necessary dependencies installed and the proper dynamic libs copied.

[–] [email protected] 2 points 5 months ago

Using containers for build environments is probably my favorite use of containers.

I have an application I build for Linux, Mac and Windows and frankly building two or three Linux builds in containers is easier than the Windows and Mac builds alone. A github action automates it easily.

[–] lazyneet 1 points 5 months ago

I'll probably have to use chroot or docker. I tried with glibc force link but when I objdump -T I see symbols that slip through with newer glibc, even when they're .symver'd in the header. That project hasn't been updated in a long time.

[–] [email protected] 3 points 5 months ago (1 children)

At $DAYJOB, we're currently using this to cross-compile a C/C++ application: https://github.com/dockcross/dockcross

If you just want x86-64 with old glibc, you can use their "manylinux" image.

But yes, this does not solve C applications being a massive pain in the ass to distribute, so you may still want to ship it as an AppImage or container image or with static linking or whatever.

[–] lazyneet 2 points 5 months ago

Thanks for the info! If I'm doing container builds anyways, this looks tasty.

[–] [email protected] 2 points 5 months ago (1 children)

Maybe I'm missing something but this seems like a chicken egg problem. It seems like you are trying to approach this from a Windows perspective when you should approach it from a Linux perspective.

What's your end goal?

[–] lazyneet 3 points 5 months ago (1 children)

Mainly getting builds onto platforms catering to Windows users and gamers. The consensus here seems to be using containerized build environments.

[–] [email protected] 1 points 5 months ago (1 children)

Then why don't you build for Windows? Maybe I'm missing something

[–] lazyneet 1 points 5 months ago (1 children)

I do, but Linux should be a first-class platform alongside Windows.

[–] [email protected] 1 points 5 months ago (1 children)

Proton on Linux works pretty well. It might be easier for you to use bottles. As a Linux user I would rather use a complete Windows port than a half complete Linux port.

Kudos for the effort though

[–] lazyneet 1 points 5 months ago (2 children)

Thanks. In my experience, Wine and Proton don't work as well as native for one of the apps I'm building, so I will need to either build in a container or say "use X Ubuntu version".

[–] [email protected] 1 points 5 months ago (1 children)

In that case go the Flatpak route. Best option would be to upload to Flathub.

[–] lazyneet 1 points 5 months ago

Nope!

[–] [email protected] 1 points 5 months ago

I've once had difficulties running some apps on Proton that used .NET features not supported by mono, which has been updated since then and is now working out of the box.

I'm playing Trackmania on wine, I've played Elden Ring and Monster Hunter: World on Proton, so I'm wondering which issue you're running into.

Regardless, building precompiled Linux native binaries is a commendable goal. Others have mentioned Flatpak, which imo is a good and user-friendly way to handle that.

[–] oessessnex 2 points 5 months ago

I quite enjoy Nix flakes for this. Only certain languages have good support though (C, Rust, Haskell, OCaml, ...).

[–] [email protected] 1 points 5 months ago (2 children)

I'm curious why you would think that containers are bloat? They require virtually no resources and are built into the kernel. A container is literally just a flag that you add when you exec on an executable binary.

[–] lazyneet 1 points 5 months ago (1 children)

Containers aren't too bad for storage from a developer's perspective. I'm talking about the dependency versioning bullshit of flatpak and snap specifically for end users. I don't know if AppImage technically counts as a container, but the whole point of it is to ship libraries the end user doesn't have, which implies a fundamental flaw in the hierarchical dependency tree or distribution model - the end user should already have everything they need to run software.

[–] [email protected] 1 points 5 months ago (1 children)

How can you guarantee that depencies are compatible across versions? That's a fundamental point I think you are missing.

[–] lazyneet 1 points 5 months ago

I don't use dependencies that don't have a history of backwards compatibility, and when I do, I ship them. It's SOP to assume basic things like a GUI "just work", and it's also SOP for Ubuntu to ship non-functional programs that were broken by GTK and Qt updates. I'd rather have buggy/broken software with undefined behavior than software that just doesn't run.

[–] [email protected] -1 points 5 months ago (1 children)

Sorry to interrupt but I want to point out that Flatpaks are not containers. It is just sandboxed with bubblewrap which isn't the same thing. It is closer to chroot and firejail.

For desktop containers use Podman and Distrobox

[–] [email protected] 2 points 5 months ago* (last edited 5 months ago) (1 children)

Flatpak absolutely does use containers for sandboxing. Bubblewrap is wrapper for Linux namespaces. Containers is just another name for the underlying kernel technology called namespaces. Same goes for Docker, LXC, Podman, systemd-nspawn, Firejail, etc. It's all just userland frontends for kernel namespaces. man bwrap, you can also use the generic unshare to create them and nsenter to enter those same namespaces. It's cool technology, it's very easy to use, a simple flag on your exec or opening of an existing fd is all that is required. I used to work on one of the many userland frontend, even have gotten a couple PRs from Jess Fraz who was one of the core Docker devs. Userns still scares the shit out of me (pretty much every single escape has come from them).

Here's a fun experiment for you: create a root fs using debootstrap and then enter it using unshare and chroot! Tada! Container!

[–] [email protected] 1 points 5 months ago* (last edited 5 months ago) (1 children)

I don't believe a chroot is a container. You are just switching root for the process. The same thing happens when you boot with a initramfs

It also might just be a terminology difference

[–] [email protected] 1 points 5 months ago

No a chroot is indeed not a container/namespace. I'm not sure what you're getting at here. Flatpak isn't a chroot and what I suggest you try isn't either.

[–] [email protected] 1 points 5 months ago

flatpak