Selfhosted

Just fyi - running TrueNAS with zfs as a VM under Proxmox is a recipe for disaster, as me how I know.

Zfs needs direct drive access, with VMs, the hypervisor virtualizes the adapter which is then passed through, which can mess things up.

What you'd need to do is buy a sata/sas card and pass the whole card through, then you can use a vm.

My best advice is use that your old setup hasn't died yet while you can. I.e. start now and setup Proxmox because it's vastly superior to TrueNAS for the more general type hardware you have and then run a more focused NAS project like Openmediavault in a proxmox VM.

My recommendation, from experience, would be to setup a VM for anything touching hardware directly, like a NAS or Jellyfin (if you want to have GPU assisted transcoding) and I personally find it smoothest to run all my Docker containers from one Docker dedicated VM. LXCs are popular for some but I strongly dislike how you set hardware allocations for them, and running all Docker containers in one LXC is just worse than doing it in a VM. My future approach will be to move to more dedicated container setup as opposed to the VM focused proxmox but that is another topic.

I also strongly recommend using portainer or similar to get a good overview of your containers and centralize configuration management.

As for external access all I can say is do be careful. Direct internet exposure is likely a really bad idea unless you know what you're doing and trust the project you expose. Hiding access behind a VPN is fairly easy if your router has a VPN server built in. And WireGuard (like Netbird / tailscale / Cloudflare tunnels etc all use) is great if not.

As for authentication it's pretty tricky but well worth it and imo needed if you want to expose stuff to friends/family. I recommend Authentik over other alternatives.

As a general rule: One system, one service. That system can be metal, vm, or container. Keeping things isolated makes maintenance much easier. Though sometimes it makes sense to break the rules. Just do so for the right reasons and not out of laziness.

Your file server should be it's own hardware. Don't make that system do anything else. Keeping it simple means it will be reliable.

Proxmox is great for managing VMs. Your could start with one server, and add more as needed to a cluster.

It's easy enough to setup wireguard for roaming systems that you should. Make a VM for your VPN endpoint and off you go.

I'm a big fan of automation. Look into ansible and terraform. At least consider ansible for updating all your systems easily - that way you're more likely to do it often.

I need everything to be fully but securely accessible from outside the network

I wouldn't be able to sleep at night. Who is going to need to access it from outside the network? Is it good enough for you to set up a VPN?

The more stuff visible on the internet, the more you have to play IT to keep it safe. Personally, I don't have time for that. The safest and easiest system to maintain a system is one where possible connections are minimized.

The right way is the way that works best for your own use case. I like a 3 box setup, firewall, hypervisor, nas, with a switch in between. Let's you set up vlans to your heart's content, manage flows from an external point (virtual firewalls are fine, but if it's the authoritative DNS/DHCP for your net it gets a bit chicken and egg when it's inside a vm host), and store the actual data like vids/pics/docs on the NAS that has just that one job of storing the files, less chance of borking it up that way.

Not sure why you need a new router for PiHole. If your machines all point to the Pihole for DNS, it works. Router has almost nothing to do with what provides DNS, other than maybe having it's DHCP config include the Pihole for DNS.

Even then, you can setup the Pihole to be both DHCP and DNS (which helps for local name resolution anyway), and then just turn off DHCP in your router.

As I understand it, Tailscale and Nginx fulfill the same requirements. I lean toward TS myself, I like how administration works, and how it's a virtual network instead of an in-bound VPN. This means devices just see each other on this network, regardless of the physical network to which they're connected. This makes it easy to use the same local-network tools you normally use. For example, you can use just one sync tool, rather than one inside the LAN, and one that can span the internet. You can map shares right across a virtual network as if it were a LAN. TS also enables you to access devices that can't run TS, such as printers, routers, access points, etc, by enabling its Subnet Router.

Tailscale also has a couple features (Funnel and Share) which enable you to (respectively), provide internet access to specific resources for anyone, or enable foreign Tailscale networks to access specific resources.

I see Proxmox and TrueNAS as essentially the same kind of thing - they're both Hypervisors (virtualizatiin hosts) with True adding NAS capability. So I can't think of a use-case for running one on the other (TrueNAS has some docs around virtualizing it, I assume the use-case is for a test lab, I wouldn't think running TN, or any NAS, virtualized is an optimal choice, but hey, what do I know? ).

While I haven't explored both deeply, I lean toward TrueNAS, but that's because I need a NAS solution and a hypervisor, and I've seen similar solutions spec'd many times for businesses - I've seen it work well. Plus TrueNAS as a company seems to know what they're doing, they have a strong commercial arm with an array of hardware options. This tells me they are very invested in making True work well, and they do a lot of testing to ensure it works, at least on their hardware. Having multiple hardware products requires both an extensive test group and support organization.

Proxmox seems equivalent, except they do just the software part, as far as I've seen.

Two similar products for different, but similar/overlapping use-cases.

Best advice I have is to make a list of Functional Requirements, abstract/high-level needs, such as "need external access to network for management". Don't think about specific solutions, just make the list of requirements. Then map those Functional requirements to System requirements. This is often a one-to-many mapping, as it often takes multiple System requirements to address a single functional requirement.

For example, that "external access" requirement could map out to a VPN system requirement, but also to an access control requirement like SSO, and then also to user management definitions.

You don't have to be that detailed, but it's good to at least have the Functional-to-System mapping so you always know why you did something.

I will provide a word of advice since you mentioned messiness. My original server was just one phyiscla host which I would install new stuff to. And then I started realizing that I would forget about stuff or that if I removed something later there may still be lingering related files or dependencies. Now I run all my apps in docker containers and use docker-compose for every single one. No more messiness or extra dependencies. If I try out an app and don’t like it, boom container deleted, end of story.

Extra benefit is that I have less to backup. I only need to backup the docker compose files themselves and whatever persistent volumes are mounted to each container.

If z2m, zwavejs,... Are installed from the adon store of HA, all you have to do is create a full backup of HA, and all your automations will be saved and restored automatically.

I'll freely admit to skimming a bit but yes proxmox can run trunas inside of it. Proxmox is powerful but might be a little frustrating to learn at first. For example by default proxmox expects to use the boot drive for itself and it's not immediately clear how to change that to use that disk for other things.

The noctua dh-15 is overkill for that cpu btw unless you're doing an overclock which I wouldn't recommend for server use. What's your plans for the 1060? If using proxmox you'll want to get one of the "G" series AMD CPUs do that proxmox binds to the apu and then you should be able to do gpu passthrough on the 1060.

For ease of setup and use, I've found Twingate to be great for outside access to my network.

I have had a lot of fun setting up Unraid. It has everything you are looking for. It does cost some money to start, but was very much worth it to me.

yunohost; setup modules, are custom modules (there is a program made to do so from source files; infact theres even a yunohost module of some form for that (and even if that was only a template; any programming & related module (including anything from basic ide to full on llm assists; are even just a fullon system like turbopilot; in a vm module), can be used to make a module for that if needed; im certain vm modules & thus yunohost and modules for it can be stacked, and also general preexisting programming modules can be repurposed for (if those and pre-repurposed versions dont exist already) doing so already; if not in setup then inside said module & then that current modules in current configuration, can be exported & keep such config, so varients can be made; same for the entire yunohost system)