1:1 stateless NAT is useful for static IPs. Since all your addresses are otherwise global, if you need to switch providers or give up your /64, then you'll need to re-address your static addresses. Instead, you can give your machines static private IPs, and just translate the prefix when going through NAT. It's a lot less horrible than IPv4 NAT since there's no connection tracking needed.

This is something I probably should have done setting up my home Kubernetes cluster. My current IPv6 prefix is from Hurricane Electric, and if my ISP ever gives me a real IPv6 prefix, I will have to delete the entire cluster and recreate it with the new prefix.

[–] [email protected] 7 points 4 months ago (4 children)

It should only be needed if your ISP is brain-dead and only gives you a /64 instead of what they should be doing and also giving you a /56 or /48 with prefix delegation (I.e it should be getting both a 64 for the wan interface, and a delegation for routing)

You router should be using that prefix and sticking just a /64 on the lan interface which it advertises appropriately (and you can route the others as you please)

Internal ipv6 should be using site-local ipv6, and if they have internet access they would have both addresses.

[–] LaggyKar 2 points 4 months ago* (last edited 4 months ago) (1 children)

64 for the wan interface

Nitpicking, but the address for the wan interface wouldn't have a prefix, so the host would just set it as a /128 (point-to-point)

[–] [email protected] 1 points 4 months ago

Ehh, I've seen both. Perhaps not in a home router context though, never really bothered to check

[–] [email protected] 2 points 4 months ago* (last edited 4 months ago) (1 children)

Hurricane Electric gives me a /48.

Site-local ipv6 would work here as well, true. But then my containers wouldnt have internet access. Kubernetes containers use Ipam with a single subnet, they can't use SLAAC.

[–] [email protected] 1 points 4 months ago (1 children)

Point is, you should be able to have them have both. Or stick a reverse proxy in front that can translate. Unless they're somehow meant to be directly internet reachable the public addresses could be autogenerated

Full disclosure though I don't know anything about kubernetes.

[–] [email protected] 1 points 4 months ago (1 children)

Yeah, I wonder if there's any proposals to allow for multiple IPV6 addresses in Kubernetes, it would be a much better solution than NAT.

As far as I know, it's currently not possible. Every container/Pod receives a single IPv4 and/or IPv6 address on creation from the networking driver.

[–] [email protected] 2 points 4 months ago (1 children)

Even if you give it multiple ip6 cidrs to allocate?

[–] [email protected] 1 points 4 months ago (1 children)

I haven't really looked into it, but it doesn't seem like it.

Heres the documentation about having multiple cidr pools in one cluster with the Cilium network driver, and it seems to imply that each Pod only gets one IP.

https://docs.cilium.io/en/stable/network/concepts/ipam/multi-pool/

There's something called Multus that I haven't looked into, but even then it looks like that is for multiple interfaces per Pod, not multiple IPS per interface.

https://github.com/k8snetworkplumbingwg/multus-cni

Containers are just network namespaces on Linux, and all the routing is done in iptables or ebpf, so it's theoretically possible to have multiple IP addresses, but doesn't look like anybody has started implementing it. There's actually a lot of Kubernetes clusters that just use stateful IPv6 NAT for the internal Pod network, unfortunately.

[–] [email protected] 2 points 4 months ago

Weird! Though I guess a lot of these would be sitting behind load balancers / reverse proxies anyway (so ipv4 is fine) and unlikely to up and change isps very often? Lol

If I had to, I'd be trying to add an extra cidr to one of the options listed at https://kubernetes.io/docs/concepts/services-networking/dual-stack/#configure-ipv4-ipv6-dual-stack But as you say, probably doesn't work!

[–] [email protected] 2 points 4 months ago (1 children)

And if you want static ips either use dhcp6 or disable the randomisation of eui64 addresses

[–] [email protected] 2 points 4 months ago (1 children)

I have static IPs for my Kubernetes nodes, and I actually use DHCPv6 for dynamic dns so I can reach any device with a hostname, even though most of my devices don't have static IPs.

The issue is those static IPs are tied to my current ISP, preventing me from changing ISPs without deleting my entire Kubernetes cluster.

[–] [email protected] 1 points 4 months ago

It certainly looks like you can give it a prefix to assign out, possibly even multiples

[–] [email protected] 1 points 4 months ago* (last edited 4 months ago) (1 children)

My ISP does this right (provides a /56 for routing), but unfortunately both are dynamic and change periodically. Every time I disconnect and reconnect from the internet, I get a different prefix.

I ended up needing to have ULAs for devices where I need to know the IPv6 address on my network (e.g. my internal DNS servers).

[–] [email protected] 1 points 4 months ago (2 children)

Indeed, that's correct ula usage, but shouldn't need nat rewriting. The global prefixes just need to be advertised by RA packets

[–] [email protected] 1 points 4 months ago (1 children)

I use openwrt on my home network which uses dnsmasq for dhcp. It can give a static suffix which just works with the global prefix on the interface and the site local / ula prefix it uses

[–] [email protected] 1 points 4 months ago (1 children)

Note that Android doesn't support DHCPv6, just in case you have Android devices and ever have to debug IPv6 on them.

[–] [email protected] 1 points 4 months ago

Yup indeed. That's why it advertises both dhcp and slaac

[–] [email protected] 1 points 4 months ago* (last edited 4 months ago)

Yeah I'm not using NAT, sorry for the confusion.

My router doesn't support RAs for a ULA range though, so I'm running radvd on my home server.