Installing OpenWRT on top of LXC in Proxmox
I recently realized that I don't get the full speed promised by my ISP, and the reason behind it is that I run my OpenWRT router in KVM. I do not have a Ryzen to run my Proxmox and instead of shelling some cash to get one, I decided to try to run OpenWRT on LXC instead.
OpenWRT on LXC is not supported either by OpenWRT nor Proxmox, but there are multiple successful cases, plus an image is available on vanilla LXC, so I decided to give it a try.
I am documenting here what works for me as they differ from what was being documented. Maybe this would encourage providers here to offer an OpenWRT template on their Proxmox host 
If you decide to follow what I wrote here, remember YMMV.
The combination that works: Proxmox VE 6.3-3, privileged container, OpenWRT Snapshot (20201216).
# pveversion pve-manager/6.3-3/eee5f901 (running kernel: 5.4.78-2-pve)
I couldn't make it work with unprivileged containers nor the latest stable.
First download the rootfs.tar.gz from https://downloads.openwrt.org/snapshots/targets/x86/64/.
Next, get opct.sh from https://github.com/DazzyWalkman/oplxc4pve. This will make container creation and update much, much easier.
Open EDITOR (vi, nano, emacs,...) and change parameters to fit your Proxmox install. You'll also define how much memory, swap, cpu cores you'll allocate to this container here. Remember to change unprivileged to '0' or it may not work.
Pay attention to the bind mount settings. It came biting me when I restarted my Proxmox machine and found that the OpenWRT container refused to start, since I do not have the host directory set up. I am not sure if commenting it out would work, but it is worth a try.
Once you're done editing, give opct.sh executable privileges and copy to $PATH.
Next, run opct.sh new <VMID> <LOCATION OF OPENWRT ROOTFS>
An OpenWRT container will appear on Proxmox's CP. All you need to do now is rename the container (in DNS), add network interfaces, and start the container to configure OpenWRT to your needs.
The all seeing eye sees everything...
Comments
That's weird, do you know why that was?
Thanks for the guide, however I fail to see the point of running OpenWRT on beefy x86_64 hardware. The whole point of OpenWRT is to make a distribution viable for 4MB 8MB TP-Link routers. Why would you run that on a beefy piece of hardware, when you can use a Linux distribution with way larger selection of software instead?
Probably KVM overheads. Even with virtio I am unable to push line speed.
I run a few iperf tests to reach that conclusion.
Here is what I observed (IIRC):
Host to LXC: 20+ Gbps
LXC to LXC: 20+ Gbps
LXC to KVM: 40+ Mbps (yes M, not G)
KVM to KVM: 10-20 Mbps
Because OpenWRT's origin on small ARM routers makes it an extremely efficient linux distro for networking. I had PfSense and OPNSense as my main routers in the past. While they are feature rich, they take minutes to boot. OpenWRT on x64, on the other hand, takes less than a minute to boot on KVM, even less on LXC.
OpenWRT also comes with all the routing related kernel parameters turned on. These are usually turned off for general purpose distros.
The number of packages you can install is limited by the amount of flash available on the ARM router. This limit is non existent on x64. There are thousands of packages available officially on OpenWRT's repo (see https://downloads.openwrt.org/snapshots/packages/x86_64/) and more if you count 3rd party repositories.
I won't run everything I need run on the OpenWRT. I have other LXC/KVMs for that. For those I need run on OpenWRT, I haven't found any that wouldn't.
The all seeing eye sees everything...
Everything about that sounds odd.
The proxmox bridge runs at 10gbps afaik & KVM overheads should be minimal. I'm guessing there is some element of hardware acceleration that is being passed through to LXC but not KVM?
Which Linux distribution do you recommend for routers? Do they have DHCPv6-PD clients, and servers and are able to receive IPv6 prefixes from the upstream routers, which they split into sub-prefixes that are assigned to interfaces, or delegated to other routers using its DHCPv6 server?
I am not sure...
The all seeing eye sees everything...
I agree with that - OpenWRT works wonders for what it's supposed to do. The thing is, once you go from a simple setup to a more advanced one, you will be limited by the networking scripts, and depending on your requirements, you might be after a more complex network management solution, like systemd-networkd.
OpenWRT has a great selection of network-related packages, however, they are sometimes lacking in features that are normally found on standard Linux distributions.
I would say my biggest problem with OpenWRT is updating - feature-rich package managers that are found on modern desktop/server distributions have no problems with updating even to a new release. But with OpenWRT, your best bet is to compile an image of its new version and reflash/reinstall. Even updating single packages with opkg can be problematic and is generally not recommended.
Depends what you need. If you don't need other functionality that is not baked into OpenWRT, there is probably no reason to use something else. OpenWRT is great for the basic routing it provides.
I, for example, use OpenWRT on my Wi-Fi APs and simple routers. But, for my main router, where I have a ton of tunnels, ip rules, ip addresses, network services and bunch of other crap, OpenWRT would be a nightmare.
I think you can use just about any general-purpose/server distributions. My heart in this case leans close to Debian - due to the stability and huge software repos it provides.
systemd-networkd, which is by default in every major modern distribution, meets all your requirements.
Why do you need non networking related packages on OpenWRT? It's not like I am going to develop the Next Big Thing(TM) there!
For upgrading OpenWRT on x86, all you need to do is download the kernel, place it on /boot/vmlinuz, then manually download and install the kernel.ipk which is just a flag to tell the package manager that you have the right kernel, reboot, update repos, then upgrade all packages. No recompiling, no reflashing.
Luckily I don't have anything that complex, so OpenWRT works for me. I'll probably switch to Alpine Linux if I ever graduated from OpenWRT. I currently run a few Debian VPSs from the great providers here.
The all seeing eye sees everything...
There is no DHCPv6 server in systemd anyway. A DHCPv6 server with prefix delegation was one of the requirements.
Open issue on systemd: network: implement DHCPv6 Server #16039