FreeBSD on an OVH dedicated server
I run FreeBSD on an OVH dedicated server. It took me a long time to get a setup that works well there, one small setting at a time. This post collects all of them.
The install
OVH no longer offers FreeBSD in its installation catalog, so you are on your own from the install. I attached the FreeBSD ISO as a virtual device in the KVM of the control panel, booted it, and ran the installer by hand. Root is on ZFS, mirrored on the two NVMe disks, and encrypted with GELI.
GELI means the loader asks for a passphrase at every boot, so I need a console.
The console over SOL
The KVM works, but paste is unreliable, and I do not want to type a long passphrase by hand in a web page.
OVH also gives access to the serial console of the server over SOL (Serial over LAN), from the web or over SSH. Over SSH it is a normal terminal, which makes it much easier to have my password manager fill the passphrase.
On my current server, which boots with UEFI, there is nothing to configure. The firmware redirects its console to the first serial port, and the FreeBSD loader picks it up on its own:
$ kenv | grep cons
boot_multicons="YES"
comconsole_pcidev=""
comconsole_port="1016"
comconsole_speed="115200"
console="efi"
hw.uart.console="io:1016,br:115200"
1016 is 0x3F8, the first serial port (COM1). The kernel then uses
both consoles:
$ dmesg | grep -i console
uart0: console (115200,n,8,1)
Dual Console: Video Primary, Serial Secondary
On an older setup, SOL was on the second serial port (COM2), and I
needed these lines in /boot/loader.conf:
boot_multicons="YES"
boot_serial="YES"
comconsole_speed="115200"
comconsole_port="0x2F8"
console="comconsole,efi"
I found 0x2F8 by trial and error: change the port, reboot, check the
SOL session. If SOL stays blank on your server, try 0x3F8, then
0x2F8.
IPv4
The server has one IPv4 address, in a /24 with the gateway at .254.
OVH provides DHCP for it, but I configure it statically in
/etc/rc.conf, so the boot does not wait for a lease:
ifconfig_ixl0="inet 141.94.75.198 netmask 255.255.255.0 broadcast 141.94.75.255"
defaultrouter="141.94.75.254"
IPv6
IPv6 was the real pain. For years the server ran on IPv4 only, and I tried several times to make IPv6 work before it finally held.
OVH IPv6 is a running joke: forums are full of people whose IPv6 dies after a few hours, or whose system ignores the solicitations of the gateway.
The server has the /64 2001:41d0:303:ffc6::/64, and the gateway is
2001:41d0:303:ffff:ff:ff:ff:ff, outside of it. OVH documents a static
configuration only, without router advertisements or DHCPv6. It took
three fixes to get it working.
The gateway is not on-link
This configuration does not work:
ifconfig_ixl0_ipv6="inet6 2001:41d0:303:ffc6::1 prefixlen 64"
ipv6_defaultrouter="2001:41d0:303:ffff:ff:ff:ff:ff"
FreeBSD only considers neighbors inside the prefixes of the interface,
so it cannot reach a gateway outside of the /64. A route to the gateway
through ixl0 has to exist first:
ipv6_static_routes="gw6"
ipv6_route_gw6="-host 2001:41d0:303:ffff:ff:ff:ff:ff -iface ixl0"
IPv6 dies after a few hours
In 2024, IPv6 stopped working after a few hours. I thought the neighbor cache entry of the gateway expired and never came back, so I added a static NDP entry:
ipv6_ndp_interfaces="ixl0"
ipv6_ndp_entries="2001:41d0:303:ffff:ff:ff:ff:ff <gateway MAC address>"
It did not help, and I disabled IPv6 until 2026.
The cause was pf. My ruleset had a stateful rule for ICMPv6:
pass proto icmp6 keep state
Neighbor discovery uses ICMPv6, so the neighbor solicitations of the gateway created states. When the neighbor cache entry expired, a stale state dropped the reply of the gateway, and IPv6 was gone.
Neighbor and router discovery now pass without states, before any other rule:
pass quick inet6 proto icmp6 \
icmp6-type { neighbrsol, neighbradv, routersol, routeradv } no state
One connection in three hangs
After that, IPv6 stayed up, but about one connection in three never got a reply. FreeBSD counted the dropped packets:
$ netstat -s -p icmp6 | grep 'bad neighbor solicitation'
19569 bad neighbor solicitation messages
The gateway is a virtual address shared by several OVH routers. The neighbor cache shows it, with a Cisco HSRP virtual MAC address, next to the two routers behind it:
$ ndp -an
Neighbor Linklayer Address Netif Expire S Flags
2001:41d0:303:ffff:ff:ff:ff:ff 00:05:73:a0:00:01 ixl0 23h59m55s S R
2001:41d0:303:ffff:ff:ff:ff:fe bc:4a:56:af:20:f7 ixl0 23h43m21s S R
2001:41d0:303:ffff:ff:ff:ff:fd 3c:13:cc:d6:96:37 ixl0 23h47m58s S R
Before they deliver traffic to the server, these routers send a neighbor solicitation for its address, from their own address:
$ tcpdump -n -i ixl0 'icmp6 and ip6[40] == 135'
IP6 2001:41d0:303:ffff:ff:ff:ff:fe > 2001:41d0:303:ffc6::1: ICMP6,
neighbor solicitation, who has 2001:41d0:303:ffc6::1, length 32
Their addresses are outside of the /64 of the server. Only the gateway was on-link, so FreeBSD dropped these solicitations as coming from non-neighbors, and the traffic through these routers was lost.
The fix is to make the whole router prefix on-link:
ipv6_route_gw6="-net 2001:41d0:303:ffff::/64 -iface ixl0"
In hindsight, the addresses explain the whole mess. My /64,
2001:41d0:303:ffc6::/64, and the router prefix,
2001:41d0:303:ffff::/64, are both in 2001:41d0:303:ff00::/56. The
OVH routers live in the /56, not in the /64 they give you. Some people
configure the address with a /56
prefix instead,
which makes the gateway and the routers on-link in one go. I prefer to
keep the /64 that OVH assigns and add the route.
The final configuration
In /etc/rc.conf:
ipv6_activate_all_interfaces="YES"
ifconfig_ixl0_ipv6="inet6 2001:41d0:303:ffc6::1 prefixlen 64"
ipv6_static_routes="gw6"
ipv6_route_gw6="-net 2001:41d0:303:ffff::/64 -iface ixl0"
ipv6_defaultrouter="2001:41d0:303:ffff:ff:ff:ff:ff"
In /etc/pf.conf, before any other ICMPv6 rule:
pass quick inet6 proto icmp6 \
icmp6-type { neighbrsol, neighbradv, routersol, routeradv } no state
There is no static NDP entry anymore.
None of this looks hard once it works, but SOL and IPv6 took me a long time to get right. The OVH documentation is crap: it has no FreeBSD example, and none of these problems are in it.