[HamWAN PSDR] DNS for HamWAN

[Bart Kus]

Hi,

We've had recursive DNS services running on the network for a couple 
weeks now.  These are available on the following IPs: 44.24.244.1 and 
44.24.245.1.  As long as you're coming from 44/8, these IPs will honor 
your requests.  It's a semi-fancy config in that these are anycast IPs, 
and not bound to any specific server.  If one DNS server fails, others 
automatically take over without the users having to change the IPs they 
have configured!  It was designed like this both for simplicity, and so 
that the system keeps running in case of emergency and links/cells going 
down.  No operator action is required to make this work, everything is 
automatic on both failover and failback.  Requests are always routed to 
the nearest available server to save on latency + resources.  The server 
setup is documented here for anyone interested in making use of this 
approach:

https://www.hamwan.org/t/tiki-index.php?page=Servers&structure=HamWAN

Oh, and the software fully supports DNSSEC!

ANYWAYS, that's recursive DNS, and that's really easy.  Now we face the 
challenge of offering authoritative DNS services.  These have the 
problem of possessing and disseminating global state.

With the help of several other people over the last 2 weeks, I believe 
I've now concluded what is a good draft design of the authoritative DNS 
system.  It goes a little something like this:

PowerDNS + PostgreSQL + SymmetricDS

PowerDNS is the authoritative DNS server software, and it will ride an 
anycast IP pair (44.24.244.2 / 44.24.245.2).  Instead of storing its 
data in zone files, it'll use a SQL back-end.  This is to ease 
interaction with the data.  We don't have to write custom parsers if 
users want to edit their own entries in these zone files. Everything is 
strictly parsed already into SQL rows and columns, and this allows the 
easy creation of UI front-ends to interact with the DNS data.  Each 
PowerDNS instance will have its own local PostgreSQL instance to use.  
The PostgreSQL service will also be available via anycast so that any 
applications interacting with the DB will not need location-dependent 
configs.

Now, for the tricky part.  A change to the data set can come from any 
PostgreSQL instance by way of the anycast routing there.  This change 
must be propagated to all the other instances.  This is typically known 
as a multi-master config, and is notoriously difficult to do.  This is 
where the SymmetricDS software comes in. It gets triggered (quite 
literally, with SQL TRIGGERs) whenever a change is made, and it then 
tries to propagate that change to all the other DB servers.  The 
presence of this software is invisible to the PostgreSQL users.  It 
handles updates without the need for all nodes to be connected.  When 
nodes re-connect, it sends whatever queued updates there were to the 
specific nodes which re-connected. Should there be conflicts in the data 
(due to the disconnected nodes taking on conflicting changes during the 
disconnect), it provides several resolution strategies to handle them.  
We can start with a simple strategy, something like "latest change 
wins", and develop more appropriate custom strategies as time goes on.

All the pieces of the puzzle support PAM + non-password authentication 
schemes which make them compatible with amateur licensing.

While this all sounds great, I've never played with SymmetricDS or 
PowerDNS, and the last time I touched PostgreSQL I still had long 
flowing Jesus hair.  So, if anyone on the list has any guidance to give 
here, now's a great time.  Will this actually work as envisioned?  Is 
there some huge pitfall in this design?  Is there a much simpler way to 
achieve the goals of operator-free failover + failback?

Let me know your thoughts,

--Bart

PS: There are other purposes for SQL in the network aside from DNS. It's 
important to solve this problem correctly once and for all.