为租户下的虚机提供IPv6 DNS服务(by quqi99)

问题

当虚机运行下列代码时，我们需要考虑为tenant下的VM提供DNS服务:

import dns
import dns.resolver
answers = dns.resolver.query('node1', 'AAAA')
print answers[0].address

思路

需要截获neutron port event把IP/MAC拿到写到DNS的record中去。neutron port表的fixed_ips字段（在neutron ipallocations表里）同时记录了VM的IPv4与IPv6(netaddr.IPNetwork(fixed_ip[‘ip_address’]).version == 6)地址 [1]，neutron dns_integration特性可以从这里面将IPv4与IPv6地址都取出来记录到neutron-dhcp-agent下的dnsmasq中从而实现ml2-dns内置DNS服务。

IPv6

如果只是让OpenStack tenant network支持IPv6的话，很简单，直接用下列命令（下列命令有两个重要属性：ipv6_address_mode 与 ipv6_ra_mode）。当然，如果是OpenStack Over Openstack环境的话，可以让底层provider network也支持IPv6。

neutron subnet-create --ip-version=6 --name=zhhuabj_admin_subnet_v6 --ipv6-address-mode=slaac --ipv6-ra-mode=slaac zhhuabj_admin_net 2001:db8:0:1::/64
neutron router-interface-add zhhuabj_router zhhuabj_admin_subnet_v6

上面命令相当于：

$ cat /var/lib/neutron/ra/5c33033b-a4e1-494d-ab20-e0498b423b6c.radvd.conf
interface qr-10bb0b85-53
{AdvSendAdvert on;MinRtrAdvInterval 30;MaxRtrAdvInterval 100;AdvLinkMTU 1458;prefix 2001:db8:0:1::/64{AdvOnLink on;AdvAutonomous on;};
};$ sudo ip netns exec qdhcp-be442335-ec55-4df8-b68d-dd03fa6edf00 ps -ef|grep radvd
root     16114     1  0 Nov01 ?        00:00:00 radvd -C /var/lib/neutron/ra/5c33033b-a4e1-494d-ab20-e0498b423b6c.radvd.conf -p /var/lib/neutron/external/pids/5c33033b-a4e1-494d-ab20-e0498b423b6c.pid.radvd -m syslog
root     16115 16114  0 Nov01 ?        00:00:00 radvd -C /var/lib/neutron/ra/5c33033b-a4e1-494d-ab20-e0498b423b6c.radvd.conf -p /var/lib/neutron/external/pids/5c33033b-a4e1-494d-ab20-e0498b423b6c.pid.radvd -m syslog
$ sudo ip netns exec qrouter-5c33033b-a4e1-494d-ab20-e0498b423b6c ip addr show qr-10bb0b85-53 |grep inet6 |grep globalinet6 2001:db8:0:2::1/64 scope global
$ sudo ip netns exec qdhcp-be442335-ec55-4df8-b68d-dd03fa6edf00 ip addr show ns-af35afad-b2 |grep inet6 |grep globalinet6 2001:db8:0:2:f816:3eff:feef:5190/64 scope global

如果它不work的话，多半两个原因：
1, 底层OpenStack环境 (openstack over openstack)计算节点上的security group应该disable掉, 因为它有类似这种固定的anti-dhcp-spoof for ipv6规则.

-A neutron-openvswi-od7c63bee-9 -p udp -m udp --sport 547 --dport 546 -j DROP                        #Anti-dhcp-spoof for IPv6

2, radvd server端, 至少得有下列防火墙规则:

sudo ip6tables -A INPUT -p icmpv6 -j ACCEPT
sudo ip6tables -A OUTPUT -p icmpv6 -j ACCEPT
sudo ip6tables -A FORWARD -p icmpv6 -j ACCEPT
sudo ip6tables -A INPUT -p udp --dport 546:547 -j ACCEPT
sudo ufw allow proto udp from fe80::/64 to any port 547

最好测试时先临时采用:

sudo ip6tables -F
sudo ufw disable
sudo sysctl -w net.ipv6.conf.all.forwarding=1

3, radvd进程是否正常启动

附1：若是juju搭建的OpenStack环境要enable IPv6支持的话直接在yaml里添加下列内容即可：

overrides:prefer-ipv6: true

附2：使用OpenStack IPv6环境时，直接设置OS_AUTH_URL环境变量指向keystone的IPv6地址即可。

export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://[2001:db8:0:1:f816:3eff:fe3e:5e47]:5000/v2.0

Enable ML2-DNS

该特性有dns_name与dns_domain两个重要的属性，dns_domain可用在network与floatingip中，dns_name可用在port和floatingip中。如创建network时指定dns_name (neutron port-create my-net --dns_name my-port), 这样该dns_name和IP会作为dns record。

检查OpenStack是否支持dns extention API。

neutron ext-list |grep dns
| dns-integration           | DNS Integration

如果不支持，可以修改下列两个文件去支持，dns_domain相当于dnsmasq给不同组织的IP提供DNS服务时的一个区别标志。

vi /etc/neutron/neutron.conf
dns_domain = example.org.
vi /etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
extension_drivers = port_security,dns

如果OpenStack是由juju创建，直接使用下列命令即可enable上述两个配置。

juju config neutron-api enable-ml2-dns
juju config neutron-api enable-ml2-dns=True

Test ML2-DNS

#sudo ip addr del 2001:db8:0:122::1/64 dev ens3
dig google.com @<DNS-SERVER> -p 53 AAAA
sudo tcpdump -ni ens3 ip6 host fe80::f816:3eff:feb4:8d1f
#  tcpdump -n -i ens3 icmp6 and ip6[40] == 134
sudo dhclient -6 -d ens3ubuntu@bionic:~$ sudo tcpdump -ni ens3 ip6 host fe80::f816:3eff:feb4:8d1f
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens3, link-type EN10MB (Ethernet), capture size 262144 bytes
13:35:28.263222 IP6 fe80::f816:3eff:feb4:8d1f.546 > ff02::1:2.547: dhcp6 solicit

designate架构

designate的架构如上图:

designate-api, 接收来自远端用户的HTTP/HTTPS请求，通过Keystone验证远端用户的合法性，将HTTP/HTTPS请求传递给Central模块。
designate-sink, 监听来自Nova和Neutron的某些事件，用于自动生成域名资源记录，比如当监听到Nova的compute.instance.create.end事件通知后，自动创建一条对应于刚创建的实例的A记录；当监听到Nuetron的floatingip.update.end事件通知后，自动更新一条相应的A记录。
designate-central, 业务逻辑处理核心。响应API请求以及处理Sink所监听到的来自Nova和Neutron的特定通知事件。同时会存取数据库，对业务逻辑处理所产生的数据进行持久化存储。
designate-mdns, 实现了标准的DNS Notify和Zone Transfer的处理. designate-mdns is the service that sends DNS NOTIFY and answers zone transfer (AXFR) requests. This allows Designate to integrate with any DNS server that supports these very standard methods of communicating. designate-mdns also encapsulates all other forms of DNS protocol that Designate performs. For example, sending SOA queries to check that a change is live.
designate-pool-manager, 连接后端驱动，管理DNS服务器池，与MiniDNS(即designate-mdns)配合同步DNS服务器的域名以及资源记录等数据。

MiniDNS(designate-mdns)：Hidden Master设计
https://blog.csdn.net/andyron/article/details/46053241
Hidden Master是DNS网络安全管理系统设计中所推荐的一种最佳实践。主DNS服务器“隐藏”在内网防火墙背后，负责DNS域名资源的管理并同步变更到从DNS服务器；从DNS服务器部署在DMZ区域，对外提供DNS查询服务。由于主DNS服务器不接受DNS查询，增强了安全性。Designate MiniDNS功能模块就采用了Hidden Master的设计思想。所有托管到Designate中的DNS域都将MiniDNS视为主DNS服务器，而其被委托的DNS服务器都作为从DNS服务器。MiniDNS实现了标准的DNS Notify和Zone Transfer协议，负责同步DNS域名资源记录到从DNS服务器上。
其工作流程如下图:

首先，用户通过Desingate API创建一个example.com的DNS域；
Designate API将请求传递给Central，Central先将example.com域保存到数据库，接着发送RPC请求给Pool Manager；
Pool Manager收到来自Central的创建域名的请求之后，调用DNS后端驱动，在该域名被委托的服务器池中的所有服务器中创建example.com域。同时在这些服务器中，指定example.com的master服务器是MiniDNS；
Pool Manager完成所有从服务器上example.com域的创建之后，发送RPC请求给MiniDNS。
MiniDNS收到Pool Manager的RPC请求之后，向从服务器发送DNS Notify消息，告诉从服务器example.com有资源更新。
从服务器收到DNS Notify消息后，要求主从数据库启动Zone Transfer，域迁移的方式可以是AXFR，也可以是IXFR。
主服务器从数据库中读取为example.com域自动创建的SOA和NS记录，并将SOA和NS记录传送到从服务器。
后续任何对example.com域的变更操作都会遵循上述过程，由MiniDNS将变更同步到Designate所委派管理example.com域的DNS服务器上。

看一下代码结构, designate支持很多backend(eg: bind), 安装bind服务的机器上可使用rndc命令行工具create/delete zone remotely. The traffic between rndc and bind/named(953/tcp) is authenticated with a key. designate将为每个pool生成下列配置, 这样就可以远程运行rndc命令了(rndc -s 10.5.0.29 -p 953 -k /etc/designate/rndc.key status), 其中5353是designate-mdns监听的端口:

# cat /etc/designate/pools.yaml
- id: 794ccc2c-d751-44fe-b57f-8894c9f5c842name: defaultdescription: Pool genergated by Jujuns_records:- hostname: openstack-au-east-2.oc.xxx.com.priority: 10nameservers:- host: 10.5.0.29port: 53targets:- type: bind9masters:- host: 10.5.0.23port: 5354options:host: 10.5.0.29rndc_host: 10.5.0.29rndc_key_file: /etc/designate/rndc.keyalso_notifies: []

在designate-bind节点上装有bind服务(运行在953端口, /usr/sbin/named -f -u bind), 需要确保bind能够访问/etc/bind/named.conf和/etc/bind/rndc.key, 并且能够接受从Pool Manager过来的rndc流量:

# cat /etc/bind/named.conf
include "/etc/bind/named.conf.options";
include "/etc/bind/named.conf.local";
include "/etc/bind/named.conf.default-zones";
controls {inet 127.0.0.1 allow {localhost;};inet 10.5.0.29 allow { 10.5.0.23; };
};
# cat /etc/bind/named.conf.options
options {directory "/var/cache/bind";dnssec-validation auto;auth-nxdomain no;    # conform to RFC1035listen-on-v6 { any; };allow-new-zones yes;request-ixfr no;recursion no;statistics-file "/var/cache/bind/named.stats";zone-statistics yes;allow-notify { 10.5.0.23; };
};

rndc命令:

rndc querylog
rndc status
dig -t A openstack-au-east-2.oc.xxx.com@10.5.0.23
dig -t MX openstack-au-east-2.oc.xxx.com@10.5.0.23

bind DNS服务器可作为缓存服务器, 主DNS服务器和辅助DNS服务器, 配置分配如下:

# 缓存服务器, 不负责解析，仅为加速，不需要注册
options {forward only;                    forwarders {                     168.95.1.1;139.175.10.20;};
};
# 主DNS服务器, 负责解析本地客户端请求
zone "test.com" IN {type master;file "test.com.zone";
};
# 辅助DNS服务器, 辅助服务器的区域数据都是从主服务器复制而来，其数据都是只读的. 根据序列号大小决定是否复制
zone "test.com" IN {type slave;masters {ip;};file "slaves/test.com.zone";
};

区域传送, 解析库文件同步的过程，即辅助DNS服务器从主DNS服务器或其他的辅助DNS服务器请求数据传输过程。

完全区域传送：传送区域的所有数据，简称AXFR
增量区域传送：传送区域中改变的数据部分，简称IXFR
bind配置中之DNS主从同步，区域安全传送
http://www.it165.net/admin/html/201403/2548.html

附件 - 一个designate问题

可能因为designate不断地升级, 而且是多网卡, 这样导致每个designate-bind unit所指定的master ip (designate unit)不是同一子网. 这样会导致create zone的时候有时候会发两次rndc addzone命令.

- designate/7 only programs designate-bind/3 [172.18.248.94] (it does not know about designate-bind/2). It also uses the .247 subnet for the Master IPs instead of the correct 248 subnet.
- designate/8 programs both bind9 servers [172.18.248.94, 172.18.248.99] and uses the correct .248 subnet
- designate/9 only programs designate-bind/3 and also uses the wrong master IPs.

并且因为之前的DB问题, 导致bind cache里有很多stale zone, 这样当create zone的时候, 会说zone已存在.

How to remove stale zones from bind9 dynamic zone configuration1,check the list of zones in bind9 like so:
cat /var/cache/bind/*.nzf|grep ^zone|awk '{print $2}'|sed 's/"$/.",/g' #the sed here adds a trailing dot so we can query the database for the same name
2, check if this zone is currently active by querying the database:
mysql -u root -p designate
select * from zones where name="xxx.openstack-au-east-2.oc.xxx.com." WHERE deleted_at IS NOT NULL;
3, rndc delzone xxx.openstack-au-east-2.oc.xx.com

designate recovery service检测到一个zone长期为pending状态时, 再会继续create zone. 这样在日志中会看到一个zone被反复多次create

附件 - Designate环境搭建及测试

ml2dns只能使用neutron.conf配置文件里的域名(如:ml2dns.example.), network里可以配置外部多个顶级不固定的域名(如:neutron net-update private --dns_domain extdns.example.).
ml2dns只为internal fixed ip在dnsmasq里配置域名, designate只为external fip配置域名, dnsmasq里需要配置forwarder到designate-bind中去.

# https://paste.ubuntu.com/p/y5NQwXB95R/
./generate-bundle.sh --name heat -s xenial -r queens --num-compute 1 --heat --designate
juju deploy ./b/heat/openstack.yaml --overlay ./b/heat/o/neutron-gateway.yaml --overlay ./b/heat/o/heat.yaml --overlay ./b/heat/o/neutron-ml2dns.yaml --overlay ./b/heat/o/memcached.yaml --overlay ./b/heat/o/designate.yaml
juju config neutron-api dns-domain=ml2dns.example.
juju config neutron-api reverse-dns-lookup=True
juju config neutron-api enable-ml2-dns=True
juju config designate nameservers=ns1.extdns.example.
./configure
./tools/sec_groups.shapplications:neutron-api:charm: cs:~openstack-charmers-next/neutron-apioptions:enable-ml2-dns: Truedns-domain: ml2dns.example.reverse-dns-lookup: Trueipv4-ptr-zone-prefix-size: 24designate:charm: cs:~openstack-charmers-next/designateoptions:nameservers: ns1.extdns.example.         designate-bind:charm: cs:~openstack-charmers-next/designate-bindrelations:- [ designate, designate-bind ]- [ designate, neutron-api ]# https://docs.openstack.org/python-designateclient/latest/user/shell-v2.html
sudo apt install -y python-openstackclient python-designateclient# create the zone for your external dns, the zone name must end in a '.'
DOMAIN_NAME=extdns.example.
openstack zone create --email extdns@example $DOMAIN_NAME# create an 'A' record for the automaticall created NS record that points to one or more of the designate-bind units
DESIGNATE_BIND_IP=$(juju status designate-bind --format short | awk '/designate-bind/ {print $3}')
openstack recordset create --record $DESIGNATE_BIND_IP --type A $DOMAIN_NAME ns1
dig @$DESIGNATE_BIND_IP -type NS# tell neutron that for any floating IPs for our tenant network create the the hostname in a given DNS domain
neutron net-update private --dns_domain $DOMAIN_NAME# tell neutron to use designate bind as it's forwarder by dnsmasq's configuration '--server=10.5.0.16 --domain=ml2dns.example.'
# query path: Project instance -> Neutron dnsmasq -> Designate Bind -> External DNS
# NOTE: must DO NOT SET –dns-nameserver on the tenent subnet
juju config neutron-gateway dns-servers=$DESIGNATE_BIND_IP# create a VM for integration test
nova keypair-add --pub-key ~/.ssh/id_rsa.pub mykey
openstack port create --network $(neutron net-show private -f value -c id) --dns-name i1 port_i1
$ openstack port show port_i1 -f value -c dns_assignment
fqdn='i1.ml2dns.example.', hostname='i1', ip_address='192.168.21.18'openstack server create --wait --image cirros2 --flavor m1.tiny --key-name mykey --port $(openstack port show port_i1 -f value -c id) i1
public_network=$(openstack network show ext_net -f value -c id)
fip=$(openstack floating ip create $public_network -f value -c floating_ip_address)
openstack floating ip set $fip --fixed-ip-address 192.168.21.18 --port $(openstack port list --fixed-ip ip-address=192.168.21.18 -c id -f value)# Neutron will create forward and reverse records in dnsmasq for the internal DNS.
# NOTE: must DO NOT SET –dns-nameserver on the tenent subnet, By not setting the dns server
#       the project network will default to using the dnsmasq service associated with the network.
#       This is Neutron handling our project internal DNS records automatically.
openstack subnet unset --dns-nameserver 10.230.64.2 private_subnet
nova boot --hard i1# test dns inside VM, 192.168.21.2 is the IP inside qdhcp-xxx namespace
$ ssh cirros@10.5.150.2 -- cat /etc/resolv.conf
search ml2dns.example
nameserver 192.168.21.2$ ssh cirros@10.5.150.2 -- nslookup i1.ml2dns.example 192.168.21.2
Server:    192.168.21.2
Address 1: 192.168.21.2 host-192-168-21-2.ml2dns.example
Name:      i1.ml2dns.example
Address 1: 192.168.21.18 i1.ml2dns.example$ juju ssh neutron-gateway/0 -- cat /var/lib/neutron/dhcp/bc66c327-265b-4491-95ab-2d2e14d801d2/host |grep i1
fa:16:3e:09:b7:37,i1.ml2dns.example.,192.168.21.18# Neutron will inform Designate to create the forward and reverse records for the floating IP in the external DNS
dig @$DESIGNATE_BIND_IP i1.extdns.example
dig @$DESIGNATE_BIND_IP -x $fip$ dig @$DESIGNATE_BIND_IP i1.extdns.example
...
i1.extdns.example.  3600    IN  A   10.5.150.2
extdns.example.     3600    IN  NS  ns1.extdns.example.
ns1.extdns.example. 3600    IN  A   10.5.0.16
...# We can also see the external DNS record sets in Designate
openstack recordset list extdns.example.$ openstack recordset list extdns.example.
+--------------------------------------+---------------------+------+--------------------------------------------------------------------+--------+--------+
| id                                   | name                | type | records                                                            | status | action |
+--------------------------------------+---------------------+------+--------------------------------------------------------------------+--------+--------+
| 92c8f5d6-eeca-4b0d-968b-deed4697b787 | extdns.example.     | NS   | ns1.extdns.example.                                                | ACTIVE | NONE   |
| a0167123-714d-4c34-8542-ad8b5019c318 | extdns.example.     | SOA  | ns1.extdns.example. extdns.example. 1548816241 3505 600 86400 3600 | ACTIVE | NONE   |
| 99f97832-edea-4727-be68-73536d9145d3 | ns1.extdns.example. | A    | 10.5.0.16                                                          | ACTIVE | NONE   |
| 728808e5-e6dd-4b9d-b7fa-ad260a855ac4 | i1.extdns.example.  | A    | 10.5.150.2                                                         | ACTIVE | NONE   |
+--------------------------------------+---------------------+------+--------------------------------------------------------------------+--------+--------+

20220413更新

https://bugs.launchpad.net/charm-ovn-central/+bug/1857026
https://bugs.launchpad.net/neutron/+bug/1947127

20220420更新 - 让ovn同时支持dns与dns_domain_ports扩展

LP bug: https://bugs.launchpad.net/neutron/+bug/1947127
参考：https://docs.openstack.org/neutron/latest/admin/config-dns-int-ext-serv.html#config-dns-int-ext-serv

上面我们使用的是extension_drivers=dns这个扩展，其实还有很多扩展, 对应多个场景．
场景一，Floating IPs are published with associated port DNS attributes
FIP使用dns_domain(from network)和dns_name(from port), 就是neutron早期如我们上面文章的例子．

场景二，Floating IPs are published in the external DNS service, dns_domain和dns_name不从network和port中来，而是由用户自己指定：　
openstack floating ip create --dns-domain example.org. --dns-name my-floatingip 41fa3995-9e4a-4cd9-bb51-3e5424f2ff2a

场景三中的Use case 3c - dns_domain extention (https://docs.openstack.org/neutron/xena/admin/config-dns-int-ext-serv.html)
Use case 3c中有下列限制, 即segment id必须是在vni_ranges之外的:

For use cases 3b and 3c, the externally accessible network must meet the following requirements:
The network may not have attribute router:external set to True.
The network type can be FLAT, VLAN, GRE, VXLAN or GENEVE.
For network types VLAN, GRE, VXLAN or GENEVE, the segmentation ID must be outside the ranges assigned to project networks.

针对Use case 3c的实验1显示当segment id在vni_ranges的范围之内时, 不会为fixed port来创建external dns

$juju config neutron-api vni-ranges
1001:2000
$juju config neutron-api-plugin-ovn geneve-vni-ranges
1001:2000
$ juju ssh neutron-api/0 -- sudo grep -r 'extension_drivers' /etc/neutron/plugins/ml2/ml2_conf.ini
extension_drivers=port_security,dns_domain_portsopenstack router create testrouter
# there is no '--provider-physical-network physnet1' because it's tunnel
openstack network create net1 --provider-network-type geneve --provider-segment 1012
openstack subnet create --subnet-range 192.168.4.0/24 --network net1 --allocation-pool start=192.168.4.87,end=192.168.4.100 --gateway 192.168.4.1 subnet1
openstack router add subnet testrouter subnet1openstack subnet set --no-dns-nameservers subnet1
openstack network set --dns-domain extdns.example. net1
DESIGNATE_BIND_IP=$(juju status designate-bind --format short | awk '/designate-bind/ {print $3}')
openstack zone create --email extdns@example extdns.example.
openstack recordset create --record $DESIGNATE_BIND_IP --type A extdns.example. ns1
openstack recordset list extdns.example.
juju config neutron-api-plugin-ovn dns-servers=$DESIGNATE_BIND_IP
juju config designate nameservers=ns1.extdns.example.openstack port create --network net1 --dns-name i1 i1
$ openstack port show i1 |grep dns_assignment
| dns_assignment          | fqdn='i1.openstack.example.', hostname='i1', ip_address='192.168.4.88'      |
openstack recordset list extdns.example. --name i1.extdns.example.
dig @$DESIGNATE_BIND_IP i1.extdns.example.

针对Use case 3c的实验2显示当segment id不在vni_ranges的范围之内时, 就会为fixed port来创建external dns

openstack network create net2 --provider-network-type geneve --provider-segment 55
openstack subnet create --subnet-range 192.168.5.0/24 --network net2 --allocation-pool start=192.168.5.87,end=192.168.5.100 --gateway 192.168.5.1 subnet2
openstack router add subnet testrouter subnet2
openstack subnet set --no-dns-nameservers subnet2
openstack network set --dns-domain extdns.example. net2
openstack port create --network net2 --dns-name i2 i2
$ openstack port show i2 |grep dns_assignment
| dns_assignment          | fqdn='i2.extdns.example.', hostname='i2', ip_address='192.168.5.96'         |
$ openstack recordset list extdns.example. --name i2.extdns.example.
+--------------------------------------+--------------------+------+--------------+--------+--------+
| id                                   | name               | type | records      | status | action |
+--------------------------------------+--------------------+------+--------------+--------+--------+
| 7c62badf-635a-433d-8738-0af377298aa8 | i2.extdns.example. | A    | 192.168.5.96 | ACTIVE | NONE   |
+--------------------------------------+--------------------+------+--------------+--------+--------+
$ dig @$DESIGNATE_BIND_IP i2.extdns.example. |grep 192
i2.extdns.example.      3600    IN      A       192.168.5.96
$ juju ssh ovn-central/1 -- sudo ovn-nbctl list dhcp_options |grep '192.168.5.1'
options             : {classless_static_route="{169.254.169.254/32,192.168.5.87, 0.0.0.0/0,192.168.5.1}", dns_server="{10.5.3.19}", domain_name="\"openstack.example.\"", lease_time="43200", mtu="1492", router="192.168.5.1", server_id="192.168.5.1", server_mac="fa:16:3e:de:4c:b8"}

场景三中的Use case 3b - dns_domain_ports extentions.
正常情况dns_domain是来自network, 但extension_drivers=dns_domain_ports可以让port也能指定dns_domain, If the port is created in an externally accessible network, DNS records will be published for this port

openstack port create --network private --dns-name i1 --dns-domain port-domain.org. port_i1
openstack recordset list port-domain.org.

场景三中的Use case 3a - https://docs.openstack.org/neutron/latest/admin/config-dns-int-ext-serv.html#use-case-3a-the-subnet-dns-publish-fixed-ip-extension
让Port的fixed-IP也可以像FIP一样对外发布DNS(发布到external designate中）, 这需要将subnet_dns_publish_fixed_ip extention中的dns_publish_fixed_ips设置为true．如双栈场景中的IPv6作为fixed IP时添加dns可全球dns路由．

# grep -r 'extension_drivers' /etc/neutron/plugins/ml2/ml2_conf.ini
extension_drivers=port_security,subnet_dns_publish_fixed_ip
openstack network create dualstack
openstack subnet create --network dualstack dualstackv4 --subnet-range 192.0.2.0/24
openstack subnet create --network dualstack dualstackv6 --ip-version 6 --subnet-range 2001:db8:42:42::/64 --dns-publish-fixed-ip
openstack port create i3 --dns-domain extdns.example. --dns-name i3 --network dualstack
$ openstack recordset list extdns.example. --name i3.extdns.example.
+--------------------------------------+--------------------+------+--------------------+--------+--------+
| id                                   | name               | type | records            | status | action |
+--------------------------------------+--------------------+------+--------------------+--------+--------+
| 36f6bd1b-f935-4140-a781-a513e33c1935 | i3.extdns.example. | AAAA | 2001:db8:42:42::76 | ACTIVE | NONE   |
+--------------------------------------+--------------------+------+--------------------+--------+--------+
$ openstack network show dualstack |grep segmentation_id
| provider:segmentation_id  | 1585

注意一点, 像下面将dns_domain_ports与subnet_dns_publish_fixed_ip同时用时会导致’openstack port create i3 …’ hang在那儿, 所以似乎这两个配置不能同时使用.

# grep -r 'extension_drivers' /etc/neutron/plugins/ml2/ml2_conf.ini
extension_drivers=port_security,dns_domain_ports,subnet_dns_publish_fixed_ip

ovn dns

./generate-bundle.sh --name ovn --series focal --release xena --num-compute 1 --ovn --designate --use-stable-charms --run
./tools/vault-unseal-and-authorise.sh
source novarc
./configure
juju config neutron-api enable-ml2-dns=true
juju config neutron-api dns-domain=openstack.example.
juju config neutron-api reverse-dns-lookup=True
juju config neutron-api-plugin-ovn dns-servers="10.5.0.15"
juju config designate nameservers=ns1.ddi1.quqi.com.# dns-servers应该设置为designate_bind_IP, 同时要取消subnet中关于dns-nameservers的设置, 这样虚机内才会使用designate_bind_ip作为dns
DESIGNATE_BIND_IP=$(juju status designate-bind --format short | awk '/designate-bind/ {print $3}')
juju config neutron-api-plugin-ovn dns-servers=$DESIGNATE_BIND_IP
openstack subnet set --no-dns-nameservers private_subnet
$ juju ssh ovn-central/1 -- sudo ovn-nbctl list dhcp_options |grep options
options             : {classless_static_route="{169.254.169.254/32,192.168.21.2, 0.0.0.0/0,192.168.21.1}", dns_server="{10.5.3.19}", domain_name="\"openstack.example.\"", lease_time="43200", mtu="1492", router="192.168.21.1", server_id="192.168.21.1", server_mac="fa:16:3e:8f:d3:d4"}
openstack subnet show private_subnet |grep -E 'dns|dhcp'
| dns_nameservers      |                                      |
| dns_publish_fixed_ip | None                                 |
| enable_dhcp          | True                                 |# 在network中设置dns-domain使用prd.cloud.quqi.com.，这样也需要为它在designate中配置zone,
openstack network set --dns-domain prd.cloud.quqi.com. private
$ openstack network show private |grep -i -E 'dns|external'
| dns_domain                | prd.cloud.quqi.com.               |
| router:external           | Internal                             |openstack zone create --email extdns@example prd.cloud.quqi.com.
openstack recordset create --record $DESIGNATE_BIND_IP --type A prd.cloud.quqi.com. ns1# 这样以private_subnet来创建虚机时，因为network中倒置了dns-domain就会自动地将fixed_ip也配置使用external dns(bionic-080146.prd.cloud.quqi.com)
openstack server create --wait --image bionic --flavor m1.small --key-name testkey --nic net-id=25651e05-ecec-44a8-9ba8-534f6061d5e9 --min 1 --max 1 bionic-080146ubuntu@bionic-080146:~$ grep -r 'search' /etc/resolv.conf
search openstack.example
ubuntu@bionic-080146:~$ sudo systemd-resolve --status |grep -E 'DNS Servers|DNS Domain'DNS Servers: 10.5.3.19DNS Domain: openstack.example
ubuntu@bionic-080146:~$ nslookup bionic-080146.openstack.example 10.5.3.19
;; connection timed out; no servers could be reached
ubuntu@bionic-080146:~$ nslookup bionic-080146.prd.cloud.quqi.com 10.5.3.19
Name:   bionic-080146.prd.cloud.quqi.com
Address: 192.168.21.187$ openstack recordset list prd.cloud.quqi.com. --name bionic-080146.prd.cloud.quqi.com.
+--------------------------------------+--------------------------------------+------+----------------+--------+--------+
| id                                   | name                                 | type | records        | status | action |
+--------------------------------------+--------------------------------------+------+----------------+--------+--------+
| 9ed3bf26-343f-44c6-bee1-c1e2ebff084e | bionic-080146.prd.cloud.quqi.com. | A    | 192.168.21.187 | ACTIVE | NONE   |
+--------------------------------------+--------------------------------------+------+----------------+--------+--------+$ dig @$DESIGNATE_BIND_IP bionic-080146.prd.cloud.quqi.com |grep 192
bionic-080146.prd.cloud.quqi.com. 3600 IN A  192.168.21.187

20221025 - SECONDARY zone

designate v2 api(https://docs.openstack.org/designate/latest/user/secondary-zones.html)允许将designate作为一个dns slave rather than a master for a zone, This is accomplished by completing a zone transfer (AXFR) from a DNS server managed outside of Designate.

1, set up a designate as dns slave./generate-bundle.sh --name dns -s focal -r xena --designate --run
./configure
juju config designate nameservers=ns1.extdns.example.
DOMAIN_NAME=extdns.example.
openstack zone create --email extdns@example $DOMAIN_NAME
DESIGNATE_BIND_IP=$(juju status designate-bind --format short | awk '/designate-bind/ {print $3}')
juju config neutron-gateway dns-servers=$DESIGNATE_BIND_IP
openstack recordset create --record $DESIGNATE_BIND_IP --type A $DOMAIN_NAME ns1
dig @$DESIGNATE_BIND_IP -type NS
neutron net-update private --dns_domain $DOMAIN_NAME
nova keypair-add --pub-key ~/.ssh/id_rsa.pub mykey
openstack port create --network $(neutron net-show private -f value -c id) --dns-name i1 port_i1
openstack server create --wait --image cirros2 --flavor m1.tiny --key-name mykey --port $(openstack port show port_i1 -f value -c id) i1
public_network=$(openstack network show ext_net -f value -c id)
fip=$(openstack floating ip create $public_network -f value -c floating_ip_address)
openstack floating ip set $fip --fixed-ip-address 192.168.21.107 --port $(openstack port list --fixed-ip ip-address=192.168.21.107 -c id -f value)
#openstack subnet unset --dns-nameserver 10.230.64.2 private_subnet
nova boot --hard i12, create a dns masteropenstack server create --image auto-sync/ubuntu-focal-daily-amd64-server-20221021-disk1.img --flavor m1.large bind --network zhhuabj_admin_net --key mykey
BIND_IP=$(openstack server show bind | grep addresses | sed -e 's/.*=//' -e 's/ .*//')
ssh $BIND_IP
sudo apt install bind9 -y
cat <<EOF | sudo tee /etc/bind/named.conf.options
options {
listen-on port 53 { 0.0.0.0/0; };
allow-query { any; };
allow-transfer { any; };
directory "/var/cache/bind";
dnssec-validation auto;
};
EOF
cat <<EOF | sudo tee /etc/bind/named.conf.local
zone "example.tld" IN {
type master;
file "/etc/bind/example.tld.db";
};
EOF
cat <<"EOF" | sudo tee /etc/bind/example.tld.db
$TTL 5m
@ IN SOA ns.example.tld. email.example.tld. 90 4h 15m 8h 4m
@ IN NS ns.example.tld.
ns IN A 1.1.1.1
test IN A 1.2.3.4
EOF
sudo systemctl restart named$ dig +noall +authority @$BIND_IP -p 53 example.tld
example.tld.            240     IN      SOA     ns.example.tld. email.example.tld. 90 14400 900 28800 240
$ dig +noall +answer @$BIND_IP -p 53 example.tld axfr
example.tld.            300     IN      SOA     ns.example.tld. email.example.tld. 90 14400 900 28800 240
example.tld.            300     IN      NS      ns.example.tld.
ns.example.tld.         300     IN      A       1.1.1.1
test.example.tld.       300     IN      A       1.2.3.4
example.tld.            300     IN      SOA     ns.example.tld. email.example.tld. 90 14400 900 28800 240
#test AXFR with dnspython:
sudo apt install python3-dnspython -y
ubuntu@bind:~$ python3 -q
>>> import dns.query
>>> import dns.zone
>>> axfr = dns.zone.from_xfr(dns.query.xfr(where='127.0.0.1', port=12753, zone='example.tld', rdtype=dns.rdatatype.AXFR))
>>> axfr = dns.zone.from_xfr(dns.query.xfr(where='127.0.0.1', port=53, zone='example.tld', rdtype=dns.rdatatype.AXFR))
>>> for node in axfr.nodes.keys():
...     print(axfr.nodes[node].to_text(node))
...
@ 300 IN SOA ns email 90 14400 900 28800 240
@ 300 IN NS ns
ns 300 IN A 1.1.1.1
test 300 IN A 1.2.3.43, create SECONDARY zonedig @$BIND_IP example.tld axfr
openstack zone create example.tld. --type SECONDARY --master $BIND_IP
openstack zone list | grep SECONDARY | awk '{ print $2 }' | xargs -n 1 openstack zone show
openstack zone list --all-projects4, some outputs# grep 'example.tld' /var/log/designate/designate-mdns.log |tail -n2
2022-10-25 06:23:09.377 27318 INFO designate.dnsutils [req-fbef1b90-3773-45cb-aaf3-6a464f84b65b 7877a98221744a0d914794e928f5d2fa 82e14f00cc03478fb165c496678242d2 - - -] Doing AXFR for example.tld. from {'zone_id': '44fddde9-3499-4495-920d-81d55a130657', 'host': '10.5.3.184', 'port': 53, 'id': '98d4b9d8-3849-4dd4-8610-b3d85555d5da', 'created_at': datetime.datetime(2022, 10, 25, 6, 19, 53), 'updated_at': datetime.datetime(2022, 10, 25, 6, 23, 9), 'version': 495} 10.5.3.184
2022-10-25 06:23:09.391 27318 DEBUG designate.dnsutils [req-fbef1b90-3773-45cb-aaf3-6a464f84b65b 7877a98221744a0d914794e928f5d2fa 82e14f00cc03478fb165c496678242d2 - - -] AXFR Successful for example.tld. do_axfr /usr/lib/python3/dist-packages/designate/dnsutils.py:364# grep 'ZoneMasterNotFound' /var/log/designate/designate-mdns.log |tail -n2
2022-10-25 06:24:16.056 27318 ERROR oslo_messaging.rpc.server designate.exceptions_Remote.ZoneMasterNotFound_Remote: Could not find ZoneMaster
2022-10-25 06:24:16.056 27318 ERROR oslo_messaging.rpc.server designate.exceptions.ZoneMasterNotFound: Could not find ZoneMaster

Reference

[1] https://github.com/openstack/neutron/blob/stable/pike/neutron/plugins/ml2/extensions/dns_integration.py#L285
[2] https://docs.openstack.org/mitaka/networking-guide/config-dns-int.html
[3] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/8/html-single/dns-as-a-service_guide/index
[4] https://openstackdevops.wordpress.com/2018/01/27/designate-and-neutron-dns-integration/