juju创建lxd容器时如何使用本地镜像(by quqi99)

作者：张华发表于：2023-03-01
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明

问题

没有外网，所以配置了一个local custom镜像库，也使用了container-image-metadata-url进行配置，但是用juju创建lxd容器时还是说找不着image.

实验环境

1, 使用 juju创建一个focal的machine 0, 然后再machine 0上部署一个xenial的lxd容器。

juju add-model test
juju add-machine --series focal
juju model-config logging-config="<root>=DEBUG"
juju remove-application ceph-radosgw && juju deploy ceph-radosgw --series=xenial --to="lxd:0"

2, 在juju controller(juju ssh -m controller 0)与machine 0上运行下列iptables来模拟和cloud-images.ubuntu.com断网。这里我发现：

machine 0的日志（/var/log/juju/machine-0.log）显示它好像是从juju controller处下载镜像的

2023-03-01 07:58:21 INFO juju.cloudconfig userdatacfg_unix.go:613 Fetching agent: curl -sSf --connect-timeout 20 --noproxy "*" --insecure -o $bin/tools.tar.gz <[https://10.5.0.31:17070/model/deb85179-10a6-4877-88f7-012ef768d726/tools/2.9.38-ubuntu-amd64 https://252.0.31.1:17070/model/deb85179-10a6-4877-88f7-012ef768d726/tools/2.9.38-ubuntu-amd64]>
2023-03-01 07:59:03 INFO juju.container.lxd container.go:256 starting new container "juju-68d726-0-lxd-2" (image "ubuntu-16.04-server-cloudimg-amd64-lxd.tar.xz")
2023-03-01 07:59:03 DEBUG juju.container.lxd container.go:257 new container has profiles [default]
2023-03-01 07:59:42 DEBUG juju.container.lxd container.go:286 created container "juju-68d726-0-lxd-2", waiting for start...

但如果不在machine 0上运行下载iptables，测试表明machine0也能直接扰开juju controller从cloud-images.ubuntu.com处下镜像.
似乎二者均相关，那就二者将运行下列iptables吧

dig cloud-images.ubuntu.com  #185.125.190.37 and 185.125.190.40
juju ssh -m controller 0 -- sudo iptables -A OUTPUT -d 185.125.190.37 -j DROP
juju ssh -m controller 0 -- sudo iptables -A OUTPUT -d 185.125.190.40 -j DROP
cat << EOF |tee test.yaml
cloudinit-userdata: |postruncmd:- bash -c 'echo 10.5.0.126 quqi.com >> /etc/hosts'- bash -c 'iptables -A OUTPUT -d 185.125.190.37 -j DROP'- bash -c 'iptables -A OUTPUT -d 185.125.190.40 -j DROP'
EOF
juju model-config ./test.yaml

3, bastion上运行sstream-mirror将cloud-images.ubuntu.com中的xenial amd64镜像mirror了下来。

sudo apt -y install simplestreams -y
workdir=/home/ubuntu/simplestreams2
sudo sstream-mirror --keyring=/usr/share/keyrings/ubuntu-cloudimage-keyring.gpg --progress --max=1 --path=streams/v1/index.json https://cloud-images.ubuntu.com/releases/ $workdir 'arch=amd64' 'release~(xenial)' 'ftype~(lxd.tar.xz|squashfs|root.tar.xz|root.tar.gz|disk1.img|.json|.sjson)'

然后用nginx为它配置了https:

#https://goharbor.io/docs/2.6.0/install-config/configure-https/
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -sha512 -days 3650 -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=quqi.com" -key ca.key -out ca.crt
openssl genrsa -out quqi.com.key 4096
openssl req -sha512 -new -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=quqi.com" -key quqi.com.key -out quqi.com.csr
#complies with the Subject Alternative Name (SAN) and x509 v3 extension requirements to avoid 'x509: certificate relies on legacy Common Name field, use SANs instead'
cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names[alt_names]
DNS.1=quqi.com
DNS.2=quqi
DNS.3=hostname
EOF
openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in quqi.com.csr -out quqi.com.crt
#for docker, the Docker daemon interprets .crt files as CA certificates and .cert files as client certificates.
openssl x509 -inform PEM -in quqi.com.crt -out quqi.com.cert
curl --resolve quqi.com:443:10.5.0.126 --cacert ~/ca/ca.crt https://quqi.com:443/streams/v1/index.json
sudo cp ~/ca/ca.crt /usr/local/share/ca-certificates/ca.crt
sudo chmod 644 /usr/local/share/ca-certificates/ca.crt
sudo update-ca-certificates --fresh
curl --resolve quqi.com:443:10.5.0.126 https://quqi.com:443/streams/v1/index.json$ cat /etc/nginx/sites-available/default
server {listen 443 ssl http2;listen [::]:443 ssl http2;server_name quqi.com;ssl_certificate /home/ubuntu/ca/quqi.com.crt;ssl_certificate_key /home/ubuntu/ca/quqi.com.key;#ssl_protocols TLSv1.2;ssl_prefer_server_ciphers on; location / {root /home/ubuntu/simplestreams2;index index.html;}
}
# 注意：由于上面使用了一个新目录/home/ubuntu/simplestreams2作为root，那需要将/etc/nginx/nginx.conf中添加'user root;'来避免权限问题
#curl --resolve quqi.com:443:10.5.0.126 --cacert ~/ca/ca.crt https://quqi.com:443/images/streams/v1/index.json
curl --resolve quqi.com:443:10.5.0.126 --cacert ~/ca/ca.crt https://quqi.com:443/streams/v1/index.json

4, 配置juju中的container-image-metadata-url使用上面的https based local image mirror

juju model-config container-image-metadata-url=https://quqi.com:443
juju model-config image-metadata-url=https://quqi.com:443

5, juju controller由于访问local image mirror, 所以配置hosts与添加ca key

echo '10.5.0.126 quqi.com' >> /etc/hostscurl --resolve quqi.com:443:10.5.0.126 --cacert ~/ca/ca.crt https://quqi.com:443/streams/v1/index.json
sudo cp ~/ca/ca.crt /usr/local/share/ca-certificates/ca.crt
sudo chmod 644 /usr/local/share/ca-certificates/ca.crt
sudo update-ca-certificates --fresh
curl --resolve quqi.com:443:10.5.0.126 https://quqi.com:443/streams/v1/index.json

6, 记得重新测试之前将machine 0上的image cache删除

juju ssh 0 -- sudo lxc image delete juju/xenial/amd64
juju remove-application ceph-radosgw

7, 重新测试

juju deploy ceph-radosgw --series=xenial --to="lxd:0"
sudo tail -f /var/log/juju/machine-0.log

能在machine 0的/var/log/juju/machine-0.log中观察下列日志:

2023-03-01 08:26:45 WARNING juju.worker.lxdprovisioner provisioner_task.go:1371 machine 0/lxd/3 failed to start: acquiring LXD image: no matching image found
2023-03-01 08:26:45 WARNING juju.worker.lxdprovisioner provisioner_task.go:1410 failed to start machine 0/lxd/3 (acquiring LXD image: no matching image found), retrying in 10s (10 more attempts)

在juju controller上有时能搜到quqi, 有时候又不能，奇怪.

2023-02-23 07:33:52 WARNING juju.apiserver.provisioner provisioninginfo.go:801 encountered "https://quqi.com:443/images/streams/v1/streams/v1/index.json": Get "https://quqi.com:443/images/streams/v1/streams/v1/index.json": dial tcp 49.234.171.74:443: i/o timeout while getting published images metadata from image-metadata-url
2023-03-01 08:52:56 WARNING juju.environs.simplestreams datasource.go:184 Got error requesting "https://quqi.com:443/streams/v1/index.json": Get "https://quqi.com:443/streams/v1/index.json": x509: certificate relies on legacy Common Name field, use SANs instead

juju controller上仍然能看到cloud-images.ubuntu.com

2023-03-01 08:34:54 WARNING juju.apiserver.provisioner provisioninginfo.go:801 encountered "http://cloud-images.ubuntu.com/releases/streams/v1/index.sjson": Get "http://cloud-images.ubuntu.com/releases/streams/v1/index.sjson": dial tcp 185.125.190.37:80: i/o timeout while getting published images metadata from default ubuntu cloud images

使用glance中的image来提供simplestreams

上面是使用来提供simplestreams, 我们现在换用glance中的image来提供simplestreams继续测试（不确定是否这种只适用于创建juju controller，还是说也可以用于VM/LXD创建，试一下)

mkdir -p ~/simplestreams/images
IMAGE_ID=26751c0e-4282-415e-b8dc-a7a21d2f781d
SERIES=xenial
juju metadata generate-image -d ~/simplestreams -i $IMAGE_ID -s $SERIES -r RegionOne -u $OS_AUTH_URL

然后修改/etc/nginx/sites-available/default将上面测试用的/home/ubuntu/simplestreams2改成/home/ubuntu/simplestreams，重启nginx之后, 设置container-image-metadata-url (注意：此时后面链接多出了/images)

juju model-config container-image-metadata-url=https://quqi.com:443/images
juju model-config image-metadata-url=https://quqi.com:443/images#注意下面的并不是由上两句形成的，而是由人工运行lxc命令(lxc remote add xxx)形成的，但即使有它也不 work
root@juju-4e4d8f-test-0:~# cat ~/snap/lxd/common/config/config.yml
default-remote: local
remotes:images:addr: https://images.linuxcontainers.orgprotocol: simplestreamspublic: truelocal:addr: unix://public: falsetest:addr: https://quqi.com:443protocol: simplestreamspublic: true
aliases: {}

为保证测试环境干净，我也在controller与machine0上运行了下列命令：

systemctl restart jujud-machine-0.service

然后重复测试后，问题依旧， controller上看到下列日志：

2023-03-01 10:29:32 WARNING juju.environs.simplestreams datasource.go:184 Got error requesting "https://streams.canonical.com/juju/tools/streams/v1/index.sjson": Get "https://streams.canonical.com/juju/tools/streams/v1/index.sjson": dial tcp 185.125.190.37:443: i/o timeout
2023-03-01 10:29:36 INFO juju.state addmachine.go:505 new machine "0/lxd/11" has preferred addresses: private "", public ""
2023-03-01 10:29:37 WARNING juju.apiserver.instancemutater lxdprofilewatcher.go:206 unit ceph-radosgw/11 has no machine id, start watching when machine id assigned.
2023-03-01 10:29:41 WARNING juju.apiserver.provisioner provisioninginfo.go:801 encountered index file has no data for cloud {stsstack http://10.230.19.53:5000/v3} not found while getting published images metadata from image-metadata-url
2023-03-01 10:30:11 WARNING juju.environs.simplestreams datasource.go:184 Got error requesting "http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson": Get "http://cloud-images.ubuntu.com/releases/streams/v1/index2.sjson": dial tcp 185.125.190.40:80: i/o timeout
2023-03-01 10:30:41 WARNING juju.environs.simplestreams datasource.go:184 Got error requesting "http://cloud-images.ubuntu.com/releases/streams/v1/index.sjson": Get "http://cloud-images.ubuntu.com/releases/streams/v1/index.sjson": dial tcp 185.125.190.37:80: i/o timeout
2023-03-01 10:30:41 WARNING juju.apiserver.provisioner provisioninginfo.go:801 encountered "http://cloud-images.ubuntu.com/releases/streams/v1/index.sjson": Get "http://cloud-images.ubuntu.com/releases/streams/v1/index.sjson": dial tcp 185.125.190.37:80: i/o timeout while getting published images metadata from default ubuntu cloud images

看样子和simplestreams类型无关。

测试cloudinit-userdata

即然与simplestreams类型无关，那我们将nginx再恢复之前的/home/ubuntu/simplestreams2

juju model-config container-image-metadata-url=https://quqi.com:443/
juju model-config image-metadata-url=https://quqi.com:443/

然后测试cloudinit-userdata, 这个是没问题的，可以作workaround

cat << EOF |tee cloudinit-userdata.yaml
cloudinit-userdata: |postruncmd:- echo '10.5.0.126 quqi.com' >> /etc/hosts- if hostname |grep -qv lxd; then wget --tries=15 --retry-connrefused --timeout=15 --random-wait=on -O /home/ubuntu/ubuntu-16.04-server-cloudimg-amd64-lxd.tar.xz https://quqi.com:443/server/releases/xenial/release-20211001/ubuntu-16.04-server-cloudimg-amd64-lxd.tar.xz --no-check-certificate; wget --tries=15 --retry-connrefused --timeout=15 --random-wait=on -O /home/ubuntu/ubuntu-16.04-server-cloudimg-amd64.squashfs https://quqi.com:443/server/releases/xenial/release-20211001/ubuntu-16.04-server-cloudimg-amd64.squashfs --no-check-certificate; fi- sleep 30- if hostname |grep -qv lxd; then lxc image import /home/ubuntu/ubuntu-16.04-server-cloudimg-amd64-lxd.tar.xz /home/ubuntu/ubuntu-16.04-server-cloudimg-amd64.squashfs --alias juju/xenial/amd64; fi
EOF
juju model-config ./cloudinit-userdata.yaml
juju model-config cloudinit-userdata --format yaml
#juju model-config --reset cloudinit-userdata

注意：之前一直不work的原因是因为在postruncmd:后加了 | 的原因，找到答案的过程见下列的"调试cloud-init"一节。

调试cloud-init

最后发现用下面的是不work的：

cat << EOF |tee test.yaml
cloudinit-userdata: |postruncmd: |- echo '10.5.0.126 quqi.com' >> /etc/hosts- echo 'test' > /home/ubuntu/cloud-init.txt
EOF

需要改成下面的：

cat << EOF |tee test.yaml
cloudinit-userdata: |postruncmd:- bash -c 'echo 10.5.0.126 quqi.com >> /etc/hosts'- bash -c 'echo test > /home/ubuntu/cloud-init.txt'
EOF

下面的也不会work

cat << EOF |tee test.yaml
cloudinit-userdata: |postruncmd: |bash -c 'echo 10.5.0.126 quqi.com >> /etc/hosts'bash -c 'echo test > /home/ubuntu/cloud-init.txt'
EOF

下面的更不会work, 会直接报：ERROR json: unsupported type: map[interface {}]interface {}’

cat << EOF |tee test.yaml
cloudinit-userdata: |postruncmd:bash -c 'echo 10.5.0.126 quqi.com >> /etc/hosts'bash -c 'echo test > /home/ubuntu/cloud-init.txt'
EOF

其他调试方法如下：

juju add-model test
juju model-config ./test.yaml
juju model-config cloudinit-userdata --format yaml
juju model-config ssl-hostname-verification=false
juju add-machine --series focal1, check cloud-init log:    cloud-init collect-logs & tar -xf cloud-init.tar.gz
2, check cloud-init config: /etc/cloud/cloud.cfg
3, cloud-init is enabled: systemctl list-unit-files | grep cloud
4, /var/lib/cloud/instances/af2d721e-e38e-4937-81ad-7cc72a49c184/cloud-config.txt

lp bug 1797168

试图排除https://bugs.launchpad.net/juju/+bug/1797168

juju add-model test2
juju model-config container-image-metadata-url=https://quqi.com:443/
juju model-config image-metadata-url=https://quqi.com:443/
juju model-config logging-config="<root>=DEBUG"
juju model-config ssl-hostname-verification=false
juju add-machine --series xenial#一定要拷ca.crt到machine 0上（而不是controller 0)
juju scp -m m ~/ca/ca.crt 0:~/
juju ssh -m m 0 -- sudo cp /home/ubuntu/ca.crt /usr/local/share/ca-certificates/ca.crt
juju ssh -m m 0 -- sudo update-ca-certificates --freshjuju add-machine --series xenial lxd:0
#juju remove-application ceph-radosgw && juju deploy ceph-radosgw --series=xenial --to="lxd:0"

NOTE: 一直不work的原因是将ca.crt拷贝到了controller 0，而是应该将它拷到machine 0

lxc remote端的测试

lxc端用cloud-images.ubuntu.com作default ，这个default不能replace,

# lxc remote list |grep releases
| ubuntu          | https://cloud-images.ubuntu.com/releases | simplestreams | none        | YES    | YES    | NO     |root@juju-4e4d8f-test-7:~# lxc remote set-url ubuntu https://quqi.com:443
Error: Remote ubuntu is static and cannot be modified

它只能添加，所以我添加了自己的也干脆将它设置成public类型的

lxc remote add test https://quqi.com:443 --protocol=simplestreams
lxc remote remove test & lxc remote add test https://quqi.com:443 --protocol=simplestreams --public
sudo snap set lxd daemon.debug=true
sudo systemctl reload snap.lxd.daemon

也要machine里设置了LXD_INSECURE_TLS=true(remote error: tls: protocol version not supported), 总之确保了使用test mirror (lxc launch test:16.04 i1)能正常运行。

vim /etc/systemd/system/snap.lxd.daemon.service
Environment=LXD_INSECURE_TLS=true
#或者去掉nginx中的ssl_protocols TLSv1.2也行

但是测试还是失败，machine端看到了日志：

2023-03-02 03:26:30 DEBUG juju.container.lxd manager.go:283 checking default image metadata sources
2023-03-02 03:27:51 WARNING juju.worker.lxdprovisioner provisioner_task.go:1371 machine 7/lxd/3 failed to start: acquiring LXD image: no matching image found
2023-03-02 03:27:51 WARNING juju.worker.lxdprovisioner provisioner_task.go:1410 failed to start machine 7/lxd/3 (acquiring LXD image: no matching image found), retrying in 10s (10 more attempts)

20230607更新

今天终于知道 lp bug 2008993中的测试时而可以时而不行的原因了： https://bugs.launchpad.net/juju/+bug/2008993
原来是juju model中配置了juju-https-proxy导致container-image-metadata-url不work，这样:

若start nginx (image mirror), machine 0’s jujud will call nginx (node1.lan:443), then ca

138  66.866261    10.0.0.13 → 192.168.99.179 HTTP 159 CONNECT node1.lan:443 HTTP/1.1 140  66.871290 192.168.99.179 → 10.0.0.13    HTTP 111 HTTP/1.1 200 Connection established 160  66.890098    10.0.0.13 → 192.168.99.179 HTTP 187 CONNECT cloud-images.ubuntu.com:443 HTTP/1.1 162  66.896207 192.168.99.179 → 10.0.0.13    HTTP 111 HTTP/1.1 200 Connection established 164  66.896723    10.0.0.13 → 192.168.99.179 TLSv1 341 Client Hello

若stop nginx, jujud call nginx的时候返回了’503 Service Unavailable’, 然后CONNECT cloud-images.ubuntu.com:443时会走juju-https-proxy, 所以这是我们为什么在controller与machine0上都设置了iptables rules来block到cloud-images.ubuntu.com:443的流量之后jujud仍然可以下载到image的原因

    1   574  62.682108    10.0.0.13 → 192.168.99.179 HTTP 159 CONNECT node1.lan:443 HTTP/1.1                                                                                             2   576  62.688821 192.168.99.179 → 10.0.0.13    HTTP 3861 HTTP/1.1 503 Service Unavailable  (text/html)                                                                             3   592  62.691982    10.0.0.13 → 192.168.99.179 HTTP 187 CONNECT cloud-images.ubuntu.com:443 HTTP/1.1                                                                               4   594  62.697651 192.168.99.179 → 10.0.0.13    HTTP 111 HTTP/1.1 200 Connection established

上面的抓包采用下列方法，它能看到完整的http流：

sudo tcpdump -i any -w capture.pcap'
tshark -r capture2.pcap -Y http

注意，不能用下列的方法分析, 因为它是https流量，看到都指向cloud-images.ubuntu.com很正常，所以用上面的’tshark-rcapture2.pcap -Y http’来分析：

$ tshark -r capture2.pcap -Y "http.request.method == GET" -T fields -e http.host -e http.request.uri |grep 18.04 |grep -v deb
cloud-images.ubuntu.com http://cloud-images.ubuntu.com/releases/server/releases/bionic/release-20230530/ubuntu-18.04-server-cloudimg-amd64-lxd.tar.xz
cloud-images.ubuntu.com http://cloud-images.ubuntu.com/releases/bionic/release-20230530/ubuntu-18.04-server-cloudimg-amd64-lxd.tar.xz
cloud-images.ubuntu.com http://cloud-images.ubuntu.com/releases/server/releases/bionic/release-20230530/ubuntu-18.04-server-cloudimg-amd64.squashfs
cloud-images.ubuntu.com http://cloud-images.ubuntu.com/releases/bionic/release-20230530/ubuntu-18.04-server-cloudimg-amd64.squashfs

另外一个问题是我是用下列命令创建的image mirror, 它少了bionc，而我却在用bionic做实验。

workdir=/images/simplestreams/lxdkvm
sudo sstream-mirror --keyring=/usr/share/keyrings/ubuntu-cloudimage-keyring.gpg --progress --max=1 --path=streams/v1/index.json https://cloud-images.ubuntu.com/releases/ $workdir 'arch=amd64' 'release~(xenial|focal|jammy)' 'ftype~(lxd.tar.xz|squashfs|root.tar.xz|root.tar.gz|disk1.img|.json|.sjson)'

所以正确的方法还是得在一个multipass虚机中做实验，来隔离一切可能的网络影响，方法如下：

openstack server create --wait --image auto-sync/ubuntu-jammy-daily-amd64-server-20220423-disk1.img --flavor m1.2xlarge --key-name mykey --network=zhhuabj_admin_net test
ssh ubuntu@10.5.3.62sudo snap install multipass
multipass launch 22.04 --name test -c 1 --disk 20G --memory 8G
multipass exec test -- bashmkdir ~/ca && cd ~/ca
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -sha512 -days 3650 \-subj "/C=CN/L=Bordeaux/O=example/OU=Personal/CN=lp2008993.com" \-key ca.key \-out ca.crt
openssl genrsa -out lp2008993.com.key 4096
openssl req -sha512 -new \-subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=lp2008993.com" \-key lp2008993.com.key \-out lp2008993.com.csr
cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names[alt_names]
DNS.1=lp2008993.com
DNS.2=lp2008993
DNS.3=hostname
EOF
openssl x509 -req -sha512 -days 3650 \-extfile v3.ext \-CA ca.crt -CAkey ca.key -CAcreateserial \-in lp2008993.com.csr \-out lp2008993.com.crtsudo apt update
sudo apt -y install simplestreams -y
workdir=/home/ubuntu/simplestreams
sudo sstream-mirror --keyring=/usr/share/keyrings/ubuntu-cloudimage-keyring.gpg --progress --max=1 --path=streams/v1/index.json https://cloud-images.ubuntu.com/releases/ $workdir 'arch=amd64' 'release~(bionic)' 'ftype~(lxd.tar.xz|squashfs|root.tar.xz|root.tar.gz|disk1.img|.json|.sjson)'sudo apt install nginx -y
cat << EOF |sudo tee /etc/nginx/sites-available/default
server {listen 443 ssl http2;listen [::]:443 ssl http2;server_name lp2008993.com;ssl_certificate /home/ubuntu/ca/lp2008993.com.crt;ssl_certificate_key /home/ubuntu/ca/lp2008993.com.key;ssl_prefer_server_ciphers on;location / {root /home/ubuntu/simplestreams;index index.html;}
}
EOF
sudo sed -i 's/user www-data;/user root;/' /etc/nginx/nginx.conf
sudo systemctl restart nginx#confirm image mirror works
#juju scp -m m ~/ca/ca.crt 0:~/
#juju ssh -m m 0 -- sudo cp ~/ca.crt /usr/local/share/ca-certificates/ca.crt
#juju ssh -m m 0 -- sudo update-ca-certificates --fresh
echo '10.177.102.119 lp2008993.com' |sudo tee -a /etc/hosts
curl https://lp2008993.com:443/streams/v1/index.json -k |tail -n3sudo snap install juju --classic
juju bootstrap localhost c juju add-model m
juju model-config -m m logging-config="<root>=DEBUG"
juju model-config container-image-metadata-url=https://lp2008993.com:443
juju model-config image-metadata-url=https://lp2008993.com:443
juju model-config ssl-hostname-verification=false
cat << EOF |tee test.yaml
cloudinit-userdata: |postruncmd:- bash -c 'echo 10.177.102.119 lp2008993.com >> /etc/hosts'- bash -c 'iptables -A OUTPUT -d 185.125.190.37 -j DROP'- bash -c 'iptables -A OUTPUT -d 185.125.190.40 -j DROP'
EOF
juju model-config ./test.yamljuju add-machine --series bionic --constraints "mem=4G"
juju ssh -m m 0 -- sudo iptables-save |grep 185juju ssh -m controller 0 -- sudo iptables -A OUTPUT -d 185.125.190.37 -j DROP
juju ssh -m controller 0 -- sudo iptables -A OUTPUT -d 185.125.190.40 -j DROP
juju ssh -m controller 0 -- sudo iptables-save |grep 185#confirm
ubuntu@test:~$ juju model-config |grep -E 'image|init|ssl|juju-https-proxy|juju-http-proxy'
cloudinit-userdata                 model    <value set, see juju model-config cloudinit-userdata>
container-image-metadata-url       model    https://lp2008993.com:443
container-image-stream             default  released
image-metadata-url                 model    https://lp2008993.com:443
image-stream                       default  released
juju-http-proxy                    default  ""
juju-https-proxy                   default  ""
ssl-hostname-verification          model    false
ubuntu@test:~$ juju model-config cloudinit-userdata
postruncmd:- bash -c 'echo 10.193.21.137 lp2008993.com >> /etc/hosts'- bash -c 'iptables -A OUTPUT -d 185.125.190.37 -j DROP'- bash -c 'iptables -A OUTPUT -d 185.125.190.40 -j DROP'#debug hacks
juju ssh -m m 0
sudo tcpdump -i any -w capture.pcap
sudo tail -f /var/log/nginx/access.log
juju debug-log -m m --replay | grep "Found image"#trigger
date; juju add-machine --series bionic lxd:0

20230628 - 结论

今天把问题找到了，使用https://blog.csdn.net/quqi99/article/details/103175749 中的方法debug juju发现问题是machine0上哪怕设置了ssl-hostname-verification=false也必须安装证书，另外安装证书后必须重启machine0上的jujud服务