ingress-nginx更改默认端口/修改apiserver端口范围

问题：

直接更改ingress svc yml文件更改端口报错。

环境说明

1、 k8s部署在纯内网环境，想要外网可以访问，需要外层的服务器做nginx转发。由于公司网络限制，内网服务器和外层服务器互相通信需要使用固范围内的端口。
2、ingress默认使用的http 80；https 443；采用nodeport方式默认的端口暴露范围在 30000-32767。通过更改apiserver nodeport端口范围，达到更改ingress端口的作用。

修改apiserver端口范围

安装方式为kubeadm，会在master产生节点上会有一个文件/etc/kubernetes/manifests/kube-apiserver.yaml。添加 - --service-node-port-range=7000-8000 我设定的端口范围为7000-8000.

apiVersion: v1
kind: Pod
metadata:annotations:kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 172.30.210.23:5709creationTimestamp: nulllabels:component: kube-apiservertier: control-planename: kube-apiservernamespace: kube-system
spec:containers:- command:- kube-apiserver- --advertise-address=192.168.11.8- --allow-privileged=true- --authorization-mode=Node,RBAC- --client-ca-file=/etc/kubernetes/pki/ca.crt- --enable-admission-plugins=NodeRestriction- --enable-bootstrap-token-auth=true- --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt- --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt- --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key- --etcd-servers=https://127.0.0.1:2709- --insecure-port=0- --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt- --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname- --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt- --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key- --requestheader-allowed-names=front-proxy-client- --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt- --requestheader-extra-headers-prefix=X-Remote-Extra-- --requestheader-group-headers=X-Remote-Group- --requestheader-username-headers=X-Remote-User- --secure-port=6001- --service-account-issuer=https://kubernetes.default.svc.cluster.local- --service-account-key-file=/etc/kubernetes/pki/sa.pub- --service-account-signing-key-file=/etc/kubernetes/pki/sa.key- --service-cluster-ip-range=10.10.0.0/16- --tls-cert-file=/etc/kubernetes/pki/apiserver.crt- --tls-private-key-file=/etc/kubernetes/pki/apiserver.key- --service-node-port-range=7000-8000image: 192.168.11.13:6010/k8s/kube-apiserver:v1.21.0imagePullPolicy: IfNotPresentlivenessProbe:failureThreshold: 8httpGet:host: 192.168.11.8path: /livezport: 5709scheme: HTTPSinitialDelaySeconds: 10periodSeconds: 10timeoutSeconds: 15name: kube-apiserverreadinessProbe:failureThreshold: 3httpGet:host: 192.168.11.8path: /readyzport: 5709scheme: HTTPSperiodSeconds: 1timeoutSeconds: 15resources:requests:cpu: 250mstartupProbe:failureThreshold: 24httpGet:host: 192.168.11.8path: /livezport: 5709scheme: HTTPSinitialDelaySeconds: 10periodSeconds: 10timeoutSeconds: 15volumeMounts:- mountPath: /etc/ssl/certsname: ca-certsreadOnly: true- mountPath: /etc/pkiname: etc-pkireadOnly: true- mountPath: /etc/kubernetes/pkiname: k8s-certsreadOnly: truehostNetwork: truepriorityClassName: system-node-criticalvolumes:- hostPath:path: /etc/ssl/certstype: DirectoryOrCreatename: ca-certs- hostPath:path: /etc/pkitype: DirectoryOrCreatename: etc-pki- hostPath:path: /etc/kubernetes/pkitype: DirectoryOrCreatename: k8s-certs
status: {}

重新启动apiserver

获取apiserver pod名：
export apiserver_pods=$(kubectl get pods --selector=component=kube-apiserver -n kube-system --output=jsonpath={.items..metadata.name})
删除上面获取的pod：
kubectl delete pod $apiserver_pods -n kube-system

更改ingress端口

1、找到部署ingress-nginx的yml文件注释如下内容：

hostNetwork: true

不进行注释默认使用80，443端口。
更改默认端口记得注释！由于本人忘记注释，netstat -tnlp |grep 80 发现端口仍被占用，这可能就会导致ingress使用出现异常。

2、修改ingress service yml文件配置，添加nodePort后，重新部署。

apiVersion: v1
kind: Service
metadata:name: ingress-nginxnamespace: ingress-nginxlabels:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginx
spec:type: NodePortports:- name: httpport: 80nodePort: 6001targetPort: 80protocol: TCP- name: httpsport: 443nodePort: 6002targetPort: 443protocol: TCPselector:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginx

查看ingress-nginx使用的端口是否更改

[root@k8s-master ingress]# kubectl get svc -n gray
NAME     TYPE         CLUSTER-IP       EXTERNAL-IP    PORT(S)                     AGE
gray     nodeport    10.10.213.255     <none>         80:6001/TCP,443:6002/TCP    2m12s

ingress-nginx部署方式不同，yml文件不同，这里只是个人的解决方式，仅限参考。