helm部署jenkins到k8s并创建pipeline构建项目
安装部署nfs
1. nfs服务器创建目录
[yeqiang@harbor ~]$ sudo mkdir /home/nfs/jenkins -p
2. 设置其他用户可以写入该目录
[yeqiang@harbor ~]$ sudo chmod o+w /home/nfs/jenkins/ -R
3. 配置/etc/exports
/home/nfs/jenkins 192.168.1.167(rw,sync,no_subtree_check,no_root_squash)
/home/nfs/jenkins/agent 192.168.1.167(rw,sync,no_subtree_check,no_root_squash)
4. 启动nfs服务,配置开启自启动
[yeqiang@harbor ~]$ sudo systemctl enable nfs-server --now
Created symlink /etc/systemd/system/multi-user.target.wants/nfs-server.service → /usr/lib/systemd/system/nfs-server.service.
5. 检查服务及端口(111/tcp)状态
[yeqiang@harbor ~]$ netstat -lnetp | grep 111
(Not all processes could be identified, non-owned process infowill not be shown, you would have to be root to see it all.)
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 0 19646 -
tcp6 0 0 :::111 :::* LISTEN 0 19650 -
[yeqiang@harbor ~]$ sudo exportfs -v
/home/nfs/jenkins192.168.1.167(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
/home/nfs/jenkins/agent192.168.1.167(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
创建k8s存储
1. 创建StorageClass,配置文件nfs-storege-class.yml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:name: nfs-storageannotations:storageclass.kubernetes.io/is-default-class: 'false'
provisioner: kubernetes.io/no-provisioner
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
执行创建
[yeqiang@harbor k8s]$ kubectl apply -f nfs-storege-class.yml
storageclass.storage.k8s.io/nfs-storage created
[yeqiang@harbor k8s]$ kubectl get storageclass
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-storage kubernetes.io/no-provisioner Delete WaitForFirstConsumer false 6s
2. 创建PersistantVolume,配置文件jenkins-pv.yml
apiVersion: v1
kind: PersistentVolume
metadata:name: jenkins-pv
spec:capacity:storage: 10GiaccessModes:- ReadWriteOncevolumeMode: FilesystempersistentVolumeReclaimPolicy: RecyclestorageClassName: "nfs-storage"nfs:# 根据实际共享目录修改path: /home/nfs/jenkins# 根据实际 nfs服务器地址修改server: 192.168.1.167
执行创建
[yeqiang@harbor k8s]$ kubectl apply -f jenkins-pv.yml
persistentvolume/jenkins-pv created
[yeqiang@harbor k8s]$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
jenkins-pv 10Gi RWO Recycle Available nfs-storage 6s
helm部署jenkins
1. helm查看jenkins信息
[yeqiang@harbor k8s]$ helm repo add jenkins https://charts.jenkins.io
"jenkins" has been added to your repositories
[yeqiang@harbor k8s]$ helm search repo jenkins
NAME CHART VERSION APP VERSION DESCRIPTION
jenkins/jenkins 2.15.1 lts Open source continuous integration server. It s...
stable/jenkins 2.5.4 lts DEPRECATED - Open source continuous integration...
注意stable/jenkins已经标注过时了,我们选择jenkins/jenkins
2. 创建namespace
[yeqiang@harbor k8s]$ kubectl create ns jenkins
namespace/jenkins created
3. helm安装
[yeqiang@harbor k8s]$ helm install jenkins jenkins/jenkins -n jenkins --set persistence.storageClass="nfs-storage"
NAME: jenkins
LAST DEPLOYED: Thu Oct 29 10:25:54 2020
NAMESPACE: jenkins
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'admin' user password by running:printf $(kubectl get secret --namespace jenkins jenkins -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode);echo
2. Get the Jenkins URL to visit by running these commands in the same shell:export POD_NAME=$(kubectl get pods --namespace jenkins -l "app.kubernetes.io/component=jenkins-master" -l "app.kubernetes.io/instance=jenkins" -o jsonpath="{.items[0].metadata.name}")echo http://127.0.0.1:8080kubectl --namespace jenkins port-forward $POD_NAME 8080:80803. Login with the password from step 1 and the username: admin4. Use Jenkins Configuration as Code by specifying configScripts in your values.yaml file, see documentation: http:///configuration-as-code and examples: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demosFor more information on running Jenkins on Kubernetes, visit:
https://cloud.google.com/solutions/jenkins-on-container-engine
For more information about Jenkins Configuration as Code, visit:
https://jenkins.io/projects/jcasc/
参数说明:
persistence.storageClass 赋值为刚才创建的StorageClass名称nfs-sotrage
4. 故障排除
[yeqiang@harbor k8s]$ kubectl get pods -n jenkins
NAME READY STATUS RESTARTS AGE
jenkins-58978c6d6d-dszqz 1/2 Running 2 6m8s
[yeqiang@harbor k8s]$ kubectl describe pod jenkins-58978c6d6d-dszqz -n jenkins
Name: jenkins-58978c6d6d-dszqz
Namespace: jenkins
Priority: 0
Node: 192.168.1.167/192.168.1.167
Start Time: Thu, 29 Oct 2020 10:25:55 +0800
Labels: app.kubernetes.io/component=jenkins-masterapp.kubernetes.io/instance=jenkinsapp.kubernetes.io/managed-by=Helmapp.kubernetes.io/name=jenkinspod-template-hash=58978c6d6d
Annotations: checksum/config: 9823a20535eb4e992b6741805b204c7c82d33080079887578e27c24328df7b29
Status: Running
IP: 172.20.0.108
IPs:IP: 172.20.0.108
Controlled By: ReplicaSet/jenkins-58978c6d6d
Init Containers:copy-default-config:Container ID: docker://e29d8d6285ddebaf565a206213da47aac723ae79a883ff6108aafd7108a5b1fdImage: jenkins/jenkins:ltsImage ID: docker-pullable://jenkins/jenkins@sha256:aa1ec2a6a106608a286dc06c334053fced1df87a7cb307853ca1cf8fa48ba09bPort: <none>Host Port: <none>Command:sh/var/jenkins_config/apply_config.shState: TerminatedReason: CompletedExit Code: 0Started: Thu, 29 Oct 2020 10:25:58 +0800Finished: Thu, 29 Oct 2020 10:26:40 +0800Ready: TrueRestart Count: 0Limits:cpu: 2memory: 4GiRequests:cpu: 50mmemory: 256MiEnvironment:ADMIN_PASSWORD: <set to the key 'jenkins-admin-password' in secret 'jenkins'> Optional: falseADMIN_USER: <set to the key 'jenkins-admin-user' in secret 'jenkins'> Optional: falseMounts:/tmp from tmp (rw)/usr/share/jenkins/ref/plugins from plugins (rw)/var/jenkins_config from jenkins-config (rw)/var/jenkins_home from jenkins-home (rw)/var/jenkins_plugins from plugin-dir (rw)/var/run/secrets/kubernetes.io/serviceaccount from jenkins-token-krt5l (ro)
Containers:jenkins:Container ID: docker://13cb74bdc25782b5bf7258fc4c369544571866c3a029dbc26f9ae489fb7b742bImage: jenkins/jenkins:ltsImage ID: docker-pullable://jenkins/jenkins@sha256:aa1ec2a6a106608a286dc06c334053fced1df87a7cb307853ca1cf8fa48ba09bPorts: 8080/TCP, 50000/TCPHost Ports: 0/TCP, 0/TCPArgs:--argumentsRealm.passwd.$(ADMIN_USER)=$(ADMIN_PASSWORD)--argumentsRealm.roles.$(ADMIN_USER)=admin--httpPort=8080State: RunningStarted: Thu, 29 Oct 2020 10:31:24 +0800Last State: TerminatedReason: ErrorExit Code: 143Started: Thu, 29 Oct 2020 10:29:01 +0800Finished: Thu, 29 Oct 2020 10:31:18 +0800Ready: FalseRestart Count: 2Limits:cpu: 2memory: 4GiRequests:cpu: 50mmemory: 256MiLiveness: http-get http://:http/login delay=90s timeout=5s period=10s #success=1 #failure=5Readiness: http-get http://:http/login delay=60s timeout=5s period=10s #success=1 #failure=3Environment:POD_NAME: jenkins-58978c6d6d-dszqz (v1:metadata.name)JAVA_OPTS: -Dcasc.reload.token=$(POD_NAME)JENKINS_OPTS: JENKINS_SLAVE_AGENT_PORT: 50000ADMIN_PASSWORD: <set to the key 'jenkins-admin-password' in secret 'jenkins'> Optional: falseADMIN_USER: <set to the key 'jenkins-admin-user' in secret 'jenkins'> Optional: falseCASC_JENKINS_CONFIG: /var/jenkins_home/casc_configsMounts:/tmp from tmp (rw)/usr/share/jenkins/ref/plugins/ from plugin-dir (rw)/var/jenkins_config from jenkins-config (ro)/var/jenkins_home from jenkins-home (rw)/var/jenkins_home/casc_configs from sc-config-volume (rw)/var/run/secrets/kubernetes.io/serviceaccount from jenkins-token-krt5l (ro)jenkins-sc-config:Container ID: docker://a33d84511d702b29e1d1b206793666bb360f071e0f7577a24f660c8a0d4eef1bImage: kiwigrid/k8s-sidecar:0.1.193Image ID: docker-pullable://kiwigrid/k8s-sidecar@sha256:170069ff097679f8fafcf01510d1fe05260af683b949fd2b05866e18ca4d9c74Port: <none>Host Port: <none>State: RunningStarted: Thu, 29 Oct 2020 10:26:48 +0800Ready: TrueRestart Count: 0Environment:POD_NAME: jenkins-58978c6d6d-dszqz (v1:metadata.name)LABEL: jenkins-jenkins-configFOLDER: /var/jenkins_home/casc_configsNAMESPACE: jenkinsREQ_URL: http://localhost:8080/reload-configuration-as-code/?casc-reload-token=$(POD_NAME)REQ_METHOD: POSTREQ_RETRY_CONNECT: 10Mounts:/var/jenkins_home from jenkins-home (rw)/var/jenkins_home/casc_configs from sc-config-volume (rw)/var/run/secrets/kubernetes.io/serviceaccount from jenkins-token-krt5l (ro)
Conditions:Type StatusInitialized True Ready False ContainersReady False PodScheduled True
Volumes:plugins:Type: EmptyDir (a temporary directory that shares a pod's lifetime)Medium: SizeLimit: <unset>tmp:Type: EmptyDir (a temporary directory that shares a pod's lifetime)Medium: SizeLimit: <unset>jenkins-config:Type: ConfigMap (a volume populated by a ConfigMap)Name: jenkinsOptional: falseplugin-dir:Type: EmptyDir (a temporary directory that shares a pod's lifetime)Medium: SizeLimit: <unset>jenkins-home:Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)ClaimName: jenkinsReadOnly: falsesc-config-volume:Type: EmptyDir (a temporary directory that shares a pod's lifetime)Medium: SizeLimit: <unset>jenkins-token-krt5l:Type: Secret (a volume populated by a Secret)SecretName: jenkins-token-krt5lOptional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300snode.kubernetes.io/unreachable:NoExecute for 300s
Events:Type Reason Age From Message---- ------ ---- ---- -------Normal Scheduled 6m10s default-scheduler Successfully assigned jenkins/jenkins-58978c6d6d-dszqz to 192.168.1.167Normal Pulling 6m9s kubelet, 192.168.1.167 Pulling image "jenkins/jenkins:lts"Normal Pulled 6m8s kubelet, 192.168.1.167 Successfully pulled image "jenkins/jenkins:lts"Normal Created 6m7s kubelet, 192.168.1.167 Created container copy-default-configNormal Started 6m7s kubelet, 192.168.1.167 Started container copy-default-configNormal Pulled 5m18s kubelet, 192.168.1.167 Successfully pulled image "jenkins/jenkins:lts"Normal Created 5m18s kubelet, 192.168.1.167 Created container jenkinsNormal Started 5m17s kubelet, 192.168.1.167 Started container jenkinsNormal Pulled 5m17s kubelet, 192.168.1.167 Container image "kiwigrid/k8s-sidecar:0.1.193" already present on machineNormal Created 5m17s kubelet, 192.168.1.167 Created container jenkins-sc-configNormal Started 5m17s kubelet, 192.168.1.167 Started container jenkins-sc-configWarning Unhealthy 3m11s (x7 over 4m11s) kubelet, 192.168.1.167 Readiness probe failed: Get http://172.20.0.108:8080/login: dial tcp 172.20.0.108:8080: connect: connection refusedNormal Pulling 3m7s (x2 over 5m24s) kubelet, 192.168.1.167 Pulling image "jenkins/jenkins:lts"Normal Killing 3m7s kubelet, 192.168.1.167 Container jenkins failed liveness probe, will be restartedWarning Unhealthy 67s (x8 over 3m47s) kubelet, 192.168.1.167 Liveness probe failed: Get http://172.20.0.108:8080/login: dial tcp 172.20.0.108:8080: connect: connection refused
Liveness probe failed:
把deployments里面的jenkins配置项目failureThreshold改为10
原因:jenkins默认引用的国外更新源,初始化时间较长,pod还未就绪就被k8s杀死了。
可能应为网络问题,pod可能还会反创建。大约十几分钟后。可以正常访问
5. 访问Jenkins
查看访问地址
[yeqiang@harbor k8s]$ kubectl get svc -n jenkins
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jenkins ClusterIP 10.68.62.85 <none> 8080/TCP 26m
jenkins-agent ClusterIP 10.68.46.25 <none> 50000/TCP 26m
查看登陆用户及密码
[yeqiang@harbor k8s]$ kubectl edit secret jenkins -n jenkins
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:jenkins-admin-password: clM5bXgxMUNvQw==jenkins-admin-user: YWRtaW4=
kind: Secret
metadata:annotations:meta.helm.sh/release-name: jenkinsmeta.helm.sh/release-namespace: jenkinscreationTimestamp: "2020-10-29T02:25:54Z"labels:app.kubernetes.io/component: jenkins-masterapp.kubernetes.io/instance: jenkinsapp.kubernetes.io/managed-by: Helmapp.kubernetes.io/name: jenkinshelm.sh/chart: jenkins-2.15.1managedFields:- apiVersion: v1fieldsType: FieldsV1fieldsV1:f:data:.: {}f:jenkins-admin-password: {}f:jenkins-admin-user: {}f:metadata:f:annotations:.: {}f:meta.helm.sh/release-name: {}f:meta.helm.sh/release-namespace: {}f:labels:.: {}f:app.kubernetes.io/component: {}f:app.kubernetes.io/instance: {}f:app.kubernetes.io/managed-by: {}f:app.kubernetes.io/name: {}f:helm.sh/chart: {}f:type: {}manager: Go-http-clientoperation: Updatetime: "2020-10-29T02:25:54Z"name: jenkinsnamespace: jenkinsresourceVersion: "2557784"selfLink: /api/v1/namespaces/jenkins/secrets/jenkinsuid: e8eb14b7-2e7c-42b4-83b5-39889042e72b
type: Opaque
[yeqiang@harbor k8s]$ echo 'YWRtaW4=' | base64 -d
admin
[yeqiang@harbor k8s]$ echo 'clM5bXgxMUNvQw==' | base64 -d
rS9mx11CoC
得到用户admin,密码rS9mx11CoC
登陆系统
创建jenkins PipeLine构建部署项目
1. 查看jenkins cloud 配置
2. 编写pipeline
def label = "jenkins-slave"
podTemplate(label: label, cloud: 'kubernetes', containers: [containerTemplate(alwaysPullImage: true, args: 'cat', command: '/bin/sh -c', image: 'maven:3.6.3-openjdk-8', livenessProbe: containerLivenessProbe(execArgs: '', failureThreshold: 0, initialDelaySeconds: 0, periodSeconds: 0, successThreshold: 0, timeoutSeconds: 0), name: 'maven', resourceLimitCpu: '', resourceLimitMemory: '', resourceRequestCpu: '', resourceRequestMemory: '', ttyEnabled: true, workingDir: '/home/jenkins/agent')], workspaceVolume: nfsWorkspaceVolume(serverAddress: '192.168.1.167', serverPath: '/home/nfs/jenkins/agent', readOnly: 'false')
) {node(label) {stage('git') {git 'https://github.com/hknarutofk/spring-demo.git'}stage('mvn'){container('maven'){ sh '''
cat > /usr/share/maven/conf/settings.xml << EOF
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><localRepository>/root/.m2</localRepository><mirrors><mirror><id>alimaven</id><name>aliyun maven</name><url>http://maven.aliyun.com/nexus/content/groups/public/</url><mirrorOf>central</mirrorOf> </mirror></mirrors>
</settings>
EOF
''' sh "mvn clean package -Dmaven.test.skip=true"} }}
}
故障排除:
Could not resolve host
放弃使用镜像maven:3.3.9-jdk-8-alpine,采用maven:3.6.3-openjdk-8。原因:maven:3.3.9-jdk-8-alpine对/etc/resolv.conf解析有故障
helm部署jenkins到k8s并创建pipeline构建项目相关推荐
- 在 Jenkins 中使用声明式 Pipeline 构建 Android 项目
Blue Ocean 是 Jenkins 推出的一套新的 UI,对比经典 UI 更具有现代化气息.2017 年 4 月 James Dumay 在博客上正式推出了 Blue Ocean 1.0. 兼容 ...
- ASP.NET Core 借助 Helm 部署应用至 K8S
前言 玩K8S也有一段时间了,借助云服务提供商的K8S控制台,已经可以很方便的快速部署应用至K8S.通过简单的点击,可以一次性帮忙创建K8S 对象:Deployment.Service.Ingress ...
- jenkins cicd k8s 多分支流水线构建
环境:centos7.6 .Docker version 19.03.5.Jenkins 2.249.3(jackson = 2.11.2.docker plugin = 1.2.1) 主要思想: 使 ...
- 持续集成部署Jenkins工作笔记0014---curl命令触发构建
技术交流QQ群[JAVA,C++,Python,.NET,BigData,AI]:170933152 好,然后咱们来看看怎么来做这个钩子程序,要先说一个curl命令 咱们先去看这个svn的仓库 可以看 ...
- helm安装_如何利用 Helm 在 Kubernetes 上快速部署 Jenkins
Jenkins 做为最著名的 CI/CD 工具,在全世界范围内被广泛使用,而随着以 Kubernetes 为首的云平台的不断发展与壮大,在 Kubernetes 上运行 Jenkins 的需求越来越多 ...
- k8s部署jenkins和Pod始终为pending状态“persistentvolume-controller no persistent volumes available.....”解决办法
文章目录 眼因多流泪而愈益清明,心因饱经风霜而愈益温厚 部署jenkins到k8s集群 1.部署持久存储卷 2.部署jenkins 报错分析 3.配置 Jenkins kubernetes 插件 眼因 ...
- Jenkins+k8s部署Jenkins slave
一.构建镜像 下载基础镜像,这里使用openvz的包,下载centos7的镜像 1.1 下载镜像后导入到本地 JENKINS VERSION为: 2.222.4 ------------------- ...
- Kubernetes部署(四):k8s项目交付----(1)持续集成
实战交付dubbo服务到k8s 一.什么是Dubbo是什么 ● Dubbo是什么● Dubbo基于java开发的,是阿里巴巴SOA服务化治理方案的核心框架,每天为2,000+个服务提供3,000,00 ...
- 基于Jenkins和k8s构建企业级DevOps容器云平台
1.1 什么是DevOps? DevOps中的Dev是Devlopment(开发),Ops是Operation(运维),用一句话来说 DevOps 就是打通开发运维的壁垒,实现开发运维一体化.DevO ...
最新文章
- deno使用rust_如何在Deno和Oak中使用MySQL
- java 反射机制_Java反射机制原理探究
- 全国大学生智能汽车竞赛证书打印方法
- 如何将Felgo程序部署到Android中
- android 补签控件,问道手游安卓12月8日维护公告 新增补签功能
- 九、二手信息站点后台完成 (IVX 快速开发教程)
- 苹果开发者账号可以创建多少测试证书_ios苹果企业账号你所不知道的那些点
- Java Jvm 中的垃圾回收机制中的思想与算法 《对Java的分析总结》-四
- 【kibana】状态异常 Status: Red Status changed from red to yellow - No existing Kibana index found
- Java中的switch分支语句
- springMVC接收数组参数
- 超定方程的最小二乘解的三维几何解释
- 解决This application failed to start because cannot find or load the qt platform plugin 'xcb'
- (转,感谢原作者!)既然选择了Linux,有何必在乎这些——Linux wine国服LOL英雄联盟,完美运行!!...
- Ubuntu16.04使用语义分割标注工具Semantic-Segmentation-Editor
- mysql杀掉sql语句,Mysql使用kill命令解决死锁问题(杀死某条正在执行的sql语句)
- mysql图形查询操作 点找面及面找点 Polygon获取中心点坐标 空间地理位置计算
- C++试题 算法训练 相邻数对、画图
- 中国银联正式开展非标商户管理试点工作 将严打“大商户模式”
- 网站域名过户查询_域名知识问答之域名过户
热门文章
- 3行代码Python解决图片清晰度识别,原来我们看到的不一定是这样
- 转载:分布式Session共享:tomcat+memcached实现session共享
- 【高级篇 / System】(7.0) ❀ 04. 高可用性 HA 配置 ❀ FortiGate 防火墙
- 64卦加密解密(娱乐用)
- C语言uint8_t和char的区别,c – int8_t和uint8_t是char类型吗?
- CallBack函数调用
- 人生之路 — 开启智慧之脑
- 平面设计必备学习软件
- Nape的回调系统 nape.callbacks
- LeetCode 热题 HOT 100 -------160. 相交链表(链表)206. 反转链表(递归、回溯)