OpenShift 4 之 GitOps（3）用Helm+ArgoCD部署应用，并保持配置同步
2024-05-06 21:47:32
《OpenShift 4.x HOL教程汇总》
说明：本文已经在OpenShift 4.8环境中验证
文章目录
- 运行环境
- 用Helm创建样例Chart
- 根据Helm Chart安装OpenShift应用
- 从Helm Chat导出要部署的应用对象
- 将应用资源配置文件推送至Github的Repo
- 根据Github的配置创建OpenShift的应用资源
- 自动调整OpenShift的配置，以保持和Github中的配置同步
- 将Github中的新版配置同步更新至OpenShift
运行环境
- 安装ArgoCD服务器环境。
- 根据《OpenShift 4 - 使用Helm部署OpenShift应用》安装helm客户端即可。
用Helm创建样例Chart
- 创建名为myapp的helm chart，然后可以查看chart中的deployment.yaml。
$ mkdir helmstuff
$ cd helmstuff/
$ helm create myapp
Creating myapp
$ tree myapp/
myapp/
├── charts
├── Chart.yaml
├── templates
│ ├── deployment.yaml
│ ├── _helpers.tpl
│ ├── hpa.yaml
│ ├── ingress.yaml
│ ├── NOTES.txt
│ ├── serviceaccount.yaml
│ ├── service.yaml
│ └── tests
│ └── test-connection.yaml
└── values.yaml
根据Helm Chart安装OpenShift应用
- 创建OpenShift的helmstuff项目，然后通过helm安装应用。
$ oc new-project helmstuff
$ helm install adventure1 myapp/ -n helmstuff
NAME: adventure1
LAST DEPLOYED: Sat Dec 26 08:13:17 2020
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")echo "Visit http://127.0.0.1:8080 to use your application"kubectl --namespace helmstuff port-forward $POD_NAME 8080:80
- 查看Helm列表。
$ helm list -n helmstuff
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
adventure1 helmstuff 1 2020-12-26 08:13:17.653701005 +0000 UTC deployed myapp-0.1.0 1.16.0
- 查看pod状态为CrashLoopBackOff，然后helm中的adventure1，确认结果报错“Error: pod adventure1-myapp-test-connection failed”。
$ oc get pod -n helmstuff
NAME READY STATUS RESTARTS AGE
adventure1-myapp-5b64cf64cb-r65fk 0/1 CrashLoopBackOff 1 32s$ helm test adventure1 -n helmstuff
NAME: adventure1
LAST DEPLOYED: Sat Oct 2 08:36:28 2021
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 1
TEST SUITE: adventure1-myapp-test-connection
Last Started: Sat Oct 2 08:39:18 2021
Last Completed: Sat Oct 2 08:41:41 2021
Phase: Failed
NOTES:
1. Get the application URL by running these commands:export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")export CONTAINER_PORT=$(kubectl get pod --namespace helmstuff $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")echo "Visit http://127.0.0.1:8080 to use your application"kubectl --namespace helmstuff port-forward $POD_NAME 8080:$CONTAINER_PORT
Error: pod adventure1-myapp-test-connection failed
- 上面错误是由于标准的nginx容器是运行在80端口的，这在OpenShift默认是不被允许的。需要执行以下命令为名为“adventure1-myapp”的运行容器的 ServiceAccount 系统用户提权（将执行“anyuid”的权限赋给“helmstuff”项目中的“adventure1-myapp”系统用户）。
$ oc get sa adventure1-myapp -n helmstuff
NAME SECRETS AGE
adventure1-myapp 2 6m34s$ oc adm policy add-scc-to-user anyuid system:serviceaccount:helmstuff:adventure1-myapp
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "adventure1-myapp"
- 先从helm中删除adventure1，然后重新创建helm的adventure1，最后再用helm测试adventure1，确认这次可测通过。
$ helm uninstall adventure1 -n helmstuff
release "adventure1" uninstalled$ helm install adventure1 myapp/ -n helmstuff
NAME: adventure1
LAST DEPLOYED: Sat Oct 2 08:47:30 2021
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")export CONTAINER_PORT=$(kubectl get pod --namespace helmstuff $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")echo "Visit http://127.0.0.1:8080 to use your application"kubectl --namespace helmstuff port-forward $POD_NAME 8080:$CONTAINER_PORT$ helm test adventure1
NAME: adventure1
LAST DEPLOYED: Sat Oct 2 08:47:30 2021
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 1
TEST SUITE: adventure1-myapp-test-connection
Last Started: Sat Oct 2 08:48:44 2021
Last Completed: Sat Oct 2 08:48:51 2021
Phase: Succeeded
NOTES:
1. Get the application URL by running these commands:export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")export CONTAINER_PORT=$(kubectl get pod --namespace helmstuff $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")echo "Visit http://127.0.0.1:8080 to use your application"kubectl --namespace helmstuff port-forward $POD_NAME 8080:$CONTAINER_PORT
- 此时应用已经部署好。
$ oc get deployment -n helmstuff
NAME READY UP-TO-DATE AVAILABLE AGE
adventure1-myapp 1/1 1 1 17m
从Helm Chat导出要部署的应用对象
- 创建manifest目录，让后将helm中名为adventure1的manifest导出到manifest/adventure1.yaml。
$ mkdir manifest
$ helm get manifest adventure1 > manifest/adventure1.yaml
- 查看导出的manifest/adventure1.yaml文件内容。
$ cat manifest/adventure1.yaml
---
# Source: myapp/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: adventure1-myapplabels:helm.sh/chart: myapp-0.1.0app.kubernetes.io/name: myappapp.kubernetes.io/instance: adventure1app.kubernetes.io/version: "1.16.0"app.kubernetes.io/managed-by: Helm
---
# Source: myapp/templates/service.yaml
apiVersion: v1
kind: Service
metadata:name: adventure1-myapplabels:helm.sh/chart: myapp-0.1.0app.kubernetes.io/name: myappapp.kubernetes.io/instance: adventure1app.kubernetes.io/version: "1.16.0"app.kubernetes.io/managed-by: Helm
spec:type: ClusterIPports:- port: 80targetPort: httpprotocol: TCPname: httpselector:app.kubernetes.io/name: myappapp.kubernetes.io/instance: adventure1
---
# Source: myapp/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: adventure1-myapplabels:helm.sh/chart: myapp-0.1.0app.kubernetes.io/name: myappapp.kubernetes.io/instance: adventure1app.kubernetes.io/version: "1.16.0"app.kubernetes.io/managed-by: Helm
spec:replicas: 1selector:matchLabels:app.kubernetes.io/name: myappapp.kubernetes.io/instance: adventure1template:metadata:labels:app.kubernetes.io/name: myappapp.kubernetes.io/instance: adventure1spec:serviceAccountName: adventure1-myappsecurityContext:{}containers:- name: myappsecurityContext:{}image: "nginx:1.16.0"imagePullPolicy: IfNotPresentports:- name: httpcontainerPort: 80protocol: TCPlivenessProbe:httpGet:path: /port: httpreadinessProbe:httpGet:path: /port: httpresources:{}
将应用资源配置文件推送至Github的Repo
- 确认当前所在目录。
$ ls
manifest myapp
- 在你的Github账号上创建名为“gitops-helm-argocd”的Repository。
- 依次执行以下命令，将myapp应用资源推送的自己的Github账户中。
$ git init
Initialized empty Git repository in /home/xiaoyliu-redhat.com/helmstuff/.git/
$ git add *
$ git commit -m "initial commit of helm chart and working manifest"
[master (root-commit) c03cd60] initial commit of helm chart and working manifestCommitter: GTPE Student <xiaoyliu-redhat.com@clientvm.beijing-b510.internal>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:git config --global user.name "Your Name"git config --global user.email you@example.comAfter doing this, you may fix the identity used for this commit with:git commit --amend --reset-author11 files changed, 416 insertions(+)create mode 100644 manifest/adventure1.yamlcreate mode 100644 myapp/.helmignorecreate mode 100644 myapp/Chart.yamlcreate mode 100644 myapp/templates/NOTES.txtcreate mode 100644 myapp/templates/_helpers.tplcreate mode 100644 myapp/templates/deployment.yamlcreate mode 100644 myapp/templates/ingress.yamlcreate mode 100644 myapp/templates/service.yamlcreate mode 100644 myapp/templates/serviceaccount.yamlcreate mode 100644 myapp/templates/tests/test-connection.yamlcreate mode 100644 myapp/values.yaml$ git remote add origin https://github.com/YOUR-GITHUB/gitops-helm-argocd.git
$ git push -u origin master
Username for 'https://github.com': liuxiaoyu-git
Password for 'https://liuxiaoyu-git@github.com':
Counting objects: 17, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (15/15), done.
Writing objects: 100% (17/17), 5.30 KiB | 0 bytes/s, done.
Total 17 (delta 0), reused 0 (delta 0)
To https://github.com/liuxiaoyu-git/gitops-helm-argocd.git* [new branch] master -> master
Branch master set up to track remote branch master from origin.
- 确认Github中名为“gitops-helm-argocd”的Repository中已经有“myapp”和“** manifest**”了。
根据Github的配置创建OpenShift的应用资源
- 将Github资源加到ArgoCD中的Repo。
$ argocd repo add https://github.com/YOUR-GITHUB/gitops-helm-argocd.git --name gitops-helm-argocd
repository 'https://github.com/liuxiaoyu-git/gitops-helm-argocd.git' added
$ argocd repo list
TYPE NAME REPO INSECURE LFS CREDS STATUS MESSAGE
git https://github.com/liuxiaoyu-git/gitops-helm-argocd.git false false false Successful
- 创建部署应用的项目，并增加“argocd.argoproj.io/managed-by=openshift-gitops”标签。
$ TARGET=adventure1-myapp
$ oc new-project ${TARGET}
$ oc label namespace ${TARGET} argocd.argoproj.io/managed-by=openshift-gitops
- 新建一个名为adventure1的ArgoCD应用，用它在github中的配置资源与OpenShift中的helmstuff项目建立关联。
$ argocd app create --name adventure1 --project default \--repo https://github.com/liuxiaoyu-git/gitops-helm-argocd.git --revision master --path manifest \--dest-server https://kubernetes.default.svc --dest-namespace ${TARGET} \--sync-policy automated --self-heal
- 同理，也需要为
$ oc adm policy add-scc-to-user anyuid system:serviceaccount:${TARGET}:adventure1-myapp
- 进入ArgoCD的控制台，查看adventure1应用。确认当前OpenShift的用资源和Github Repo中的资源是“Synced”的。
自动调整OpenShift的配置，以保持和Github中的配置同步
- 根据名为adventure1-myapp的OpenShift Service对象手动生成一个新的对象：adventure1-mybad。
$ oc get svc adventure1-myapp -n ${TARGET} -o json \| jq 'del(.spec.clusterIP)' \| jq 'del(.spec.clusterIPs)' \| sed "s/\"name\": \"adventure1-myapp\"/\"name\": \"adventure1-mybad\"/g" \| oc create -n ${TARGET} -f -
service/adventure1-mybad created
$ oc get svc -n ${TARGET}
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
adventure1-myapp ClusterIP 172.30.12.170 <none> 80/TCP 3h38m
adventure1-mybad ClusterIP 172.30.225.179 <none> 80/TCP 10s
- 此时可在ArgoCD控制台中的adventure1应用界面中看到已经是“OutOfSync”状态，且在名为adventure1-mybad的Service下方显示了黄色标记。
- 修改ArgoCD中adventure1应用的配置（“auto-prune”用来自动删除比ArgoCD多余的对象），然后通过ArgoCD控制台确认名为adventure1-mybad的Service已经被删除，而此时adventure1也转为Synced状态。
$ argocd app set adventure1 --sync-policy automated --auto-prune --self-heal
$ oc get svc -n ${TARGET}
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
adventure1-myapp ClusterIP 172.30.12.170 <none> 80/TCP 3h59m
4. 删除本项目中OpenShift的Deployment对象，确认ArgoCD会自动根据Github的配置重新创建一个新的Deployment对象。
$ oc delete deployment adventure1-myapp -n ${TARGET} && oc get pods -n ${TARGET} -w
deployment.extensions "adventure1-myapp" deleted
NAME READY STATUS RESTARTS AGE
adventure1-myapp-5b64cf64cb-l9p5g 1/1 Terminating 1 3h47m
adventure1-myapp-5b64cf64cb-l9p5g 1/1 Terminating 1 3h47m
adventure1-myapp-5b64cf64cb-l9p5g 0/1 Terminating 1 3h47m
adventure1-myapp-5b64cf64cb-l9p5g 0/1 Terminating 1 3h47m
adventure1-myapp-5b64cf64cb-2kf2n 0/1 ContainerCreating 0 10s
adventure1-myapp-5b64cf64cb-2kf2n 0/1 ContainerCreating 0 18s
adventure1-myapp-5b64cf64cb-2kf2n 0/1 Running 0 18s
adventure1-myapp-5b64cf64cb-2kf2n 1/1 Running 0 19s
将Github中的新版配置同步更新至OpenShift
- 先设置ArogCD，关闭Github和OpenShift自动同步配置的功能。
$ argocd app set adventure1 --sync-policy none
- 修改现有Helm Chart，将version从“0.1.0”改为“0.1.1”，将appVersion从“1.16.0”改为“1.16.1”.
$ sed -i 's/1.16.0/1.16.1/g' myapp/Chart.yaml
$ sed -i 's/0.1.0/0.1.1/g' myapp/Chart.yaml
$ cat myapp/Chart.yaml
apiVersion: v2
name: myapp
description: A Helm chart for Kubernetes# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
version: 0.1.1# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application.
appVersion: 1.16.1
- 更新Helm中的adventure1配置，并查看改配置修改前后的变化。
$ helm list -n helmstuff
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
adventure1 helmstuff 1 2020-12-26 08:18:17.200103853 +0000 UTC deployed myapp-0.1.0 1.16.0$ helm upgrade adventure1 myapp/ -n helmstuff
Release "adventure1" has been upgraded. Happy Helming!
NAME: adventure1
LAST DEPLOYED: Sat Dec 26 09:03:56 2020
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 2
NOTES:
1. Get the application URL by running these commands:export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")export CONTAINER_PORT=$(kubectl get pod --namespace helmstuff $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")echo "Visit http://127.0.0.1:8080 to use your application"kubectl --namespace helmstuff port-forward $POD_NAME 8080:$CONTAINER_PORT$ helm list -n helmstuff
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
adventure1 helmstuff 2 2020-12-26 09:03:56.028781132 +0000 UTC deployed myapp-0.1.1 1.16.1
- 用helm重新生成manifest/adventure1.yaml文件，然后可确认对象标签已经变为“myapp-0.1.1”和“1.16.1”。
$ helm get manifest adventure1 -n helmstuff > manifest/adventure1.yaml
$ more manifest/adventure1.yaml
# Source: myapp/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: adventure1-myapplabels:helm.sh/chart: myapp-0.1.1app.kubernetes.io/name: myappapp.kubernetes.io/instance: adventure1app.kubernetes.io/version: "1.16.1"app.kubernetes.io/managed-by: Helm
- 比较新旧版adventure1.yaml文件后，将本地变化的配置文件提交到Github。提交后可在Github上确认“manifest/adventure1.yaml”已经更新。
$ git diff manifest/adventure1.yaml
$ git add *
$ git commit -m "updated app version and manifests"
[master b844df1] updated app version and manifestsCommitter: GTPE Student <xiaoyliu-redhat.com@clientvm.beijing-b510.internal>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:git config --global user.name "Your Name"git config --global user.email you@example.comAfter doing this, you may fix the identity used for this commit with:git commit --amend --reset-author2 files changed, 9 insertions(+), 9 deletions(-)$ git push origin master
Username for 'https://github.com': liuxiaoyu-git
Password for 'https://liuxiaoyu-git@github.com':
Counting objects: 11, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 534 bytes | 0 bytes/s, done.
Total 6 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/liuxiaoyu-git/gitops-helm-argocd.gitc03cd60..b844df1 master -> master
- 通过命令和控制台查看ArgoCD的adventure1应用的同步状态，发现此时对于变化的配置，OpenShift和Github是没有同步的。
$ argocd app get adventure1
Name: adventure1
Project: default
Server: https://kubernetes.default.svc
Namespace: helmstuff
URL: https://argocd-server-argocd.apps.cluster-pek-99bc.pek-99bc.sandbox309.opentlc.com/applications/adventure1
Repo: https://github.com/liuxiaoyu-git/gitops-helm-argocd.git
Target: master
Path: manifest
SyncWindow: Sync Allowed
Sync Policy: <none>
Sync Status: OutOfSync from master (f294628)
Health Status: HealthyGROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE
apps Deployment helmstuff adventure1-myapp OutOfSync Healthy deployment.apps/adventure1-myapp createdPod helmstuff adventure1-myapp-test-connection HealthyService helmstuff adventure1-myapp OutOfSync HealthyServiceAccount helmstuff adventure1-myapp OutOfSync
- 在ArdoCD控制台中查看名为adventure1-myapp的Service的详细配置，其中在DIFF中显示了这个服务在OpenShift和Github的配置差异。
- 此时再次打开ArgoCD的同步选项。然后在ArgoCD控制台中确认adventure1应用的同步状态应已经变为“Synced”。
$ argocd app set adventure1 --sync-policy automated --auto-prune --self-heal
OpenShift 4 之 GitOps（3）用Helm+ArgoCD部署应用，并保持配置同步相关推荐
- OpenShift 4 之 GitOps（1）通过OpenShift GitOps Operator安装ArgoCD
<OpenShift 4.x HOL教程汇总> 说明:本文已经在OpenShift 4.8环境中验证 文章目录 安装 ArgoCD 用 OpenShift GitOps Operator ...
- OpenShift 4 之 GitOps（7）用ArgoCD部署Pacman应用集群
<OpenShift 4.x HOL教程汇总> 文章目录 Pacman应用部署架构 部署HAProxy 部署Pacman应用 本文是<OpenShift 4 之 GitOps(6)用 ...
- OpenShift 4 之 GitOps（6）用ArgoCD部署MongoDB主从集群
<OpenShift 4.x HOL教程汇总> 文章目录 部署架构 部署MongoDB主从集群 准备操作环境 配置ArgoCD 准备MongoDB所需证书 更新被部署的YAML文件 在三个 ...
- OpenShift 4 之 GitOps（2）用ArgoCD部署应用
<OpenShift 4.x HOL教程汇总> 说明:本文已经在OpenShift 4.8环境中验证 文章目录 用ArgoCD部署应用 向ArgoCD增加Github Repo 通过Arg ...
- OpenShift 之 Quarkus（2）使用S2I部署Quarkus应用
<OpenShift 4.x HOL教程汇总> 文章目录 将Quarkus源码按可执行程序部署到OpenShift 将Quarkus源码按Java应用部署到OpenShift 我们除了可以 ...
- Helm入门+部署mysql
Helm入门+部署mysql 1简介 2 安装 3 指定k8s集群 4安装MySQL 5 PVC添加PV支持 1简介 Helm 为团队提供了在 Kubernetes 内部创建.安装和管理应用程序时需要 ...
- OpenShift 4 之 GitOps（8）用ArgoCD实现的应用迁移、金丝雀部署、DR
<OpenShift 4.x HOL教程汇总> 使用ArgoCD还可实现以下场景,可参考文档说明. 应用可移植性 金丝雀部署 灾难恢复
- OpenShift 4 之 GitOps（5）用ArgoCD配置其他OpenShift资源
<OpenShift 4.x HOL教程汇总> 说明:本文已经在OpenShift 4.6环境中验证 文章目录 创建用户 设置Build的全局属性 设置Import Image的全局属性 ...
- OpenShift 4 之 GitOps（4）用ArgoCD向Multi-Cluster发布应用
<OpenShift 4.x HOL教程汇总> 文章目录 运行环境 通过Config Context访问不同OpenShift集群 在ArgoCD注册OpenShift集群 通过ArgoC ...
最新文章
- Android 根据从服务器中获取的rgb值实现动态改变圆角加框的Imageview 的背景色
- php里怎么添加计时器,如何使用php显示计时器?
- label美化css,表单label美化代码
- kafka解决了什么问题?
- 2021云数据库RDS重磅升级发布会
- 如何将本地项目上传到gitee
- 水滴石穿C语言之可变参数问题
- springboot项目发布JAR包
- verilog实现多周期处理器之——（四）逻辑，移位操作与空指令的添加
- /usr/bin/ld: cannot find -l*
- PAT——1054. 求平均值
- 三种方法在地图上绘制网络图
- 动网论坛“数据库连接出错”处理记-间歇博客
- 微型计算机主机作用,微型计算机的主机包括()。
- 【Transfer Learning】泛化到未知域：域泛化 (Domain Generalization) 综述论文
- protoc库的卸载与安装
- （已更新）日常记账微信小程序模板源码
- C语言计算n阶行列式
- 19 01 18 dango 模型
- 三、运算符 | Verilog
热门文章
- 用螺纹铣刀铣螺纹转速进给怎么设_机加工中最常用的几种钨钢铣刀刀具介绍
- kafka使用_Kafka生产者的使用和原理
- linux 格式化u盘_使用Universal USB Installer，打造你的U盘版Linux系统
- raid5 合适 多少块硬盘_分析Linux raid6同步成raid5导致数据丢失的情况
- 锤子t1重置后怎么显示无服务器,解决锤子手机smartisanT1关机后无法正常开机（附带刷机教程图文）...
- java发送请求_Java发送Http请求
- 设计灵感|延展画面的插画Banner设计！
- arm nodejs_英伟达打算收购Arm，或将成有史以来最大半导体收购案
- std::auto_ptr简单使用
- 直接插入排序（C语言）实现