Terraform 学习总结（7）—— 基于 AWS 云平台上的 Terraform 实战
2024-04-24 10:02:57
一、AWS 上基础环境介绍
相信大家已经知道 Terraform 的基本使用方式了。在我们以后的场景中，主要讨论一些典型架构的实现。在实际使用中，Terraform 在多云基础架构创建中，是非常方便和简洁的。Terraform 造了足够多的轮子，使用起来非常顺手，用于创建云上的基础架构。在应用发布上，可以使用 Packer 进行应用的集成，与 Terraform 配合做到应用的自动发布。同时在云上的基础构建完成后，想构建一些复杂系统，基于 OS 之上时，就需要交给 Ansible 和 PowerShell 来进行实现了。例如创建 K8S/Openshift 的集群环境，在创建这些复杂架构的方法上，存在多种工具，但是我们需要企业级发布的时候， 在系统层面的配和应用层面的配置就需要更高的控制精细程度。而这个时候，就凸显了 Terraform 和 Ansible 等工具的控制能力。我们可以精确的控制到每个配置项。无论是 Infra 还是 OS。
这里是一个典型的 AWS 多账号设计，左边的是 Operation 运维组用户的人员的核心账号下的运维主机 Terraform，这台主机上同时集成安装了 ansible 和 packer 工具，用于实现后面的实验，但是实际生产环境中可能会依据性能分拆，当前实验环境中，都是单机实现即可。由图可知，我们通过对于右边受管理的账号 Project or Team 的项目账号进行管理。例如：初始化账号，包括创建组网 VPC/Subnet/Role/Tags、创建 IAM、安全组等一系列的动作。完成这个账号的初始化动作。
环境安装：
Terraform下载包位置(建议实际使用中，选择最后版本进行安装)：
Downloads | Terraform by HashiCorp
Downloads | Packer by HashiCorp
2 个账号，左边的账号是运维部门管理的账号，里面创建一台可以访问公网的 VM，建议使用 Amazon Linux 2。 右边账号是一个新账号，用于提供给新项目组或者新部门。具体方法：
选用 amaz2 的 AMI 进行系统准备，使用以下命令，进行环境依赖的安装。
$ sudo amazon-linux-extras install epel -y$ sudo yum install -y ansible$ sudo curl -o /tmp/terraform.zip https://releases.hashicorp.com/terraform/0.12.3/terraform_0.12.13_linux_amd64.zip$ sudo curl -o /tmp/packer.zip https://releases.hashicorp.com/packer/1.4.5/packer_1.4.5_linux_amd64.zip$ sudo unzip /tmp/terraform.zip -d /usr/local/bin$ sudo unzip /tmp/packer.zip -d /usr/local/bin权限配置方法：
添加 IAM 权限，我们先去在右侧是受管账号（Project or Team Account）下面去建立Assume Role，具体如图：IAM->Roles->Create Role
这里的 Role 关联了 Policy AdministratorAccess, 仅用于测试。但是实际生产上，要依据实际实际用户来进行权限划分。
这一步创建 Tags, 啥也没有填。你随意。
这里给 Role name 添加了一个名字 ”terraform-assume-role“
请查看这个 role 的 ARN，如下图，请记录好，后面我们在 Operation Account 里面会用到。
下面我们在左侧的运维账号（Operation Account）进行Policy/Role的创建。
首先创建Policy, IAM->Policies->Create Policy
{"Version": "2012-10-17","Statement": {"Effect": "Allow","Action": "sts:AssumeRole","Resource": ["arn:aws-cn:iam::123456780001:role/terraform-assume-role"]}
}
添加 Policy Name: terraform-assume-policy, 后面我们会关联到 terraform ec2 role 里面。
现在建立 Terraform EC2 Role, 请如下操作 IAM-> Roles->Create role， 选择 AWS Service -> EC2
搜索之前创建的 Policy, terraform, 勾选之前创建的 Policy
填写 role 的名字： terraform-ec2-role
选中之前创建的 Terraform EC2, 依次执行如图
更新为我们创建的Role: terraform-ec2-role
如上，我就完成基础环境的准备。
二、与 Packer 配合定制 AMI ，实现 ELB + Auto Scaling Group
围绕 Packer 的集成来实现，自动应用的发布方法。如下图：
如图示架构，我们需要 2 个步骤：
1、使用 Packer 制作 AMI
目录内的文件说明
| 文件 | 描述 |
|---|---|
| amaz2-stress-hk.json和amaz2-stress-nx.json | 这两个文件分别是香港和中国宁夏两个region内的AMI定义的描述文件，用于Packer创建文件 |
| base_install.sh | 这个文件是会在json文件里面定义和被调用的脚本。 |
| burnCPU.sh和index.php | 是压缩包burn.tgz里面的文件内容。 |
| burn.tgz | 是压缩包，里面包含了burnCPU.sh 和index.php |
由于默认 AWS China 不能访问 80、443，需要额外开通，所以我们以 HK 做实验。首先查看 Packer 制作镜像的描述文件：
{"builders": [{"type": "amazon-ebs","access_key": "Your-AWS-Account-AK","secret_key": "Your-AWS-Account-SK","region": "ap-east-1","source_ami": "ami-570c7726","instance_type": "c5.large","ssh_username": "ec2-user","ami_name": "amaz2-stress {{timestamp}} by packer"}],"provisioners": [{"type": "file","source": "burn.tgz","destination": "/tmp/burn.tgz"},{"type": "shell","script": "base_install.sh"}]
}主要说明如下：builders和provisioners是代表了packer打包ami的最终要的三部分。还有一个Post-Processors，当前环境没有用到。
builders:
| 代码 | 解释 |
|---|---|
| builders | 这部分是看后端接入的类型，比如：AWS/Azure/Google Cloud |
| Type | 这个amazon-ebs是在后端为AWS的其中一种打包方法，也是我们最常见的，启动一个实例，进行修改，然后关机，创建AMI。除此之外，还有另外4种模式：chroot/EBS Surrogate/EBS Volume/Instance 总计5种的修改方法。可以去读文档查看具体的参数: : Amazon EBS - Builders | Packer by HashiCorp |
| AK/SK | AWS AK/SK |
provisioners:
| 代码 | 解释 |
|---|---|
| Type | 主要类型有集成ansible/shell/file/Powershell/Chef.. 等总计10几种的模式。 |
| source | 本地文件存放位置 |
| destination | 文件上传目标路径 |
当前这个语法 file 代表将本地一个压缩文件传入到镜像的 /tmp/ 下面，Shell 这个部分代表，后续执行本地这个 base_install.sh 的脚本。
<!–特别说明：–>
<!–由于创建AMI属于一个临时行为，所以packer这个命令在简化环境下，需要default vpc, 否则会报错。而通常生产环境里面，defalut vpc已经删除。而用生产环境里面的vpc, 需要加入更多参数，如vpc id、subnet id。–>
我们需要进入当前 ami-packer 目录，执行命令打包：
[ec2-user@ip-172-31-22-159 ami-maker]$ packer build amaz2-stree-hk.json
amazon-ebs output will be in this color.==> amazon-ebs: Prevalidating AMI Name: amaz2-stress 1566530942 by packeramazon-ebs: Found Image ID: ami-570c7726
==> amazon-ebs: Creating temporary keypair: packer_5d5f5d7e-21ca-5f6c-62c1-14517a092628
==> amazon-ebs: Creating temporary security group for this instance: packer_5d5f5d83-63fc-bffe-8049-b616426fcc26
==> amazon-ebs: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instanceamazon-ebs: Adding tag: "Name": "Packer Builder"amazon-ebs: Instance ID: i-0eb31ca09c65dfbbb
==> amazon-ebs: Waiting for instance (i-0eb31ca09c65dfbbb) to become ready...
==> amazon-ebs: Using ssh communicator to connect: 18.163.6.72
==> amazon-ebs: Waiting for SSH to become available...
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Uploading burn.tgz => /tmp/burn.tgz
burn.tgz 890 B / 890 B [===========================================================================================================================================] 100.00% 0s
==> amazon-ebs: Provisioning with shell script: base_install.shamazon-ebs: Loaded plugins: extras_suggestions, langpacks, priorities, update-motdamazon-ebs: Resolving Dependencies
==> amazon-ebs: There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).amazon-ebs: --> Running transaction checkamazon-ebs: ---> Package httpd.x86_64 0:2.4.39-1.amzn2.0.1 will be installedamazon-ebs: --> Processing Dependency: httpd-tools = 2.4.39-1.amzn2.0.1 for package: httpd-2.4.39-1.amzn2.0.1.x86_64amazon-ebs: --> Processing Dependency: httpd-filesystem = 2.4.39-1.amzn2.0.1 for package: httpd-2.4.39-1.amzn2.0.1.x86_64amazon-ebs: --> Processing Dependency: system-logos-httpd for package: httpd-2.4.39-1.amzn2.0.1.x86_64amazon-ebs: --> Processing Dependency: mod_http2 for package: httpd-2.4.39-1.amzn2.0.1.x86_64amazon-ebs: --> Processing Dependency: httpd-filesystem for package: httpd-2.4.39-1.amzn2.0.1.x86_64amazon-ebs: --> Processing Dependency: /etc/mime.types for package: httpd-2.4.39-1.amzn2.0.1.x86_64amazon-ebs: --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.39-1.amzn2.0.1.x86_64amazon-ebs: --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.39-1.amzn2.0.1.x86_64amazon-ebs: ---> Package php.x86_64 0:5.4.16-45.amzn2.0.6 will be installedamazon-ebs: --> Processing Dependency: php-cli(x86-64) = 5.4.16-45.amzn2.0.6 for package: php-5.4.16-45.amzn2.0.6.x86_64amazon-ebs: --> Processing Dependency: php-common(x86-64) = 5.4.16-45.amzn2.0.6 for package: php-5.4.16-45.amzn2.0.6.x86_64amazon-ebs: --> Running transaction checkamazon-ebs: ---> Package apr.x86_64 0:1.6.3-5.amzn2.0.2 will be installedamazon-ebs: ---> Package apr-util.x86_64 0:1.6.1-5.amzn2.0.2 will be installedamazon-ebs: --> Processing Dependency: apr-util-bdb(x86-64) = 1.6.1-5.amzn2.0.2 for package: apr-util-1.6.1-5.amzn2.0.2.x86_64amazon-ebs: ---> Package generic-logos-httpd.noarch 0:18.0.0-4.amzn2 will be installedamazon-ebs: ---> Package httpd-filesystem.noarch 0:2.4.39-1.amzn2.0.1 will be installedamazon-ebs: ---> Package httpd-tools.x86_64 0:2.4.39-1.amzn2.0.1 will be installedamazon-ebs: ---> Package mailcap.noarch 0:2.1.41-2.amzn2 will be installedamazon-ebs: ---> Package mod_http2.x86_64 0:1.15.1-1.amzn2 will be installedamazon-ebs: ---> Package php-cli.x86_64 0:5.4.16-45.amzn2.0.6 will be installedamazon-ebs: ---> Package php-common.x86_64 0:5.4.16-45.amzn2.0.6 will be installedamazon-ebs: --> Processing Dependency: libzip.so.2()(64bit) for package: php-common-5.4.16-45.amzn2.0.6.x86_64amazon-ebs: --> Running transaction checkamazon-ebs: ---> Package apr-util-bdb.x86_64 0:1.6.1-5.amzn2.0.2 will be installedamazon-ebs: ---> Package libzip010-compat.x86_64 0:0.10.1-9.amzn2.0.5 will be installedamazon-ebs: --> Finished Dependency Resolutionamazon-ebs:amazon-ebs: Dependencies Resolvedamazon-ebs:amazon-ebs: ================================================================================amazon-ebs: Package Arch Version Repository Sizeamazon-ebs: ================================================================================amazon-ebs: Installing:amazon-ebs: httpd x86_64 2.4.39-1.amzn2.0.1 amzn2-core 1.3 Mamazon-ebs: php x86_64 5.4.16-45.amzn2.0.6 amzn2-core 1.4 Mamazon-ebs: Installing for dependencies:amazon-ebs: apr x86_64 1.6.3-5.amzn2.0.2 amzn2-core 118 kamazon-ebs: apr-util x86_64 1.6.1-5.amzn2.0.2 amzn2-core 99 kamazon-ebs: apr-util-bdb x86_64 1.6.1-5.amzn2.0.2 amzn2-core 19 kamazon-ebs: generic-logos-httpd noarch 18.0.0-4.amzn2 amzn2-core 19 kamazon-ebs: httpd-filesystem noarch 2.4.39-1.amzn2.0.1 amzn2-core 23 kamazon-ebs: httpd-tools x86_64 2.4.39-1.amzn2.0.1 amzn2-core 87 kamazon-ebs: libzip010-compat x86_64 0.10.1-9.amzn2.0.5 amzn2-core 30 kamazon-ebs: mailcap noarch 2.1.41-2.amzn2 amzn2-core 31 kamazon-ebs: mod_http2 x86_64 1.15.1-1.amzn2 amzn2-core 147 kamazon-ebs: php-cli x86_64 5.4.16-45.amzn2.0.6 amzn2-core 2.9 Mamazon-ebs: php-common x86_64 5.4.16-45.amzn2.0.6 amzn2-core 566 kamazon-ebs:amazon-ebs: Transaction Summaryamazon-ebs: ================================================================================amazon-ebs: Install 2 Packages (+11 Dependent packages)amazon-ebs:amazon-ebs: Total download size: 6.7 Mamazon-ebs: Installed size: 22 Mamazon-ebs: Downloading packages:amazon-ebs: --------------------------------------------------------------------------------amazon-ebs: Total 18 MB/s | 6.7 MB 00:00amazon-ebs: Running transaction checkamazon-ebs: Running transaction testamazon-ebs: Transaction test succeededamazon-ebs: Running transaction
==> amazon-ebs: ** Found 2 pre-existing rpmdb problem(s), 'yum check' output follows:
==> amazon-ebs: 32:bind-license-9.9.4-74.amzn2.1.2.noarch is a duplicate with 32:bind-license-9.9.4-73.amzn2.1.2.noarch
==> amazon-ebs: python-libs-2.7.16-2.amzn2.0.1.x86_64 is a duplicate with python-libs-2.7.14-58.amzn2.0.4.x86_64amazon-ebs: Installing : apr-1.6.3-5.amzn2.0.2.x86_64 1/13amazon-ebs: Installing : apr-util-bdb-1.6.1-5.amzn2.0.2.x86_64 2/13amazon-ebs: Installing : apr-util-1.6.1-5.amzn2.0.2.x86_64 3/13amazon-ebs: Installing : httpd-tools-2.4.39-1.amzn2.0.1.x86_64 4/13amazon-ebs: Installing : generic-logos-httpd-18.0.0-4.amzn2.noarch 5/13amazon-ebs: Installing : mailcap-2.1.41-2.amzn2.noarch 6/13amazon-ebs: Installing : httpd-filesystem-2.4.39-1.amzn2.0.1.noarch 7/13amazon-ebs: Installing : mod_http2-1.15.1-1.amzn2.x86_64 8/13amazon-ebs: Installing : httpd-2.4.39-1.amzn2.0.1.x86_64 9/13amazon-ebs: Installing : libzip010-compat-0.10.1-9.amzn2.0.5.x86_64 10/13amazon-ebs: Installing : php-common-5.4.16-45.amzn2.0.6.x86_64 11/13amazon-ebs: Installing : php-cli-5.4.16-45.amzn2.0.6.x86_64 12/13amazon-ebs: Installing : php-5.4.16-45.amzn2.0.6.x86_64 13/13amazon-ebs: Verifying : apr-util-1.6.1-5.amzn2.0.2.x86_64 1/13amazon-ebs: Verifying : libzip010-compat-0.10.1-9.amzn2.0.5.x86_64 2/13amazon-ebs: Verifying : php-cli-5.4.16-45.amzn2.0.6.x86_64 3/13amazon-ebs: Verifying : apr-util-bdb-1.6.1-5.amzn2.0.2.x86_64 4/13amazon-ebs: Verifying : httpd-tools-2.4.39-1.amzn2.0.1.x86_64 5/13amazon-ebs: Verifying : httpd-2.4.39-1.amzn2.0.1.x86_64 6/13amazon-ebs: Verifying : httpd-filesystem-2.4.39-1.amzn2.0.1.noarch 7/13amazon-ebs: Verifying : php-5.4.16-45.amzn2.0.6.x86_64 8/13amazon-ebs: Verifying : mod_http2-1.15.1-1.amzn2.x86_64 9/13amazon-ebs: Verifying : apr-1.6.3-5.amzn2.0.2.x86_64 10/13amazon-ebs: Verifying : mailcap-2.1.41-2.amzn2.noarch 11/13amazon-ebs: Verifying : generic-logos-httpd-18.0.0-4.amzn2.noarch 12/13amazon-ebs: Verifying : php-common-5.4.16-45.amzn2.0.6.x86_64 13/13amazon-ebs:amazon-ebs: Installed:amazon-ebs: httpd.x86_64 0:2.4.39-1.amzn2.0.1 php.x86_64 0:5.4.16-45.amzn2.0.6amazon-ebs:amazon-ebs: Dependency Installed:amazon-ebs: apr.x86_64 0:1.6.3-5.amzn2.0.2amazon-ebs: apr-util.x86_64 0:1.6.1-5.amzn2.0.2amazon-ebs: apr-util-bdb.x86_64 0:1.6.1-5.amzn2.0.2amazon-ebs: generic-logos-httpd.noarch 0:18.0.0-4.amzn2amazon-ebs: httpd-filesystem.noarch 0:2.4.39-1.amzn2.0.1amazon-ebs: httpd-tools.x86_64 0:2.4.39-1.amzn2.0.1amazon-ebs: libzip010-compat.x86_64 0:0.10.1-9.amzn2.0.5amazon-ebs: mailcap.noarch 0:2.1.41-2.amzn2amazon-ebs: mod_http2.x86_64 0:1.15.1-1.amzn2amazon-ebs: php-cli.x86_64 0:5.4.16-45.amzn2.0.6amazon-ebs: php-common.x86_64 0:5.4.16-45.amzn2.0.6amazon-ebs:amazon-ebs: Complete!amazon-ebs: Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
==> amazon-ebs: Existing lock /var/run/yum.pid: another copy is running as pid 2748.
==> amazon-ebs: Another app is currently holding the yum lock; waiting for it to exit...
==> amazon-ebs: The other application is: yum
==> amazon-ebs: Memory : 42 M RSS (332 MB VSZ)
==> amazon-ebs: Started: Fri Aug 23 03:29:45 2019 - 00:01 ago
==> amazon-ebs: State : Sleeping, pid: 2748
==> amazon-ebs: Another app is currently holding the yum lock; waiting for it to exit...
==> amazon-ebs: The other application is: yum
==> amazon-ebs: Memory : 101 M RSS (392 MB VSZ)
==> amazon-ebs: Started: Fri Aug 23 03:29:45 2019 - 00:03 ago
==> amazon-ebs: State : Running, pid: 2748amazon-ebs: Cleaning repos: amzn2-core amzn2extra-docker amzn2extra-epelamazon-ebs: 13 metadata files removedamazon-ebs: 6 sqlite files removedamazon-ebs: 0 metadata files removedamazon-ebs: Loaded plugins: extras_suggestions, langpacks, priorities, update-motdamazon-ebs: Resolving Dependencies
==> amazon-ebs: There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).amazon-ebs: --> Running transaction checkamazon-ebs: ---> Package epel-release.noarch 0:7-11 will be installedamazon-ebs: --> Finished Dependency Resolutionamazon-ebs:amazon-ebs: Dependencies Resolvedamazon-ebs:amazon-ebs: ================================================================================amazon-ebs: Package Arch Version Repository Sizeamazon-ebs: ================================================================================amazon-ebs: Installing:amazon-ebs: epel-release noarch 7-11 amzn2extra-epel 15 kamazon-ebs:amazon-ebs: Transaction Summaryamazon-ebs: ================================================================================amazon-ebs: Install 1 Packageamazon-ebs:amazon-ebs: Total download size: 15 kamazon-ebs: Installed size: 24 kamazon-ebs: Downloading packages:amazon-ebs: Running transaction checkamazon-ebs: Running transaction testamazon-ebs: Transaction test succeededamazon-ebs: Running transactionamazon-ebs: Installing : epel-release-7-11.noarch 1/1amazon-ebs: Verifying : epel-release-7-11.noarch 1/1amazon-ebs:amazon-ebs: Installed:amazon-ebs: epel-release.noarch 0:7-11amazon-ebs:amazon-ebs: Complete!amazon-ebs: Installing epel-releaseamazon-ebs: 0 ansible2 available [ =2.4.2 =2.4.6 ]amazon-ebs: 2 httpd_modules available [ =1.0 ]amazon-ebs: 3 memcached1.5 available [ =1.5.1 =1.5.16 ]amazon-ebs: 4 nginx1.12 available [ =1.12.2 ]amazon-ebs: 5 postgresql9.6 available [ =9.6.6 =9.6.8 ]amazon-ebs: 6 postgresql10 available [ =10 ]amazon-ebs: 8 redis4.0 available [ =4.0.5 =4.0.10 ]amazon-ebs: 9 R3.4 available [ =3.4.3 ]amazon-ebs: 10 rust1 available \amazon-ebs: [ =1.22.1 =1.26.0 =1.26.1 =1.27.2 =1.31.0 ]amazon-ebs: 11 vim available [ =8.0 ]amazon-ebs: 13 ruby2.4 available [ =2.4.2 =2.4.4 ]amazon-ebs: 15 php7.2 available \amazon-ebs: [ =7.2.0 =7.2.4 =7.2.5 =7.2.8 =7.2.11 =7.2.13 =7.2.14amazon-ebs: =7.2.16 =7.2.17 =7.2.19 ]amazon-ebs: 16 php7.1 available \amazon-ebs: [ =7.1.22 =7.1.25 =7.1.27 =7.1.28 =7.1.30 ]amazon-ebs: 17 lamp-mariadb10.2-php7.2 available \amazon-ebs: [ =10.2.10_7.2.0 =10.2.10_7.2.4 =10.2.10_7.2.5amazon-ebs: =10.2.10_7.2.8 =10.2.10_7.2.11 =10.2.10_7.2.13amazon-ebs: =10.2.10_7.2.14 =10.2.10_7.2.16 =10.2.10_7.2.17amazon-ebs: =10.2.10_7.2.19 ]amazon-ebs: 18 libreoffice available [ =5.0.6.2_15 =5.3.6.1 ]amazon-ebs: 19 gimp available [ =2.8.22 ]amazon-ebs: 20 docker=latest enabled \amazon-ebs: [ =17.12.1 =18.03.1 =18.06.1 ]amazon-ebs: 21 mate-desktop1.x available [ =1.19.0 =1.20.0 ]amazon-ebs: 22 GraphicsMagick1.3 available [ =1.3.29 =1.3.32 ]amazon-ebs: 23 tomcat8.5 available \amazon-ebs: [ =8.5.31 =8.5.32 =8.5.38 =8.5.40 =8.5.42 ]amazon-ebs: 24 epel=latest enabled [ =7.11 ]amazon-ebs: 25 testing available [ =1.0 ]amazon-ebs: 26 ecs available [ =stable ]amazon-ebs: 27 corretto8 available \amazon-ebs: [ =1.8.0_192 =1.8.0_202 =1.8.0_212 =1.8.0_222 ]amazon-ebs: 28 firecracker available [ =0.11 ]amazon-ebs: 29 golang1.11 available [ =1.11.3 =1.11.11 ]amazon-ebs: 30 squid4 available [ =4 ]amazon-ebs: 31 php7.3 available \amazon-ebs: [ =7.3.2 =7.3.3 =7.3.4 =7.3.6 ]amazon-ebs: 32 lustre2.10 available [ =2.10.5 ]amazon-ebs: 33 java-openjdk11 available [ =11 ]amazon-ebs: 34 lynis available [ =stable ]amazon-ebs: 35 kernel-ng available [ =stable ]amazon-ebs: 36 BCC available [ =0.x ]amazon-ebs: 37 mono available [ =5.x ]amazon-ebs: Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
==> amazon-ebs: https://mirrors.sonic.net/epel/7/x86_64/repodata/repomd.xml: [Errno -1] repomd.xml does not match metalink for epel
==> amazon-ebs: Trying other mirror.
==> amazon-ebs: https://ewr.edge.kernel.org/fedora-buffet/epel/7/x86_64/repodata/d748a548825eb7ebeca7c8cb8e98387afe904e7bc00dab7c9c35795379cc183d-primary.sqlite.bz2: [Errno 14] HTTPS Error 404 - Not Found
==> amazon-ebs: Trying other mirror.amazon-ebs: 191 packages excluded due to repository priority protectionsamazon-ebs: Resolving Dependencies
==> amazon-ebs: There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).amazon-ebs: --> Running transaction checkamazon-ebs: ---> Package stress.x86_64 0:1.0.4-16.el7 will be installedamazon-ebs: --> Finished Dependency Resolutionamazon-ebs:amazon-ebs: Dependencies Resolvedamazon-ebs:amazon-ebs: ================================================================================amazon-ebs: Package Arch Version Repository Sizeamazon-ebs: ================================================================================amazon-ebs: Installing:amazon-ebs: stress x86_64 1.0.4-16.el7 epel 39 kamazon-ebs:amazon-ebs: Transaction Summaryamazon-ebs: ================================================================================amazon-ebs: Install 1 Packageamazon-ebs:amazon-ebs: Total download size: 39 kamazon-ebs: Installed size: 94 kamazon-ebs: Downloading packages:
==> amazon-ebs: warning: /var/cache/yum/x86_64/2/epel/packages/stress-1.0.4-16.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEYamazon-ebs: Public key for stress-1.0.4-16.el7.x86_64.rpm is not installedamazon-ebs: Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
==> amazon-ebs: Importing GPG key 0x352C64E5:
==> amazon-ebs: Userid : "Fedora EPEL (7) <epel@fedoraproject.org>"
==> amazon-ebs: Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5
==> amazon-ebs: Package : epel-release-7-11.noarch (@amzn2extra-epel)
==> amazon-ebs: From : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7amazon-ebs: Running transaction checkamazon-ebs: Running transaction testamazon-ebs: Transaction test succeededamazon-ebs: Running transactionamazon-ebs: Installing : stress-1.0.4-16.el7.x86_64 1/1amazon-ebs: Verifying : stress-1.0.4-16.el7.x86_64 1/1amazon-ebs:amazon-ebs: Installed:amazon-ebs: stress.x86_64 0:1.0.4-16.el7amazon-ebs:amazon-ebs: Complete!
==> amazon-ebs: Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.amazon-ebs: burnCPU.shamazon-ebs: index.php
==> amazon-ebs: Stopping the source instance...amazon-ebs: Stopping instance
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating AMI amaz2-stress 1566530942 by packer from instance i-0eb31ca09c65dfbbbamazon-ebs: AMI: ami-04262661376a925a7
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
ap-east-1: ami-04262661376a925a7请看最后一行： ami-04262661376a925a7，后面我们将这个作为变量，提供后面的 terraform 执行，LB+ASG 的创建。
2: 创建LB + ASG
查看 main.tf 文件, 修改参数值。
##
provider "aws" {region = "ap-east-1" \\ 替换成您实际操作的Regionaccess_key = "Your-AWS-AK" \\ 替换您自己账号的AKsecret_key = "Your-AWS-SK" \\ 替换成你自己账号的SK
}
##
variable "asg_instances_id" {default = "From_Packer_create_AMI_ID" \\ 替换成你上面从Packer获取到的AMI ID
}
###
data "aws_vpc" "default" {default = true
}
data "aws_subnet_ids" "all" {vpc_id = data.aws_vpc.default.id
}
data "aws_security_group" "default" {vpc_id = data.aws_vpc.default.idname = "default"
}
data "aws_ami" "amazon_linux" {most_recent = trueowners = ["amazon"]filter {name = "name"values = ["amzn-ami-hvm-*-x86_64-gp2",]}filter {name = "owner-alias"values = ["amazon",]}
}####
module "elb" {source = "../terraform-modules/terraform-aws-elb/"name = "elb-example"subnets = data.aws_subnet_ids.all.idssecurity_groups = [data.aws_security_group.default.id]internal = falselistener = [{instance_port = "80"instance_protocol = "HTTP"lb_port = "80"lb_protocol = "HTTP"},]health_check = {target = "HTTP:80/"interval = 30healthy_threshold = 2unhealthy_threshold = 2timeout = 5}tags = {Owner = "user"Environment = "dev"}}####module "example_asg" {source = "../my-modules/terraform-aws-autoscaling/"name = "example-with-lb-asg"# Launch configuration## launch_configuration = "my-existing-launch-configuration" # Use the existing launch configuration# create_lc = false # disables creation of launch configurationlc_name = "example-lc"image_id = "${var.asg_instances_id}"instance_type = "c5.large"security_groups = [data.aws_security_group.default.id]load_balancers = [module.elb.this_elb_id]key_name = "hongkong"# Auto-scaling policies and CloudWatch metric alarmsautoscaling_policies_enabled = "true"cpu_utilization_high_threshold_percent = "70"cpu_utilization_low_threshold_percent = "20"root_block_device = [{volume_size = "10"volume_type = "gp2"},]# Auto scaling groupasg_name = "example-asg"vpc_zone_identifier = data.aws_subnet_ids.all.idshealth_check_type = "EC2"min_size = 1max_size = 3desired_capacity = 1wait_for_capacity_timeout = 0tags = [{key = "Environment"value = "dev"propagate_at_launch = true},{key = "Project"value = "megasecret"propagate_at_launch = true},]
}其他参数按照我们上一个LAB的方法获取参数含义，并自行调整。执行命令：
$ terraform init$ terrrform plan$ terraform apply -auto-approve检验结果：
执行成功后，默认只会在 ASG 创建 1 台主机，找到 LB 的 DNS Name，进行访问，例如： https://elb-example-291281982.ap-east-1.elb.amazonaws.com
同时结合 AWS Console 观察 EC2 的 create 和 terminal，也可以通过 ASG 查看：
三、与 Ansible 集成的主要方法，实现中国区自己的 “EFS”
下面介绍与 Ansible 集成的方法，实现一套 Pacemaker + DRBD + NFS 来实现一个跨 AZ 的集群部署方案。所以在这种框架下，我们要做到以下两件事：
1：通过 Terraform 创建目标账号下所需要的 Infra 层面的所有资源.
2：通过 Terraform 调用 Ansible Playbook 实现对所有目标主机的配置。
AWS 上 HA 实现方案：
VIP：在 AWS 由于 Subnet 无法跨 AZ，所以实现的方法就存在两种。Overlay 的假 IP，就是一个存粹的路由表指向，将这个不真实存在于 VPC 内的 IP 指向一个 ENI ; 还有一种方式就是 EIP，也就是拥有固定外网 IP 的 Public IP.
IAM Role 的权限：这个权限定义通常是集群主机内所有主机的开关机权限。然后以 ec2 role 的方式赋予。之所以需要这个权限，是因为系统内的 HA agent 会通过 AWS CLI 进行状态的 monitor 和 Switch 动作。所以需要这样的权限，在发生 Failover 后，能够正确处置后续的资源位置。
整体上HA的架构：
1：新增的 EBS，通过 DRBD 的方式进行底层数据的同步。
2：基于 DRBD 所创建的文件系统 PV-VG-LV, 做到提供给 NFS 作为基本的文件系统，用于文件的分享。
3：最上层的 NFS 服务，构建与文件系统上，提供基础服务。
4：VIP 随着 DRBD 的 Master 绑定进行切换。
而关于我们上面提到的 EIP 的架构图如下：
集成 Ansible
1：与 Ansible 的集成生成 inventory，在生成所需的 ec2 主机后，构建 inventory，请看如下代码：
[ec2-user@ip-172-31-22-159 withAnsible-55523423422-nx-centos]$ cat upload.tfdata "template_file" "inventory" {template = "${file("${path.module}/templates/hosts.tpl")}"vars = {dns01_hostname = "nfs01.liujia.com"dns02_hostname = "nfs02.liujia.com"dns01_ip = tolist(module.ec2-nfs01.private_ip)[0]dns02_ip = tolist(module.ec2-nfs02.private_ip)[0]key_path = "~/.ssh/id_rsa"}
}resource "local_file" "save_inventory" {content = "${data.template_file.inventory.rendered}"filename = "./ansible-playbook/hosts"
}[ec2-user@ip-172-31-22-159 withAnsible-55523423422-nx-centos]$ cat templates/hosts.tpl
[master]
${nfs01_hostname} ansible_ssh_host=${nfs01_ip}[slave]
${nfs02_hostname} ansible_ssh_host=${nfs02_ip}[all:vars]
ansible_ssh_private_key_file = ${key_path}
在这里通过对模板 inventory 的定义，在通过 data template_file 的参数传递，最终渲染生成 local_file 到 ansible 的 playbook 里面生成 inventory，用于最终 playbook 的生成。
2：完成 Ansible 的上传和 playbook 的执行
[ec2-user@ip-172-31-22-159 withAnsible-55523423422-nx-centos]$ cat upload.tf
...
resource "null_resource" "utility" {connection {timeout = "5m"type = "ssh"user = "ansible"host = tolist(module.ec2-utility.public_ip)[0]private_key = "${file("mykey.pem")}"}provisioner "local-exec" {command = "tar zcvf ./ansible-playbook.tgz ./ansible-playbook"}provisioner "file" {source = "./ansible-playbook.tgz"destination = "/tmp/ansible-playbook.tgz"}provisioner "remote-exec" {inline = ["tar zxvf /tmp/ansible-playbook.tgz -C /tmp","cd /tmp/ansible-playbook","ansible-playbook -i ./hosts site.yml",]}depends_on = [local_file.save_inventory]
}在这里通过对模板 inventory 的定义，在通过 data template_file 的参数传递，最终渲染生成 local_file 到 ansible 的 playbook 里面生成 inventory，用于最终 playbook 的生成。
四、典型场景的应用和常见问题
场景一： 实现 Openshift 3 在 AWS 上的定制快速部署
以下就是典型 openshift 在 AWS 上部署架构图。其中 ansible config server，就是我们在上面讲到的类似方法，在这台主机上已经操作主机，从而进行整个集群环境的配置和管理。
同理，在 AWS 上部署原生的 Kubernates 有很多工具，例如：kops，可以在 AWS 快速构建起一套环境。但是在插件调整，系统配置和调优方面，基本上完全没有入口可以让你自己精细化的调整整个集群的架构。而通过我们 terraform + ansible 的组合，可以从底层资源到系统 OS 的所有层面可控。
场景二： 通用场景说明
在下面的模式下，我们开通过 Terraform 走绿线创建右侧受管账号下的所有资源，并且实现自动化。当然包括计算资源。然后我们通过推出的一台 Ansible 配置管理主机作为部署服务器，进而推出目标账号下，相应的系统。只要有标准的配置方法，我们都可以实现到自动化的创建。至于究竟是什么监控平台、测试平台、容器平台，都可以。
常见问题总结：
问题 1：为什么我执行 terrafom init 执行这么慢？
回答 1：terraform 基本分拆了所有的调用模块，terraform 默认安装里面只包括主要调用。在 terraform 执行 init 后，依据读取的 tf 文件，再去下载对应的 provider 模块或者其他相关模块。通常这些文件都不小，而且通常都存在工作目录的 .terraform 下，如果你确定是类似工作模块，建议你直接复制这个目录到新的项目目录下。这样比较节省时间。
问题 2：如何 debug Terraform ？
回答 2：在执行命令前加入 TF_LOG=TRACE
[ec2-user@ip-172-31-22-159 log]$ TF_LOG=TRACE terraform init
2019/07/18 01:33:09 [INFO] Terraform version: 0.12.4
2019/07/18 01:33:09 [INFO] Go runtime version: go1.12.4
2019/07/18 01:33:09 [INFO] CLI args: []string{"/usr/local/bin/terraform", "init"}
2019/07/18 01:33:09 [DEBUG] Attempting to open CLI config file: /home/ec2-user/.terraformrc
2019/07/18 01:33:09 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/07/18 01:33:09 [INFO] CLI command args: []string{"init"}
Terraform initialized in an empty directory!The directory has no Terraform configuration files. You may begin working
with Terraform immediately by creating Terraform configuration files.Terraform 学习总结（7）—— 基于 AWS 云平台上的 Terraform 实战相关推荐
- Terraform 学习总结（6）—— 基于阿里云平台上的 Terraform 实战
Terraform 是什么 Terraform(https://www.terraform.io/)是 HashiCorp 旗下的一款开源(Go 语言开发)的 DevOps 基础架构资源管理运维工具, ...
- 亚马逊AWS在线系列讲座——基于AWS云平台的高可用应用设计
设计高可用的应用是架构师的一个重要目标,可是基于云计算平台设计高可用应用与基于传统平台的设计有很多不同.云计算在给架构师带来了很多新的设计挑战的时候,也给带来了很多新的设计理念和可用的服务.怎样在设计 ...
- AWS云平台的服务概览
当我们向别人解释AWS云平台所包含的那些服务的时候,许多人对AWS服务种类的丰富程度都表示惊讶.对于部分听说过AWS的人来说,他们知道AWS云平台的功能主要限于EC2(弹性计算云).S3(简单存储服务 ...
- 数据分析与可视化 --aws云平台
数据分析与可视化 项目一:搭建AWS数据分析开发环境 1.安装Notebook开发环境 Step1:下载支持Notebook 的开发环境 远程登录到Amazon EC2实例后 在命令行中输入并执行: ...
- ZStack实践汇 | 基于ZStack云平台部署FortiGate
2019年国际知名信息安全峰会RSA Conference在美国旧金山举行,在所有演讲主题中,云安全超过网络安全和数据安全,成为热门关键词第一.ZStack实践汇这次带来的是<基于ZStack云 ...
- 基于阿里云平台的短信验证码服务API的使用
基于阿里云平台的短信验证码服务API的使用 第一步:登录阿里云平台 第二步:申请国内文本短信签名 如果是个人作业项目(如作者的签名),可以直接申请测试和学习用的测试签名,该签名的缺点是必须绑定测试手机 ...
- aws云平台架构师 收入_收入15万美元的云架构师可能被低薪
aws云平台架构师 收入 根据该消息来源, "云架构师的年收入在14万至15万美元之间". 我支付的费用越来越少,具体取决于建筑师的住所. 但是,具有良好的经验和行之有效的成功经验 ...
- 基于开放式云平台的开源在线评测系统设计与实现
基于开放式云平台的开源在线评测系统设计与实现 张浩斌 ZHANG Hao-Bin 浙江传媒学院 新媒体学院 浙江省 杭州市 310018 Department of New Media, Zhejia ...
- 基于机智云平台的泵站智能巡检系统
本文是由开发者设计开发的一款泵站智能巡检系统.该系统改变了常规的专人值守的工作模式,实现泵站运行状态的远程智能巡检工作,达到自动监测.故障报警.提前预测等功能. 随着我国水利事业不断发展,泵站在水利调 ...
最新文章
- KVM中virtio之vring（八）
- 连续七天熬夜3D建模师终于出手，让老板增加薪资待遇，3D建模初学者的4个技巧
- python解析器是什么_如何用python写一个简单的词法分析器
- 未能加载文件或程序集“Enyim.Caching”或它的某一个依赖项。未能验证强名称签名...
- 怎么打包图片_房产也能批发！澳村庄40栋房打包出售，总价$175万，买了变村长！...
- Eplan破解文件名称说明：
- AVR单片机实现modbus通讯协议
- 机器学习实战之信用卡欺诈案列
- 运维常用工具命令/知识总结
- Cell：代谢组学肠道微生物群介导生酮饮食的抗癫痫作用
- web期末作业设计网页——_传统节日--端午节(9页)主题节日网页
- 神经网络种类及应用领域,常用的神经网络有哪些
- 项目管理思维是什么？
- Python小工具：将对象转换为不可变类型并计算其哈希值
- 利用FME实现DLG数据无损转CAD地形图，实现CAD地物符号完美还原的解决方案，解决CAD地图符号难以还原的痛点问题，gdb转CAD，mdb转CAD，shp转CAD，shapefile转CAD
- 赋能医疗行业数字化转型，蓝网影像云平台装上鲲鹏云引擎
- Javascript 香港身份证号校验
- android 电视台列表,Android电视：获取频道列表
- ThinkPHP6利用phpoffice/phpexcel导入表格数据
- 收藏！Git命令大全