openstack

私有云搭建

实验环境:rhel7.2

禁用或移除所有自动更新的服务,因为它们会影响到您的 OpenStack 环境。
systemctl stop NetworkManager
systemctl disable NetworkManager

独立节点的配置

1.添加网卡
2.更改配置文件

DEVICE="ens9"
ONBOOT=yes
BOOTPROTO=none

3.重启服务

vim /boot/grub2/grub.cfg
在内核引导文件中添加这个net.ifnames=0(99行)就可以添加网卡之后默认显示的是eth*

一、网络时间的同步;

物理机同步其他时间,虚拟机同步物理机时间
[root@foundation39 images]# vim /etc/chrony.conf

  7 server 172.25.254.251 iburst24 allow 172.25/16

[root@foundation39 images]# chronyc sources -v
210 Number of sources = 1

  .-- Source mode  '^' = server, '=' = peer, '#' = local clock./ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| /   '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
||                                                 .- xxxx [ yyyy ] +/- zzzz
||      Reachability register (octal) -.           |  xxxx = adjusted offset,
||      Log2(Polling interval) --.      |          |  yyyy = measured offset,
||                                \     |          |  zzzz = estimated error.
||                                 |    |           \
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^* 172.25.254.251                3   6     7     1    -81us[ +27.0s] +/-   30ms

虚拟机同步物理机时间:
[root@open1 yum.repos.d]# vim /etc/chrony.conf

  3 server 172.25.39.250 iburst

[root@open1 yum.repos.d]# systemctl restart chronyd
[root@open1 yum.repos.d]# chronyc sources -v——同步成功
210 Number of sources = 1

  .-- Source mode  '^' = server, '=' = peer, '#' = local clock./ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| /   '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
||                                                 .- xxxx [ yyyy ] +/- zzzz
||      Reachability register (octal) -.           |  xxxx = adjusted offset,
||      Log2(Polling interval) --.      |          |  yyyy = measured offset,
||                                \     |          |  zzzz = estimated error.
||                                 |    |           \
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^* 172.25.39.250                 4   6    17    32  -1473ns[  -27us] +/-   32ms

二、yum源的配置
(将openstack的包放到yum仓库中【/var/www/html】)
[root@open1 yum.repos.d]# vim yum.repo
事先准备好自己所要的包,放到http的默认发布目录下:

[openstack]
name=mitaka
baseurl=http://172.25.39.250/mitaka
gpgcheck=0
enabled=1

三、openstack环境的部署

控制节点的部署:
1)openstack包的安装
所有节点上执行这些程序

yum upgrade—在主机上升级包(如果更新了一个新内核,重启主机来使用新内核)
yum install python-openstackclient -y—安装 OpenStack 客户端

2)SQL数据库

大多数 OpenStack 服务使用 SQL 数据库来存储信息。 典型地,数据库运行在控制节点上。

yum install mariadb mariadb-server python2-PyMySQL—安装软件包

创建并编辑 /etc/my.cnf.d/openstack.cnf,然后完成如下动作:

在[mysqld]部分,设置 bind-address值为控制节点的管理网络IP地址以使得其它节点可以通过管理网络访问数据库:
在[mysqld]部分,设置如下键值来启用一起有用的选项和 UTF-8 字符集:

[root@open1 yum.repos.d]# vim /etc/my.cnf.d/openstack.cnf

[mysqld]
bind-address = 172.25.39.1
default-storage-engine = innodb-指定存储引擎
innodb_file_per_table----独立表空间
max_connections = 4096----最大连接数
collation-server = utf8_general_ci
character-set-server = utf8

启动数据库服务,并将其配置为开机自启
[root@open1 yum.repos.d]# systemctl enable mariadb.service
Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service.
[root@open1 yum.repos.d]# systemctl start mariadb.service

初始化:
[root@open1 yum.repos.d]# mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDBSERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!In order to log into MariaDB to secure it, we'll need the current
password for the root user.  If you've just installed MariaDB, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.Enter current password for root (enter for none):
OK, successfully used password, moving on...Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.Set root password? [Y/n]
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..... Success!By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.Remove anonymous users? [Y/n] ... Success!Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.Disallow root login remotely? [Y/n] ... Success!By default, MariaDB comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.Remove test database and access to it? [Y/n] - Dropping test database...... Success!- Removing privileges on test database...... Success!Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.Reload privilege tables now? [Y/n] ... Success!Cleaning up...All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.Thanks for using MariaDB!

[root@open1 yum.repos.d]# mysql -pwestos

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 10
Server version: 10.1.20-MariaDB MariaDB ServerCopyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.MariaDB [(none)]>  CREATE DATABASE keystone;(认证服务、先决条件)
Query OK, 1 row affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' \->   IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%'    IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> CREATE DATABASE nova_api;(计算服务、先决条件)
Query OK, 1 row affected (0.00 sec)MariaDB [(none)]> CREATE DATABASE nova;
Query OK, 1 row affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'localhost' \->   IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'%'    IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%'    IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost'    IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> CREATE DATABASE neutron;(Networking 服务、安装和配置计算节点)
Query OK, 1 row affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' \->   IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%'    IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> CREATE DATABASE glance;(Networking 服务、先决条件)
Query OK, 1 row affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' \->   IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' \->   IDENTIFIED BY 'westos';
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| keystone           |
| mysql              |
| neutron            |
| nova               |
| nova_api           |
| glance             |
| performance_schema |
+--------------------+
7 rows in set (0.00 sec)MariaDB [(none)]> 

3)消息队列
OpenStack 使用 message queue 协调操作和各服务的状态信息。消息队列服务一般运行在控制节点上。OpenStack支持好几种消息队列服务包括 RabbitMQ, Qpid, 和 ZeroMQ。

yum install rabbitmq-server
systemctl enable rabbitmq-server.service
systemctl start rabbitmq-server.service

添加 openstack 用户:
rabbitmqctl add_user openstack westos
给openstack用户配置写和读权限:
rabbitmqctl set_permissions openstack ".*" ".*" ".*"
[root@open1 network-scripts]# rabbitmq-plugins enable rabbitmq_management--------管理工具的开启
The following plugins have been enabled:mochiwebwebmachinerabbitmq_web_dispatchamqp_clientrabbitmq_management_agentrabbitmq_managementApplying plugin configuration to rabbit@open1... started 6 plugins
根据上述显示,与之依赖的也会被打开

netstat -antlp(发现15672打开)
tcp 0 0 0.0.0.0:15672 0.0.0.0:* LISTEN 2293/beam

这个时候在浏览器,会出现RabbitMQ界面
http://172.25.39.1:15672/
登陆:用户、密码均为guest

[root@open1 network-scripts]# cat /etc/hosts—一定要解析
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.25.39.1 open1
172.25.39.2 open2

用户的查看: rabbitmqctl list_users

[root@open1 network-scripts]# rabbitmqctl list_users
Listing users ...
openstack   []
guest   [administrator]

openstack的权限的查看

[root@open1 network-scripts]# rabbitmqctl list_user_permissions openstack
Listing permissions for user "openstack" ...
/   .*  .*  .*

用户的认证

[root@open1 network-scripts]# rabbitmqctl authenticate_user openstack openstack
Authenticating user "openstack" ...
Error: failed to authenticate user "openstack"
[root@open1 network-scripts]# rabbitmqctl authenticate_user openstack westos
Authenticating user "openstack" ...
Success

4)memcached
认证服务认证缓存使用Memcached缓存令牌。缓存服务memecached运行在控制节点。
yum install memcached python-memcached
systemctl enable memcached.service
systemctl start memcached.service

[root@open1 network-scripts]# netstat -antlp
tcp 0 0 127.0.0.1:11211 0.0.0.0:* LISTEN 3847/memcached

监听地址的更改:vim /etc/sysconfig/memcached 1 PORT="11211"2 USER="memcached"3 MAXCONN="1024"4 CACHESIZE="64"5 OPTIONS="-l 127.0.0.1,::1,172.25.39.1"

这个时候就会发现监听了设置ip的主机

[root@open1 network-scripts]# netstat -antlp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:25672           0.0.0.0:*               LISTEN      2293/beam
tcp        0      0 172.25.39.1:3306        0.0.0.0:*               LISTEN      1024/mysqld
tcp        0      0 172.25.39.1:11211       0.0.0.0:*               LISTEN      3890/memcached
tcp        0      0 127.0.0.1:11211         0.0.0.0:*           LISTEN      3890/memcached 

四、认证服务(提供用户认证和服务目录,服务目录记录了所有云计算的api端口)

OpenStack:term:Identity service为认证管理,授权管理和服务目录服务管理提供单点整合。

身份认证服务提供服务的目录和他们的位置。每个你添加到OpenStack环境中的服务在目录中需要一个 service 实体和一些 API endpoints 。

安全并配置组件

[root@open1 network-scripts]# yum install openstack-keystone httpd mod_wsgi -y

[root@open1 network-scripts]# openssl rand -hex 10---生成admin token
e185f1e79b487c6eb9aa(m版的api默认放到memcache,可以定义过期自动删除的token)

[root@open1 conf.d]# vim /etc/keystone/keystone.conf

在[DEFAULT]部分,定义初始管理令牌的值:3 admin_token = e185f1e79b487c6eb9aa在[database]部分,配置数据库访问528 [database]529 connection = mysql+pymysql://keystone:westos@172.25.39.1/keys     tone在[token]部分,配置Fernet UUID令牌的提供者。
1986 [token]
1987 provider = fernet初始化身份认证服务的数据库(同步数据库):
[root@open1 conf.d]# su -s /bin/sh -c "keystone-manage db_sync" keystone
(指定shell,用keystone的身份执行指令:keystone-manage db_sync)

验证:

[root@open1 conf.d]# mysql -pwestos
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.1.20-MariaDB MariaDB ServerCopyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.MariaDB [(none)]> use keystone;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -ADatabase changedMariaDB [keystone]> show tables;
+------------------------+
| Tables_in_keystone     |
+------------------------+
| access_token           |
| assignment             |
| config_register        |
| consumer               |
| credential             |
| domain                 |
| endpoint               |
| endpoint_group         |
| federated_user         |
| federation_protocol    |
| group                  |
| id_mapping             |
| identity_provider      |
| idp_remote_ids         |
| implied_role           |
| local_user             |
| mapping                |
| migrate_version        |
| password               |
| policy                 |
| policy_association     |
| project                |
| project_endpoint       |
| project_endpoint_group |
| region                 |
| request_token          |
| revocation_event       |
| role                   |
| sensitive_config       |
| service                |
| service_provider       |
| token                  |
| trust                  |
| trust_role             |
| user                   |
| user_group_membership  |
| whitelisted_config     |
+------------------------+
37 rows in set (0.00 sec)MariaDB [keystone]> 

初始化Fernet keys:

 keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone

五、配置 Apache HTTP 服务器

[root@open1 keystone]# vim /etc/httpd/conf/httpd.conf

 97 ServerName 172.25.39.1:80

[root@open1 conf.d]# vim /etc/httpd/conf.d/wsgi-keystone.conf

Listen 5000--------5000是public公共端口
Listen 35357---------管理员端口<VirtualHost *:5000>WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}WSGIProcessGroup keystone-publicWSGIScriptAlias / /usr/bin/keystone-wsgi-publicWSGIApplicationGroup %{GLOBAL}WSGIPassAuthorization OnErrorLogFormat "%{cu}t %M"ErrorLog /var/log/httpd/keystone-error.logCustomLog /var/log/httpd/keystone-access.log combined<Directory /usr/bin>Require all granted</Directory>
</VirtualHost><VirtualHost *:35357>WSGIDaemonProcess keystone-admin processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}WSGIProcessGroup keystone-adminWSGIScriptAlias / /usr/bin/keystone-wsgi-adminWSGIApplicationGroup %{GLOBAL}WSGIPassAuthorization OnErrorLogFormat "%{cu}t %M"ErrorLog /var/log/httpd/keystone-error.logCustomLog /var/log/httpd/keystone-access.log combined<Directory /usr/bin>Require all granted</Directory>
</VirtualHost>

systemctl enable httpd.service
systemctl start httpd.service

此时发现端口5000和35357打开

[root@open1 conf.d]# netstat -antlp
tcp6 0 0 :::5000 :::* LISTEN 5682/httpd
tcp6 0 0 :::35357 :::* LISTEN 5682/httpd

六、创建服务实体和API端点
[root@open1 keystone]# head keystone.conf

[DEFAULT]admin_token = e185f1e79b487c6eb9aa#
# From keystone
## A "shared secret" that can be used to bootstrap Keystone. This "token" does
# not represent a user, and carries no explicit authorization. If set to

配置认证令牌:

[root@open1 keystone]# export OS_TOKEN=e185f1e79b487c6eb9aa

配置端点URL:

[root@open1 keystone]# export OS_URL=http://172.25.39.1:35357/v3

配置认证 API 版本:

[root@open1 keystone]# export OS_IDENTITY_API_VERSION=3

创建服务实体和身份认证服务

[root@open1 keystone]# openstack service create \
>   --name keystone --description "OpenStack Identity" identity
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | OpenStack Identity               |
| enabled     | True                             |
| id          | dc180da37ec3499ead33bc46bba8c493 |
| name        | keystone                         |
| type        | identity                         |
+-------------+----------------------------------+
[root@open1 keystone]# openstack service list
+----------------------------------+----------+----------+
| ID                               | Name     | Type     |
+----------------------------------+----------+----------+
| dc180da37ec3499ead33bc46bba8c493 | keystone | identity |
+----------------------------------+----------+----------+

创建认证服务的 API 端点:

[root@open1 keystone]# openstack endpoint create --region RegionOne \
>   identity public http://172.25.39.1:5000/v3
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 9e5fc0e3c7114a23b8dce6bac93cebb7 |
| interface    | public                           |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | dc180da37ec3499ead33bc46bba8c493 |
| service_name | keystone                         |
| service_type | identity                         |
| url          | http://172.25.39.1:5000/v3       |
+--------------+----------------------------------+
[root@open1 keystone]# openstack endpoint create --region RegionOne   identity internal http://172.25.39.1:5000/v3
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 891cc508dbc14b9c818a5de6e70185b6 |
| interface    | internal                         |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | dc180da37ec3499ead33bc46bba8c493 |
| service_name | keystone                         |
| service_type | identity                         |
| url          | http://172.25.39.1:5000/v3       |
+--------------+----------------------------------+
[root@open1 keystone]# openstack endpoint create --region RegionOne \
>   identity admin http://172.25.39.1:5000/v3
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 951113d2355848e89d1e3bbcf730e6b3 |
| interface    | admin                            |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | dc180da37ec3499ead33bc46bba8c493 |
| service_name | keystone                         |
| service_type | identity                         |
| url          | http://172.25.39.1:5000/v3       |
+--------------+----------------------------------+
[root@open1 keystone]# openstack endpoint list
+----------+----------+--------------+--------------+---------+-----------+------------+
| ID       | Region   | Service Name | Service Type | Enabled | Interface | URL        |
+----------+----------+--------------+--------------+---------+-----------+------------+
| 891cc508 | RegionOn | keystone     | identity     | True    | internal  | http://172 |
| dbc14b9c | e        |              |              |         |           | .25.39.1:5 |
| 818a5de6 |          |              |              |         |           | 000/v3     |
| e70185b6 |          |              |              |         |           |            |
| 951113d2 | RegionOn | keystone     | identity     | True    | admin     | http://172 |
| 355848e8 | e        |              |              |         |           | .25.39.1:5 |
| 9d1e3bbc |          |              |              |         |           | 000/v3     |
| f730e6b3 |          |              |              |         |           |            |
| 9e5fc0e3 | RegionOn | keystone     | identity     | True    | public    | http://172 |
| c7114a23 | e        |              |              |         |           | .25.39.1:5 |
| b8dce6ba |          |              |              |         |           | 000/v3     |
| c93cebb7 |          |              |              |         |           |            |
+----------+----------+--------------+--------------+---------+-----------+------------+

如过不慎手残加错,那么使用下述办法删除(如果相同类型名加了多个,那么就要把相同的全部删除)

[root@open1 keystone]# openstack endpoint delete 951113d2355848e89d1e3bbcf730e6b3
[root@open1 keystone]# openstack endpoint list
+------------+-----------+--------------+--------------+---------+-----------+------------+
| ID         | Region    | Service Name | Service Type | Enabled | Interface | URL        |
+------------+-----------+--------------+--------------+---------+-----------+------------+
| 891cc508db | RegionOne | keystone     | identity     | True    | internal  | http://172 |
| c14b9c818a |           |              |              |         |           | .25.39.1:5 |
| 5de6e70185 |           |              |              |         |           | 000/v3     |
| b6         |           |              |              |         |           |            |
| 9e5fc0e3c7 | RegionOne | keystone     | identity     | True    | public    | http://172 |
| 114a23b8dc |           |              |              |         |           | .25.39.1:5 |
| e6bac93ceb |           |              |              |         |           | 000/v3     |
| b7         |           |              |              |         |           |            |
+------------+-----------+--------------+--------------+---------+-----------+------------+
[root@open1 keystone]# openstack endpoint create --region RegionOne   identity admin http://172.25.39.1:35357/v3
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | ffd8c30bda9a48248a8996e1d5c7df2d |
| interface    | admin                            |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | dc180da37ec3499ead33bc46bba8c493 |
| service_name | keystone                         |
| service_type | identity                         |
| url          | http://172.25.39.1:35357/v3      |
+--------------+----------------------------------+

七、创建域、项目、用户和角色

创建域default:openstack domain create –description “Default Domain” default

[root@open1 keystone]# openstack domain create --description "Default Domain" default
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | Default Domain                   |
| enabled     | True                             |
| id          | a083730276e148f7aeb7e5945a72e313 |
| name        | default                          |
+-------------+----------------------------------+

在你的环境中,为进行管理操作,创建管理的项目、用户和角色:

创建 admin 项目:openstack project create –domain default –description “Admin Project” admin

[root@open1 keystone]#  openstack project create --domain default \
>   --description "Admin Project" admin
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | Admin Project                    |
| domain_id   | a083730276e148f7aeb7e5945a72e313 |
| enabled     | True                             |
| id          | 1ab2b4aef473437facd40c17255061d8 |
| is_domain   | False                            |
| name        | admin                            |
| parent_id   | a083730276e148f7aeb7e5945a72e313 |
+-------------+----------------------------------+

创建 admin 用户:openstack user create –domain default –password-prompt admin
也可以使用非交互式openstack user create –domain default –password westos admin

[root@open1 keystone]#  openstack user create --domain default \
> --password-prompt admin
User Password:
Repeat User Password:
+-----------+----------------------------------+
| Field     | Value                            |
+-----------+----------------------------------+
| domain_id | a083730276e148f7aeb7e5945a72e313 |
| enabled   | True                             |
| id        | d27c9e220e24407dad0d68ef8b5b6dd1 |
| name      | admin                            |
+-----------+----------------------------------+

创建 admin 角色:openstack role create admin

[root@open1 keystone]# openstack role create admin
+-----------+----------------------------------+
| Field     | Value                            |
+-----------+----------------------------------+
| domain_id | None                             |
| id        | 3fe8bce6e65346a8a8a3f407b3f7013e |
| name      | admin                            |
+-----------+----------------------------------+
[root@open1 keystone]#  openstack role add --project admin --user admin admin-----添加admin角色到 admin 项目和用户上

添加你的环境中每个服务包含独有用户的service 项目。
创建service项目:

[root@open1 keystone]# openstack project create --domain default \
>   --description "Service Project" service
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | Service Project                  |
| domain_id   | a083730276e148f7aeb7e5945a72e313 |
| enabled     | True                             |
| id          | 93a2d1312ac6436c9c67f00146f501ae |
| is_domain   | False                            |
| name        | service                          |
| parent_id   | a083730276e148f7aeb7e5945a72e313 |
+-------------+----------------------------------+

常规(非管理)任务应该使用无特权的项目和用户。

[root@open1 keystone]# openstack project create --domain default \
>   --description "Demo Project" demo----创建 demo 项目
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | Demo Project                     |
| domain_id   | a083730276e148f7aeb7e5945a72e313 |
| enabled     | True                             |
| id          | ecced8c00e8249edb3c5730a8a6a4cd1 |
| is_domain   | False                            |
| name        | demo                             |
| parent_id   | a083730276e148f7aeb7e5945a72e313 |
+-------------+----------------------------------+[root@open1 keystone]# openstack user create --domain default \
>   --password-prompt demo0---------创建demo用户
User Password:
Repeat User Password:
+-----------+----------------------------------+
| Field     | Value                            |
+-----------+----------------------------------+
| domain_id | a083730276e148f7aeb7e5945a72e313 |
| enabled   | True                             |
| id        | 33a3c544d28c4c3bb8cb1291d224e7b2 |
| name      | demo                             |
+-----------+----------------------------------+[root@open1 keystone]# openstack role create user----创建 user 角色
+-----------+----------------------------------+
| Field     | Value                            |
+-----------+----------------------------------+
| domain_id | None                             |
| id        | b78b59789c844680b95383239cc84362 |
| name      | user                             |
+-----------+----------------------------------+
[root@open1 keystone]# openstack role add --project demo --user demo user----添加 user角色到demo项目和用户

八、验证操作

重置OS_TOKENOS_URL 环境变量

[root@open1 keystone]# unset OS_TOKEN OS_URL

作为 admin 用户,请求认证令牌:

[root@open1 keystone]# openstack --os-auth-url http://172.25.39.1:35357/v3 \
> --os-project-domain-name default --os-user-domain-name default \
> --os-project-name admin --os-username admin token issue
Password:
+------------+------------------------------------------------------------------------+
| Field      | Value                                                                  |
+------------+------------------------------------------------------------------------+
| expires    | 2017-08-15T07:40:50.336000Z                                            |
| id         | gAAAAABZkpdyzjLa5a3HO0p3XAEI33VfwR6mTum3WL-                            |
|            | cmlDgwLKMZHH6mAZQPKUXeT7Msl7j7IZKAK6sU5PDD8N-                          |
|            | A8WOgER3GPL1Kk6gZFZJZ3HhMYJ9oJ7bKXB_09hGXpH3Nv_u4duIqw4J5udyaQ7Lp-     |
|            | 40tswbn8-nh2GlyRcBU4TiSZUGmWE                                          |
| project_id | 1ab2b4aef473437facd40c17255061d8                                       |
| user_id    | d27c9e220e24407dad0d68ef8b5b6dd1                                       |
+------------+------------------------------------------------------------------------+

作为demo用户,请求认证令牌:

[root@open1 keystone]# openstack --os-auth-url http://172.25.39.1:5000/v3 \
>  --os-project-domain-name default --os-user-domain-name default \
> --os-project-name demo --os-username demo token issue
Password:
+------------+------------------------------------------------------------------------+
| Field      | Value                                                                  |
+------------+------------------------------------------------------------------------+
| expires    | 2017-08-15T07:42:20.192604Z                                            |
| id         | gAAAAABZkpfMItDJXBmZegknfJkvPecAy5CWvGqQkJ0DXqhNGzUvIlrYaQysVf6IlzAko1 |
|            | N8HVk-PcunGkc36CDdOeKzuL1rQTvpqoU2PcAEoJ6aZG16G680Wa8iGHfCJEMdi_1vE0u_ |
|            | Lr9hX7bxbRLf87xrvKnDBhLgQMU31ORf_Jmo5tsT1kI                            |
| project_id | ecced8c00e8249edb3c5730a8a6a4cd1                                       |
| user_id    | 33a3c544d28c4c3bb8cb1291d224e7b2                                       |
+------------+------------------------------------------------------------------------

九、创建 OpenStack 客户端环境脚本
为了提升客户端操作的效率,OpenStack支持简单的客户端环境变量脚本即OpenRC 文件。

创建admin项目和用户创建客户端环境变量脚本
[root@open1 keystone]# vim admin-openrc

[root@open1 keystone]# cat admin-openrc
export OS_PROJECT_DOMAIN_NAME=default
export OS_USER_DOMAIN_NAME=default
export OS_PROJECT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=westos
export OS_AUTH_URL=http://172.25.39.1:35357/v3
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2

创建demo项目和用户创建客户端环境变量脚本
[root@open1 keystone]# vim demo-openrc

[root@open1 keystone]# cat demo-openrc
export OS_PROJECT_DOMAIN_NAME=default
export OS_USER_DOMAIN_NAME=default
export OS_PROJECT_NAME=demo
export OS_USERNAME=demo
export OS_PASSWORD=westos
export OS_AUTH_URL=http://172.25.39.1:5000/v3
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2

[root@open1 keystone]# chmod +x admin-openrc
[root@open1 keystone]# chmod +x demo-openrc

脚本的使用:

加载admin-openrc文件来身份认证服务的环境变量位置和admin项目和用户证书:
[root@open1 keystone]# . admin-openrc
[root@open1 keystone]# source admin-openrc
[root@open1 keystone]# openstack user list

+----------------------------------+-------+
| ID                               | Name  |
+----------------------------------+-------+
| 33a3c544d28c4c3bb8cb1291d224e7b2 | demo  |
| d27c9e220e24407dad0d68ef8b5b6dd1 | admin |
+----------------------------------+-------+

请求认证令牌:
[root@open1 keystone]# openstack token issue

+------------+---------------------------------------------------+
| Field      | Value                                             |
+------------+---------------------------------------------------+
| expires    | 2017-08-15T07:56:53.860553Z                       |
| id         | gAAAAABZkps11iyK3hs0KUwHKaSohcrXe2nvS7edn-        |
|            | NTfFka0PKAF1FlBetXG8wU4QLUERmSVhGJ-               |
|            | CFdbzkgTv0YlUgdsGjohvUpyv-F9iabWHuzzi3iu-O6S_l5xG |
|            | ADfnfcsZaE3zsA_IaRb4nDGvssvQ23f1cx3F11tvNU7c-     |
|            | xBAFdFrhLRF8                                      |
| project_id | 1ab2b4aef473437facd40c17255061d8                  |
| user_id    | d27c9e220e24407dad0d68ef8b5b6dd1                  |
+------------+---------------------------------------------------+
这个时候使用不同的用户就可以发现权限不同:
[root@open1 keystone]# source demo-openrc
[root@open1 keystone]# openstack user list
You are not authorized to perform the requested action: identity:list_users (HTTP 403) (Request-ID: req-39074f7f-fad3-48b1-9d0e-4a3de732cab2)
[root@open1 keystone]# source admin-openrc
[root@open1 keystone]# openstack user list
+----------------------------------+-------+
| ID                               | Name  |
+----------------------------------+-------+
| 33a3c544d28c4c3bb8cb1291d224e7b2 | demo  |
| d27c9e220e24407dad0d68ef8b5b6dd1 | admin |
+----------------------------------+-------+

十、安装与配置
安全并配置组件

安装软件包:
[root@open1 ~]# yum install openstack-glance

[root@open1 ~]# vim /etc/glance/glance-api.conf
在 [database] 部分,配置数据库访问:

[database]
connection = mysql+pymysql://glance:westos@172.25.39.1/glance在 [keystone_authtoken] 和 [paste_deploy] 部分,配置认证服务访问:[keystone_authtoken]
auth_uri = http://172.25.39.1:5000
auth_url = http://172.25.39.1:35357
memcached_servers = 172.25.39.1:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = glance
password = westos[paste_deploy]
flavor = keystone在 [glance_store] 部分,配置本地文件系统存储和镜像文件位置:
[glance_store]
stores = file,http
default_store = file
filesystem_store_datadir = /var/lib/glance/images/

[root@open1 ~]# vim /etc/glance/glance-registry.conf

在[database] 部分,配置数据库访问:
[database]
connection = mysql+pymysql://glance:westos@172.25.39.1/glance在 [keystone_authtoken] 和 [paste_deploy] 部分,配置认证服务访问:
[keystone_authtoken]
auth_uri = http://172.25.39.1:5000
auth_url = http://172.25.39.1:35357
memcached_servers = 172.25.39.1:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = glance
password = westos[paste_deploy]
flavor = keystone

写入镜像服务数据库:
[root@open1 ~]# su -s /bin/sh -c “glance-manage db_sync” glance

Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:1056: OsloDBDeprecationWarning: EngineFacade is deprecated; please use oslo_db.sqlalchemy.enginefacadeexpire_on_commit=expire_on_commit, _conf=conf)
/usr/lib/python2.7/site-packages/pymysql/cursors.py:166: Warning: (1831, u'Duplicate index `ix_image_properties_image_id_name`. This is deprecated and will be disallowed in a future release.')result = self._query(query)

验证:
[root@open1 ~]# mysql -pwestos

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.1.20-MariaDB MariaDB ServerCopyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| glance             |
| information_schema |
| keystone           |
| mysql              |
| neutron            |
| nova               |
| nova_api           |
| performance_schema |
+--------------------+
8 rows in set (0.00 sec)MariaDB [(none)]> use glance;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -ADatabase changed
MariaDB [glance]> show tables;
+----------------------------------+
| Tables_in_glance                 |
+----------------------------------+
| artifact_blob_locations          |
| artifact_blobs                   |
| artifact_dependencies            |
| artifact_properties              |
| artifact_tags                    |
| artifacts                        |
| image_locations                  |
| image_members                    |
| image_properties                 |
| image_tags                       |
| images                           |
| metadef_namespace_resource_types |
| metadef_namespaces               |
| metadef_objects                  |
| metadef_properties               |
| metadef_resource_types           |
| metadef_tags                     |
| migrate_version                  |
| task_info                        |
| tasks                            |
+----------------------------------+
20 rows in set (0.00 sec)MariaDB [glance]> exit
Bye

启动镜像服务、配置他们随机启动:
[root@open1 ~]# systemctl enable openstack-glance-api.service \

openstack-glance-registry.service
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-glance-api.service to /usr/lib/systemd/system/openstack-glance-api.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-glance-registry.service to /usr/lib/systemd/system/openstack-glance-registry.service.
[root@open1 ~]# systemctl start openstack-glance-api.service \
openstack-glance-registry.service
端口的查看:(9292、9191)
[root@open1 ~]# netstat -antlp

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:25672           0.0.0.0:*               LISTEN      920/beam
tcp        0      0 172.25.39.1:3306        0.0.0.0:*               LISTEN      2233/mysqld
tcp        0      0 172.25.39.1:11211       0.0.0.0:*               LISTEN      928/memcached
tcp        0      0 127.0.0.1:11211         0.0.0.0:*               LISTEN      928/memcached
tcp        0      0 0.0.0.0:9292            0.0.0.0:*               LISTEN      3356/python2
tcp        0      0 0.0.0.0:4369            0.0.0.0:*               LISTEN      1/systemd
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      919/sshd
tcp        0      0 0.0.0.0:15672           0.0.0.0:*               LISTEN      920/beam
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      2201/master
tcp        0      0 0.0.0.0:9191            0.0.0.0:*               LISTEN      3357/python2
tcp        0      0 127.0.0.1:51219         127.0.0.1:4369          ESTABLISHED 920/beam
tcp        0      0 172.25.39.1:4369        172.25.39.1:52101       TIME_WAIT   -
tcp        0      0 127.0.0.1:4369          127.0.0.1:51219         ESTABLISHED 2305/epmd
tcp        0      0 172.25.39.1:22          172.25.39.250:44435     ESTABLISHED 3041/sshd: root@pts
tcp6       0      0 :::5672                 :::*                    LISTEN      920/beam
tcp6       0      0 :::5000                 :::*                    LISTEN      931/httpd
tcp6       0      0 ::1:11211               :::*                    LISTEN      928/memcached
tcp6       0      0 :::80                   :::*                    LISTEN      931/httpd
tcp6       0      0 :::22                   :::*                    LISTEN      919/sshd
tcp6       0      0 ::1:25                  :::*                    LISTEN      2201/master
tcp6       0      0 :::35357                :::*                    LISTEN      931/httpd           

获得 admin 凭证来获取只有管理员能执行的命令的访问权限:

[root@open1 ~]# cd /etc/keystone/
[root@open1 keystone]# ls
admin-openrc               keystone-paste.ini
default_catalog.templates  logging.conf
demo-openrc                policy.json
fernet-keys                sso_callback_template.html
keystone.conf
[root@open1 keystone]# . admin-openrc 

要创建服务证书,完成这些步骤:
创建 glance 用户:

[root@open1 keystone]#  openstack user create --domain default --password westos glance
+-----------+----------------------------------+
| Field     | Value                            |
+-----------+----------------------------------+
| domain_id | a083730276e148f7aeb7e5945a72e313 |
| enabled   | True                             |
| id        | 04d7b60512a14903b7c1b52c47d950d7 |
| name      | glance                           |
+-----------+----------------------------------+
添加 admin 角色到 glance 用户和 service 项目上。
[root@open1 keystone]# openstack role add --project service --user glance admin
创建``glance``服务实体:
[root@open1 keystone]# openstack service create --name glance \
>   --description "OpenStack Image" image
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | OpenStack Image                  |
| enabled     | True                             |
| id          | c2ff763f0d984c9ab4d07fef5cd11fb4 |
| name        | glance                           |
| type        | image                            |
+-------------+----------------------------------+
创建镜像服务的 API 端点:
[root@open1 keystone]# openstack endpoint create --region RegionOne \
>   image public http://172.25.39.1:9292
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 32e784619964466c90d0f81afeaaa2c5 |
| interface    | public                           |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | c2ff763f0d984c9ab4d07fef5cd11fb4 |
| service_name | glance                           |
| service_type | image                            |
| url          | http://172.25.39.1:9292          |
+--------------+----------------------------------+
[root@open1 keystone]# openstack endpoint create --region RegionOne   image internal http://172.25.39.1:9292
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | e60c8ea7213a4bc4a2d1312b39b5b433 |
| interface    | internal                         |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | c2ff763f0d984c9ab4d07fef5cd11fb4 |
| service_name | glance                           |
| service_type | image                            |
| url          | http://172.25.39.1:9292          |
+--------------+----------------------------------+
[root@open1 keystone]# openstack endpoint create --region RegionOne   image admin http://172.25.39.1:9292
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 65111feb6762436389b96cffc30c2b7b |
| interface    | admin                            |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | c2ff763f0d984c9ab4d07fef5cd11fb4 |
| service_name | glance                           |
| service_type | image                            |
| url          | http://172.25.39.1:9292          |
+--------------+----------------------------------+
[root@open1 keystone]# cd
[root@open1 ~]# ls
cirros-0.3.4-x86_64-disk.img
[root@open1 ~]# du -sh
13M .
[root@open1 ~]# openstack endpoint list
+----------+----------+--------------+--------------+---------+-----------+----------+
| ID       | Region   | Service Name | Service Type | Enabled | Interface | URL      |
+----------+----------+--------------+--------------+---------+-----------+----------+
| 32e78461 | RegionOn | glance       | image        | True    | public    | http://1 |
| 9964466c | e        |              |              |         |           | 72.25.39 |
| 90d0f81a |          |              |              |         |           | .1:9292  |
| feaaa2c5 |          |              |              |         |           |          |
| 65111feb | RegionOn | glance       | image        | True    | admin     | http://1 |
| 67624363 | e        |              |              |         |           | 72.25.39 |
| 89b96cff |          |              |              |         |           | .1:9292  |
| c30c2b7b |          |              |              |         |           |          |
| 891cc508 | RegionOn | keystone     | identity     | True    | internal  | http://1 |
| dbc14b9c | e        |              |              |         |           | 72.25.39 |
| 818a5de6 |          |              |              |         |           | .1:5000/ |
| e70185b6 |          |              |              |         |           | v3       |
| 9e5fc0e3 | RegionOn | keystone     | identity     | True    | public    | http://1 |
| c7114a23 | e        |              |              |         |           | 72.25.39 |
| b8dce6ba |          |              |              |         |           | .1:5000/ |
| c93cebb7 |          |              |              |         |           | v3       |
| e60c8ea7 | RegionOn | glance       | image        | True    | internal  | http://1 |
| 213a4bc4 | e        |              |              |         |           | 72.25.39 |
| a2d1312b |          |              |              |         |           | .1:9292  |
| 39b5b433 |          |              |              |         |           |          |
| ffd8c30b | RegionOn | keystone     | identity     | True    | admin     | http://1 |
| da9a4824 | e        |              |              |         |           | 72.25.39 |
| 8a8996e1 |          |              |              |         |           | .1:35357 |
| d5c7df2d |          |              |              |         |           | /v3      |
+----------+----------+--------------+--------------+---------+-----------+----------+

使用 QCOW2 磁盘格式, bare 容器格式上传镜像到镜像服务并设置公共可见,这样所有的项目都可以访问它:
[root@open1 ~]# openstack image create “cirros” \

–file cirros-0.3.4-x86_64-disk.img \
–disk-format qcow2 –container-format bare \
–public

+------------------+-------------------------------------+
| Field            | Value                               |
+------------------+-------------------------------------+
| checksum         | ee1eca47dc88f4879d8a229cc70a07c6    |
| container_format | bare                                |
| created_at       | 2017-08-15T08:32:49Z                |
| disk_format      | qcow2                               |
| file             | /v2/images/03b18764-941d-419c-894d- |
|                  | db3f1f4f73e0/file                   |
| id               | 03b18764-941d-419c-894d-            |
|                  | db3f1f4f73e0                        |
| min_disk         | 0                                   |
| min_ram          | 0                                   |
| name             | cirros                              |
| owner            | 1ab2b4aef473437facd40c17255061d8    |
| protected        | False                               |
| schema           | /v2/schemas/image                   |
| size             | 13287936                            |
| status           | active                              |
| tags             |                                     |
| updated_at       | 2017-08-15T08:32:49Z                |
| virtual_size     | None                                |
| visibility       | public                              |
+------------------+-------------------------------------+

确认镜像的上传并验证属性:

[root@open1 ~]# openstack image list
+--------------------------------------+--------+--------+
| ID                                   | Name   | Status |
+--------------------------------------+--------+--------+
| 03b18764-941d-419c-894d-db3f1f4f73e0 | cirros | active |
+--------------------------------------+--------+--------+
[root@open1 ~]# cd /var/lib/glance/
[root@open1 glance]# ls
images
[root@open1 glance]# cd images/
[root@open1 images]# ls
03b18764-941d-419c-894d-db3f1f4f73e0
[root@open1 images]# du -h
13M .

十一、计算服务

【【【安装并配置控制节点】】】——控制节点

获得 admin 凭证来获取只有管理员能执行的命令的访问权限:
[root@open1 ~]# cd /etc/keystone/
[root@open1 keystone]# source admin-openrc
[root@open1 keystone]# . admin-openrc
要创建服务证书

创建 nova 用户:openstack user create --domain default --password westos nova
给 nova 用户添加 admin 角色:openstack role add --project service --user nova admin
创建 nova 服务实体:openstack service create --name nova --description "OpenStack Compute" compute
创建 Compute 服务 API 端点 :openstack endpoint create --region RegionOne \compute public http://172.25.39.1:8774/v2.1/%\(tenant_id\)s
openstack endpoint create --region RegionOne \compute internal http://controller:8774/v2.1/%\(tenant_id\)sopenstack endpoint create --region RegionOne \compute admin http://controller:8774/v2.1/%\(tenant_id\)s

安装并配置组件
yum install openstack-nova-api openstack-nova-conductor openstack-nova-console openstack-nova-novncproxy openstack-nova-scheduler

编辑/etc/nova/nova.conf文件并完成下面的操作:

   1 [DEFAULT]2 enabled_apis = osapi_compute,metadata----只启用计算和元数据API3 rpc_backend = rabbit4 auth_strategy = keystone5 use_neutron = True6 firewall_driver = nova.virt.firewall.NoopFirewallDriver
(默认情况下,计算服务使用内置的防火墙服务。由于网络服务包含了防火墙服务,你必须使用``nova.virt.firewall.NoopFirewallDriver``防火墙服务来禁用掉计算服务内置的防火墙服务)配置数据库的连接:
2166 [api_database]
2167 connection = mysql+pymysql://nova:westos@172.25.39.1/nova_api
3112 [database]
3113 connection = mysql+pymysql://nova:westos@172.25.39.1/nova配置 “RabbitMQ” 消息队列访问:
4412 [oslo_messaging_rabbit]
4413 rabbit_host = 172.25.39.1
4414 rabbit_userid = openstack
4415 rabbit_password = westos配置认证服务访问:
3531 [keystone_authtoken]
3532 auth_uri = http://172.25.39.1:5000
3533 auth_url = http://172.25.39.1:35357
3534 memcached_servers = 172.25.39.1:11211
3535 auth_type = password
3536 project_domain_name = default
3537 user_domain_name = default
3538 project_name = service
3539 username = nova
3540 password = westos配置VNC代理使用控制节点的管理接口IP地址 :
5368 [vnc]
5369 vncserver_listen = 172.25.39.1
5370 vncserver_proxyclient_address = 172.25.39.1配置镜像服务 API 的位置:
3330 [glance]
3331 api_servers = http://172.25.39.1:9292配置锁路径:
4300 [oslo_concurrency]
4301 lock_path = /var/lib/nova/tmp

同步Compute 数据库:

su -s /bin/sh -c "nova-manage api_db sync" nova
su -s /bin/sh -c "nova-manage db sync" nova

启动 Compute 服务并将其设置为随系统启动:

# systemctl enable openstack-nova-api.service \openstack-nova-consoleauth.service openstack-nova-scheduler.service \openstack-nova-conductor.service openstack-nova-novncproxy.service
# systemctl start openstack-nova-api.service \openstack-nova-consoleauth.service openstack-nova-scheduler.service \openstack-nova-conductor.service openstack-nova-novncproxy.service

【【【安装和配置计算节点】】】——–计算节点

另一台节点,记得时间同步

安装软件包:
yum install openstack-nova-compute -y

编辑/etc/nova/nova.conf文件并完成下面的操作:

[DEFAULT]
rpc_backend = rabbit
auth_strategy = keystone
my_ip = 172.25.39.1-----配置 my_ip (计算节点上的管理网络接口的IP 地址)《可以不加》
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver配置RabbitMQ消息队列的连接:
[oslo_messaging_rabbit]
rabbit_host = 172.25.39.1
rabbit_userid = openstack
rabbit_password = westos配置认证服务访问:
[keystone_authtoken]
auth_uri = http://172.25.39.1:5000
auth_url = http://172.25.39.1:35357
memcached_servers = 172.25.39.1:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = westos启用并配置远程控制台访问:
[vnc]
enabled = True
vncserver_listen = 0.0.0.0
vncserver_proxyclient_address = 172.25.39.2
novncproxy_base_url = http://172.25.39.1:6080/vnc_auto.html配置镜像服务 API 的位置:
[glance]
api_servers = http://172.25.39.1:9292配置锁路径:
[oslo_concurrency]
lock_path = /var/lib/nova/tmp[确定您的计算节点是否支持虚拟机的硬件加速。egrep -c '(vmx|svm)' /proc/cpuinfo]
如果不支持那么就要在文件中编辑如下:[libvirt]
virt_type = qemu

启动计算服务及其依赖,并将其配置为随系统自动启动:
systemctl enable libvirtd.service openstack-nova-compute.service
systemctl start libvirtd.service openstack-nova-compute.service

获得 admin 凭证来获取只有管理员能执行的命令的访问权限:
在这个文件存在的目录下:
. admin-openrc

列出服务组件,以验证是否成功启动并注册了每个进程:
[root@open1 keystone]# openstack compute service list
+----+----------+-------+----------+---------+-------+------------+
| Id | Binary   | Host  | Zone     | Status  | State | Updated At |
+----+----------+-------+----------+---------+-------+------------+
|  1 | nova-con | open1 | internal | enabled | up    | 2017-08-16 |
|    | ductor   |       |          |         |       | T02:47:08. |
|    |          |       |          |         |       | 000000     |
|  2 | nova-con | open1 | internal | enabled | up    | 2017-08-16 |
|    | soleauth |       |          |         |       | T02:47:08. |
|    |          |       |          |         |       | 000000     |
|  3 | nova-sch | open1 | internal | enabled | up    | 2017-08-16 |
|    | eduler   |       |          |         |       | T02:47:08. |
|    |          |       |          |         |       | 000000     |
|  6 | nova-    | open2 | nova     | enabled | up    | 2017-08-16 |
|    | compute  |       |          |         |       | T02:47:10. |
|    |          |       |          |         |       | 000000     |
+----+----------+-------+----------+---------+-------+------------+

十二、Networking 服务

【【【安装并配置控制节点】】】
获得 admin 凭证来获取只有管理员能执行的命令的访问权限:
[root@open1 keystone]# . admin-openrc

创建``neutron``用户:
openstack user create --domain default --password westos neutron添加``admin`` 角色到``neutron`` 用户:
openstack user create --domain default --password westos neutron创建``neutron``服务实体
openstack service create --name neutron \--description "OpenStack Networking" network创建网络服务API端点:
openstack endpoint create --region RegionOne \network public http://172.25.39.1:9696
openstack endpoint create --region RegionOne \network internal http://172.25.39.1:9696
openstack endpoint create --region RegionOne \network admin http://172.25.39.1:9696

{公共网络里面}
在controller节点上安装并配置网络组件

安装组件
yum install openstack-neutron openstack-neutron-ml2 \
openstack-neutron-linuxbridge ebtables

配置服务组件

编辑/etc/neutron/neutron.conf

配置数据库访问:
[database]
connection = mysql+pymysql://neutron:westos@172.25.39.1/neutron[DEFAULT]
core_plugin = ml2---启用ML2插件并禁用其他插件
service_plugins =
rpc_backend = rabbit
auth_strategy = keystone
notify_nova_on_port_status_changes = True
notify_nova_on_port_data_changes = True配置 “RabbitMQ” 消息队列的连接:
[oslo_messaging_rabbit]
rabbit_host = 172.25.39.1
rabbit_userid = openstack
rabbit_password = westos配置认证服务访问:
[keystone_authtoken]
auth_uri = http://172.25.39.1:5000
auth_url = http://172.25.39.1:35357
memcached_servers = 172.25.39.1:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = westos配置网络服务来通知计算节点的网络拓扑变化:
[nova]
auth_url = http://172.25.39.1:35357
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = nova
password = westos配置锁路径:
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp

配置 Modular Layer 2 (ML2) 插件(ML2插件使用Linuxbridge机制来为实例创建layer-2虚拟网络基础设施)
编辑/etc/neutron/plugins/ml2/ml2_conf.ini

[ml2]
type_drivers = flat,vlan------启用flat和VLAN网络
tenant_network_types = ---------禁用私有网络
mechanism_drivers = linuxbridge -----启用Linuxbridge机制
extension_drivers = port_security ------启用端口安全扩展驱动配置公共虚拟网络为flat网络
[ml2_type_flat]
flat_networks = provider启用 ipset 增加安全组规则的高效性:
[securitygroup]
enable_ipset = True

配置Linuxbridge代理(Linuxbridge代理为实例建立layer-2虚拟网络并且处理安全组规则。)

编辑/etc/neutron/plugins/ml2/linuxbridge_agent.ini

将公共虚拟网络和公共物理网络接口对应起来:
[linux_bridge]
physical_interface_mappings = provider:eth1禁止VXLAN覆盖网络:
[vxlan]
enable_vxlan = False启用安全组并配置 Linuxbridge iptables firewall driver:
[securitygroup]
enable_security_group = True
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver

配置DHCP代理

编辑/etc/neutron/dhcp_agent.ini文件

配置Linuxbridge驱动接口,DHCP驱动并启用隔离元数据,这样在公共网络上的实例就可以通过网络来访问元数据

[DEFAULT]
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata = True

配置元数据代理

编辑`/etc/neutron/metadata_agent.ini

配置元数据主机以及共享密码:
[DEFAULT]
nova_metadata_ip = 172.25.39.1
metadata_proxy_shared_secret = westos

为计算节点配置网络服务
编辑/etc/nova/nova.conf文件

配置访问参数,启用元数据代理并设置密码:
[neutron]
url = http://172.25.39.1:9696
auth_url = http://172.25.39.1:35357
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = neutron
password = westosservice_metadata_proxy = True
metadata_proxy_shared_secret = METADATA_SECRET

完成安装

网络服务初始化脚本需要一个超链接 /etc/neutron/plugin.ini指向ML2插件配置文件/etc/neutron/plugins/ml2/ml2_conf.ini

ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini

同步数据库:

su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf \--config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron

重启计算API 服务
systemctl restart openstack-nova-api.service

当系统启动时,启动 Networking 服务并配置它启动。
对于两种网络选项:
systemctl enable neutron-server.service \
neutron-linuxbridge-agent.service neutron-dhcp-agent.service \
neutron-metadata-agent.service

systemctl start neutron-server.service \
neutron-linuxbridge-agent.service neutron-dhcp-agent.service \
neutron-metadata-agent.service

【【【安装和配置计算节点】】】

安装组件

yum install openstack-neutron-linuxbridge ebtables ipset -y

配置通用组件

编辑/etc/neutron/neutron.conf

[DEFAULT]
rpc_backend = rabbit
auth_strategy = keystone配置 “RabbitMQ” 消息队列的连接:
[oslo_messaging_rabbit]
rabbit_host = 172.25.39.1
rabbit_userid = openstack
rabbit_password = westos配置认证服务访问:
[keystone_authtoken]
auth_uri = http://172.25.39.1:5000
auth_url = http://172.25.39.1:35357
memcached_servers = 172.25.39.1:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = westos配置锁路径:
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp

配置Linuxbridge代理

编辑/etc/neutron/plugins/ml2/linuxbridge_agent.ini

将公共虚拟网络和公共物理网络接口对应起来:[linux_bridge]
physical_interface_mappings = provider:eth1禁止VXLAN覆盖网络
[vxlan]
enable_vxlan = False启用安全组并配置 Linuxbridge iptables firewall driver:
[securitygroup]
enable_security_group = True
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver

为计算节点配置网络服务

编辑/etc/nova/nova.conf

配置访问参数:
[neutron]
url = http://172.25.39.1:9696
auth_url = http://172.25.39.1:35357
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = neutron
password = westos

完成安装

重启计算服务
systemctl restart openstack-nova-compute.service
启动Linuxbridge代理并配置它开机自启动:
systemctl enable neutron-linuxbridge-agent.service
systemctl start neutron-linuxbridge-agent.service

查看:
[root@open1 keystone]# neutron ext-list
+-----------------------+------------------------+
| alias                 | name                   |
+-----------------------+------------------------+
| default-subnetpools   | Default Subnetpools    |
| availability_zone     | Availability Zone      |
| network_availability_ | Network Availability   |
| zone                  | Zone                   |
| auto-allocated-       | Auto Allocated         |
| topology              | Topology Services      |
| binding               | Port Binding           |
| agent                 | agent                  |
| subnet_allocation     | Subnet Allocation      |
| dhcp_agent_scheduler  | DHCP Agent Scheduler   |
| tag                   | Tag support            |
| external-net          | Neutron external       |
|                       | network                |
| net-mtu               | Network MTU            |
| network-ip-           | Network IP             |
| availability          | Availability           |
| quotas                | Quota management       |
|                       | support                |
| provider              | Provider Network       |
| multi-provider        | Multi Provider Network |
| address-scope         | Address scope          |
| timestamp_core        | Time Stamp Fields      |
|                       | addition for core      |
|                       | resources              |
| extra_dhcp_opt        | Neutron Extra DHCP     |
|                       | opts                   |
| security-group        | security-group         |
| rbac-policies         | RBAC Policies          |
| standard-attr-        | standard-attr-         |
| description           | description            |
| port-security         | Port Security          |
| allowed-address-pairs | Allowed Address Pairs  |
+-----------------------+------------------------+
[root@open1 keystone]# neutron agent-list
+----------+------------+-------+-------------------+-------+----------------+----------+
| id       | agent_type | host  | availability_zone | alive | admin_state_up | binary   |
+----------+------------+-------+-------------------+-------+----------------+----------+
| 79749b6e | Metadata   | open1 |                   | :-)   | True           | neutron- |
| -1342-46 | agent      |       |                   |       |                | metadata |
| 1c-a062- |            |       |                   |       |                | -agent   |
| 8de67bec |            |       |                   |       |                |          |
| c702     |            |       |                   |       |                |          |
| 8f52a78d | Linux      | open2 |                   | :-)   | True           | neutron- |
| -f525-44 | bridge     |       |                   |       |                | linuxbri |
| 15-a37c- | agent      |       |                   |       |                | dge-     |
| 6861416a |            |       |                   |       |                | agent    |
| 9a60     |            |       |                   |       |                |          |
| 98c1c94c | Linux      | open1 |                   | :-)   | True           | neutron- |
| -30b8-4e | bridge     |       |                   |       |                | linuxbri |
| 8e-8026- | agent      |       |                   |       |                | dge-     |
| e146c757 |            |       |                   |       |                | agent    |
| 475c     |            |       |                   |       |                |          |
| 9dccd17c | DHCP agent | open1 | nova              | :-)   | True           | neutron- |
| -2dfb-   |            |       |                   |       |                | dhcp-    |
| 4c8e-8e6 |            |       |                   |       |                | agent    |
| 9-84f357 |            |       |                   |       |                |          |
| 2bae47   |            |       |                   |       |                |          |
+----------+------------+-------+-------------------+-------+----------------+----------+

接下来就可以启动实例了~

在控制节点上,加载 admin 凭证来获取管理员能执行的命令访问权限:
$ . admin-openrc

创建网络:
neutron net-create --shared --provider:physical_network provider \--provider:network_type flat provider
+-------------------------+--------------------------+
| Field                   | Value                    |
+-------------------------+--------------------------+
| admin_state_up          | True                     |
| availability_zone_hints |                          |
| availability_zones      |                          |
| created_at              | 2017-08-16T07:34:31      |
| description             |                          |
| id                      | 918f9709-917c-4665-a117- |
|                         | 1f7ea7c33b49             |
| ipv4_address_scope      |                          |
| ipv6_address_scope      |                          |
| mtu                     | 1500                     |
| name                    | provider                 |
| port_security_enabled   | True                     |
| provider:network_type   | flat                     |
| provider:physical_netwo | provider                 |
| rk                      |                          |
| provider:segmentation_i |                          |
| d                       |                          |
| router:external         | False                    |
| shared                  | True                     |
| status                  | ACTIVE                   |
| subnets                 |                          |
| tags                    |                          |
| tenant_id               | 1ab2b4aef473437facd40c17 |
|                         | 255061d8                 |
| updated_at              | 2017-08-16T07:34:31      |
+-------------------------+--------------------------+
在网络上创建一个子网:
neutron subnet-create --name provider \
  --allocation-pool start=203.0.113.101,end=203.0.113.250 \
  --dns-nameserver 8.8.4.4 --gateway 203.0.113.1 \
  provider 203.0.113.0/24
+-------------------+--------------------------------+
| Field             | Value                          |
+-------------------+--------------------------------+
| allocation_pools  | {"start": "203.0.113.101",     |
|                   | "end": "203.0.113.250"}        |
| cidr              | 203.0.113.0/24                 |
| created_at        | 2017-08-16T07:35:51            |
| description       |                                |
| dns_nameservers   | 8.8.4.4                        |
| enable_dhcp       | True                           |
| gateway_ip        | 203.0.113.1                    |
| host_routes       |                                |
| id                | edd00b24-5537-4dcd-            |
|                   | abd9-2ec6006afd41              |
| ip_version        | 4                              |
| ipv6_address_mode |                                |
| ipv6_ra_mode      |                                |
| name              | provider                       |
| network_id        | 918f9709-917c-4665-a117-1f7ea7 |
|                   | c33b49                         |
| subnetpool_id     |                                |
| tenant_id         | 1ab2b4aef473437facd40c17255061 |
|                   | d8                             |
| updated_at        | 2017-08-16T07:35:51            |
+-------------------+--------------------------------+创建m1.nano规格的主机
openstack flavor create --id 0 --vcpus 1 --ram 64 --disk 1 m1.nano
+----------------------------+---------+
| Field                      | Value   |
+----------------------------+---------+
| OS-FLV-DISABLED:disabled   | False   |
| OS-FLV-EXT-DATA:ephemeral  | 0       |
| disk                       | 1       |
| id                         | 0       |
| name                       | m1.nano |
| os-flavor-access:is_public | True    |
| ram                        | 64      |
| rxtx_factor                | 1.0     |
| swap                       |         |
| vcpus                      | 1       |
+----------------------------+---------+

生成一个键值对

导入租户demo的凭证

$ . demo-openrc

生成和添加秘钥对:

$ ssh-keygen -q -N ""
$ openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey
+-------------+--------------------------------------+
| Field       | Value                                |
+-------------+--------------------------------------+
| fingerprint | e4:0f:ce:1f:7a:ac:5b:eb:98:c3:32:b5: |
|             | 9a:42:4e:22                          |
| name        | mykey                                |
| user_id     | 33a3c544d28c4c3bb8cb1291d224e7b2     |
+-------------+--------------------------------------+验证公钥的添加:
$ openstack keypair list
+-------+--------------------------------------------+
| Name  | Fingerprint                                |
+-------+--------------------------------------------+
| mykey | e4:0f:ce:1f:7a:ac:5b:eb:98:c3:32:b5:9a:42: |
|       | 4e:22                                      |
+-------+--------------------------------------------+

增加安全组规则

添加规则到 default 安全组。

允许 ICMP (ping):
openstack security group rule create --proto icmp default
+-----------------------+----------------------------+
| Field                 | Value                      |
+-----------------------+----------------------------+
| id                    | 47c78a30-02cc-4527-964e-   |
|                       | baff81f5ee95               |
| ip_protocol           | icmp                       |
| ip_range              | 0.0.0.0/0                  |
| parent_group_id       | f9c56544-0157-4227-a6af-   |
|                       | 3642b6b0f341               |
| port_range            |                            |
| remote_security_group |                            |
+-----------------------+----------------------------+允许安全 shell (SSH) 的访问:
openstack security group rule create --proto tcp --dst-port 22 default
+-----------------------+----------------------------+
| Field                 | Value                      |
+-----------------------+----------------------------+
| id                    | 78806c49-2591-4d5d-        |
|                       | a8e0-6533f7af5d56          |
| ip_protocol           | tcp                        |
| ip_range              | 0.0.0.0/0                  |
| parent_group_id       | f9c56544-0157-4227-a6af-   |
|                       | 3642b6b0f341               |
| port_range            | 22:22                      |
| remote_security_group |                            |
+-----------------------+----------------------------+

确定实例选项

在控制节点上,获得 admin 凭证来获取只有管理员能执行的命令的访问权限:
$ . demo-openrc一个实例指定了虚拟机资源的大致分配,包括处理器、内存和存储。
列出可用类型:
$ openstack flavor list
+----+----------+-------+------+-----------+-------+-----------+
| ID | Name     |   RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+----------+-------+------+-----------+-------+-----------+
| 0  | m1.nano  |    64 |    1 |         0 |     1 | True      |
| 1  | m1.tiny  |   512 |    1 |         0 |     1 | True      |
| 2  | m1.small |  2048 |   20 |         0 |     1 | True      |
| 3  | m1.mediu |  4096 |   40 |         0 |     2 | True      |
|    | m        |       |      |           |       |           |
| 4  | m1.large |  8192 |   80 |         0 |     4 | True      |
| 5  | m1.xlarg | 16384 |  160 |         0 |     8 | True      |
|    | e        |       |      |           |       |           |
+----+----------+-------+------+-----------+-------+-----------+列出可用镜像:
$ openstack image list
+--------------------------------------+--------+--------+
| ID                                   | Name   | Status |
+--------------------------------------+--------+--------+
| 03b18764-941d-419c-894d-db3f1f4f73e0 | cirros | active |
+--------------------------------------+--------+--------+
列出可用网络:
$ openstack network list
+--------------------------+----------+---------------------------+
| ID                       | Name     | Subnets                   |
+--------------------------+----------+---------------------------+
| 918f9709-917c-4665-a117- | provider | edd00b24-5537-4dcd-       |
| 1f7ea7c33b49             |          | abd9-2ec6006afd41         |
+--------------------------+----------+---------------------------+
列出可用的安全组:
$ openstack security group list
+-----------------+---------+-----------------+-------------------+
| ID              | Name    | Description     | Project           |
+-----------------+---------+-----------------+-------------------+
| f9c56544-0157-4 | default | Default         | ecced8c00e8249edb |
| 227-a6af-       |         | security group  | 3c5730a8a6a4cd1   |
| 3642b6b0f341    |         |                 |                   |
+-----------------+---------+-----------------+-------------------+

启动实例:

openstack server create --flavor m1.nano --image cirros --nic net-id=918f9709-917c-4665-a117-1f7ea7c33b49  --security-group default  --key-name mykey provider-instance检查实例的状态:
$ openstack server list使用虚拟控制台访问实例
获取你实例的 Virtual Network Computing (VNC) 会话URL并从web浏览器访问它:openstack console url show provider-instance

到此简单私有云就搭建成功啦~~

openstack实现私有云的搭建相关推荐

  1. OpenStack ussuri 私有云平台搭建

    一.OpenStack简介 openstack是一个云操作系统,这个操作系统控制着数据中心中的计算,存储和网络资源.所有这些资源的管理都是通过API来来实现的,并且管理资源都有相应的认证机制. 在op ...

  2. OpenStack 企业私有云的若干需求(7):电信行业解决方案 NFV

    本文转自网络文章,内容均为非盈利,一切版权原作者所有. 文章内容仅代表原作者独立观点,不代表本账号立场,转载此文章在于个人学习收藏,传递更多知识. 如有侵权,马上删除. 原文作者:世民谈云计算(微信公 ...

  3. OpenStack Icehouse私有云实战部署

    linux运维 OpenStack Icehouse私有云实战部署 前言 相信你一定对"云主机"一词并不陌生吧,通过在Web页面选择所需主机配置,即可快速定制一台属于自己的虚拟主机 ...

  4. 华为私有云的搭建方案_最简单的纯软件私有云搭建方案,我来教你

    目前国内各大免费网盘纷纷关停,很多用户对存储空间的需求变得愈发迫切.对于普通用户而言,选择网盘的首要因素是成本及安全的问题,拿最常用的某度云来说,非会员用户不管是容量,还是带宽,都有相当多地限制,而且 ...

  5. OpenStack 企业私有云的若干需求(1):Nova 虚机支持 GPU

    本系列会介绍OpenStack 企业私有云的几个需求: 自动扩展(Auto-scaling)支持 多租户和租户隔离 (multi-tenancy and tenancy isolation) 混合云( ...

  6. 使用ipv6内网穿透,实现私有云盘搭建,实现远程控制等功能

    文章目录 问题 获得计算机的ipv6地址 ipv6变化问题 解决-桌面远程控制 ipv6控制路由器 解决-私有云盘搭建 创建服务端B的环境配置 创建服务端可以访问的用户账户 配置服务器对ipv6地址访 ...

  7. 物联网iot私有云平台搭建

    物联网iot私有云平台搭建 物联网的平台有多种,把其中的一些列出一个开源平台比照表. IoT 软件平台 设备管理? 集成 安全 数据收集协议 分析 支持可视化? 数据库 Kaa IoT Platfor ...

  8. 私有云服务器搭建及ssh连接

    私有云服务器搭建及ssh连接 一.私有云服务器搭建 1. 镜像选择 因为我这次搭建服务器就是为了项目部署,所以镜像我选择的是Ubuntu Server 18.04,大家可以去Ubuntu官网进行下载. ...

  9. OpenStack 企业私有云的若干需求(2):自动扩展支持

    本系列会介绍OpenStack 企业私有云的几个需求: 自动扩展(Auto-scaling)支持 多租户和租户隔离 (multi-tenancy and tenancy isolation) 混合云( ...

最新文章

  1. mysql join not in_MySQL 使用左连接替换not in
  2. 致开发者:2018年AI技术趋势展望
  3. java s0 s1_Java GC 变量含义(S0 S1 E O P YGC YGCT FGC FGCT GCT)详解
  4. @jsonProperty 实现返回自定义属性名字
  5. 两个“不合理继承 ”的判定标识
  6. C语言数据类型大学霸IT达人
  7. The connected J-Link is defective,Proper operation cannot be guaranteed......的解决办法
  8. 2020年中国医疗数据中心市场规模及发展趋势预测分析
  9. 信息服务器怎么填写,如何设定服务器信息
  10. html中代码执行顺序
  11. HDFS的副本存放策略(机架感知策略)
  12. 云计算:企业商业模式创新的新战线
  13. 【Java】用for循环实现1+2+3......+100 =
  14. 关于ElasticSearch整合SpringBoot
  15. 28181之安装SPVMN的视频插件
  16. ppt模板免费下载的网站有哪些?这个宝藏网站必须make
  17. Android addr2line 工具使用
  18. 边沿触发是什么意思_边沿触发器的动作特点及主要特点
  19. yyds,Python爬虫从小白到Bigboss全套学习路线+视频+资料
  20. 清华社英语在线自动教程python版

热门文章

  1. C51_day3:入门项目电动车报警器
  2. QT qDebug打印
  3. CAD二次开发--初始欢迎界面无法加载展示问题解决办法【启动欢迎窗体在CAD界面下层无法展示问题】
  4. android6.0恢复出厂设置
  5. jQuery源码分析(二)——Sizzle
  6. 淘宝开放平台天猫国际订单关于价格的研究
  7. 天池零基础入门NLP - 新闻文本分类Top1方案的bert4torch复现
  8. DIGITALWORLD.LOCAL-BRAVERY靶场
  9. QT重写QPushButton的背景图片及填充
  10. vb.net TextBox 失去焦点和获得焦点