Airflow 安装环境 ： centos7   python3.7.0  mysql-8.0.22

一、安装mysql-8.0.22

二、安装python3.7.0

三、安装配置

[root@hadoop101 python-3.7.0]# yum install gcc

[root@hadoop101 software]# python3 -m pip install --upgrade pip

[root@hadoop101 software]# python3 -m pip --default-timeout=1000 install paramiko

[root@hadoop101 software]# yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel

四、安装airflow

[root@hadoop101 software]# python3 -m pip --default-timeout=1000 install -U apache-airflow[all]

安装所有 Airflow 功能，此过程比较漫长，可只安装基本功能

[root@hadoop101 software]# python3 -m pip --default-timeout=1000 install -U apache-airflow

五、安装pymysql

[root@hadoop101 software]# python3 -m pip install pymysql

六、配置环境变量

[root@hadoop101 software]# vim /etc/profile

#airflow

export AIRFLOW_HOME=/app/airflow

export AIRFLOW_GPL_UNIDECODE=yes

[root@hadoop101 software]# source /etc/profile

七、初始化

1、初始化数据库表(默认使用本地得sqlite数据库)

[root@hadoop101 software]# airflow db init

2、查看其生成文件

[root@hadoop101 software]# cd /app/airflow

[root@hadoop101 airflow]# ls

airflow.cfg  airflow.db  logs  unittests.cfg  webserver_config.py

3、配置MySQL数据库(创建airflow数据库，并创建用户和授权，给airflow访问数据库使用)：

[root@hadoop101 airflow]# mysql -u root -proot

mysql> CREATE DATABASE airflow CHARACTER SET UTF8mb3 COLLATE utf8_general_ci;

Query OK, 1 row affected, 2 warnings (0.15 sec)

4、配置airflow使用LocalExecutor执行器，及使用MySQL数据库：

[root@hadoop101 airflow]# vim airflow/airflow.cfg

# The executor class that airflow should use. Choices include

# SequentialExecutor, LocalExecutor, CeleryExecutor, DaskExecutor, KubernetesExecutor

#executor = SequentialExecutor

executor = LocalExecutor

# The SqlAlchemy connection string to the metadata database.

# SqlAlchemy supports many different database engine, more information

# their website

#sql_alchemy_conn = sqlite:data/airflow/airflow.db

sql_alchemy_conn = mysql+pymysql://root:root@localhost:3306/airflow

5、再次初始化数据库表

[root@hadoop101 airflow]# airflow db init

6、更改MySQL配置

[root@hadoop101 airflow]# vim /etc/my.cnf

[mysqld]

explicit_defaults_for_timestamp=1

或者在数据库中运行一下语句：

set @@global.explicit_defaults_for_timestamp=on;

7、查看创建的airflow数据库表：

mysql> use airflow;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

Database changed

mysql> show tables;

+-------------------------------+

| Tables_in_airflow             |

+-------------------------------+

| ab_permission                 |

| ab_permission_view            |

| ab_permission_view_role       |

| ab_register_user              |

| ab_role                       |

| ab_user                       |

| ab_user_role                  |

| ab_view_menu                  |

| alembic_version               |

| connection                    |

| dag                           |

| dag_code                      |

| dag_pickle                    |

| dag_run                       |

| dag_tag                       |

| import_error                  |

| job                           |

| log                           |

| rendered_task_instance_fields |

| sensor_instance               |

| serialized_dag                |

| sla_miss                      |

| slot_pool                     |

| task_fail                     |

| task_instance                 |

| task_reschedule               |

| variable                      |

| xcom                          |

+-------------------------------+

28 rows in set (0.00 sec)

8、创建用户和登录密码

[root@hadoop101 airflow]# airflow users create --username admin --firstname admin --lastname admin --role Admin --email xxx@163.com

[2021-01-13 23:57:21,825] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.

Password:

Repeat for confirmation:

Admin user admin created

八、服务启动

1、添加airflow-scheduler服务启动脚本：

[root@hadoop101 airflow]# airflow webserver

[root@hadoop101 airflow]# airflow scheduler

输入用户密码

安装完成啦~~撒花~~别忘了一键三连哦~~