dbt 是一个很不错的进行etl 中的t 处理的工具，灵活简单，我们需要写的就是select 语句

dbt 帮助我们进行处理

测试集成了graphql 以及使用docker 运行

安装

pip install dbt

创建简单demo

使用dbt 处理

dbt init demoapp

配置profile(pg 数据库)

~/.dbt/profiles.yml

default:

target: dev

outputs:

dev:

type: postgres

host: 127.0.0.1

user: root

pass: password

port: 5432

dbname: dbt

schema: dbt

threads: 3

pg:

target: dev

outputs:

dev:

type: postgres

host: 127.0.0.1

user: root

pass: password

port: 5433

dbname: dbt

schema: dbt

graphql 集成

docker-compose 文件

version: "3"

services:

graphql-engine:

image: hasura/graphql-engine:v1.0.0-alpha24

ports:

- "8080:8080"

depends_on:

- database

environment:

- "POSTGRES_PASSWORD:password"

command: >

/bin/sh -c "

graphql-engine --database-url postgres://root:password@database:5432/dbt serve --enable-console;

"

database:

image: postgres

environment:

POSTGRES_USER: "root"

POSTGRES_PASSWORD: "password"

POSTGRES_DB: "dbt"

ports:

- "5432:5432"

database2:

image: postgres

environment:

POSTGRES_USER: "root"

POSTGRES_PASSWORD: "password"

POSTGRES_DB: "dbt"

ports:

- "5433:5432"

修改简单model

models/example/my_first_dbt_model.sql

-- Welcome to your first dbt model!

-- Did you know that you can also configure models directly within

-- the SQL file? This will override configurations stated in dbt_project.yml

-- Try changing 'view' to 'table', then re-running dbt

{{ config(materialized='view') }}

select 1 as id, 'v1' as version,'default' as v,'default' as v2

union all

select 2,'v2','default','default'

union all

select 3,'v3','default','default'

union all

select 4,'v4','default','default'

union all

select 5,'v5','default','default'

union all

select 6,'v6','default','default'

union all

select 7,'v7','default','default'

union all

select 8,'v8','default','default'

union all

select 9,'v9','default','default'

运行

docker-compose 启动

docker-compose up -d

运行dbt

dbt fun

效果

支持graphql 配置

配置即可

数据查询

参考资料