
Deploy Airflow on Your Own Server
A tool to express and execute workflows as directed acyclic graphs (DAGs).
⚡ Automation
想更快部署 Airflow到服务器?试试 Senate,一条命令完成部署,自动配置 HTTPS 和监控。
什么是 Airflow?
Apache Airflow is a tool to express and execute workflows as directed acyclic graphs (DAGs). It includes utilities to schedule tasks, monitor task progress and handle task dependencies. This instance uses the Bitnami images
自托管 Airflow
按照以下步骤在您的服务器上部署。
1
安装 Docker
如果服务器上已安装 Docker,可跳过此步骤。
curl -fsSL https://get.docker.com | sh
2
配置
自定义以下设置。大多数情况下使用默认值即可。
3
部署
复制配置并在服务器上运行。
services:
airflow:
image: bitnamilegacy/airflow:2.3.3
environment:
AIRFLOW_BASE_URL: https://airflow.your-domain.com/
AIRFLOW_DATABASE_HOST: airflow-db
AIRFLOW_DATABASE_NAME: airflow_db
AIRFLOW_DATABASE_PASSWORD: 307ac85cc944958b
AIRFLOW_DATABASE_USERNAME: airflow_db_user
AIRFLOW_EMAIL: ""
AIRFLOW_EXECUTOR: CeleryExecutor
AIRFLOW_LOAD_EXAMPLES: no
AIRFLOW_PASSWORD: afc5f9156747ed4f
AIRFLOW_SECRET_KEY: 5b3897804f8695e0
AIRFLOW_USERNAME: airflow_user
AIRFLOW_WEBSERVER_HOST: airflow
AIRFLOW__SMTP__SMTP_HOST: ""
AIRFLOW__SMTP__SMTP_MAIL_FROM: ""
AIRFLOW__SMTP__SMTP_PASSWORD: ""
AIRFLOW__SMTP__SMTP_PORT: ""
AIRFLOW__SMTP__SMTP_SSL: no
AIRFLOW__SMTP__SMTP_STARTTLS: no
AIRFLOW__SMTP__SMTP_USER: ""
REDIS_HOST: airflow-redis
REDIS_PASSWORD: 14d0f6a7799d7dd7
volumes:
- airflow-dags:/opt/bitnami/airflow/dags
- airflow-requstxt:/bitnami/python/requirements.txt
depends_on:
- airflow-db
- airflow-redis
airflow-db:
image: bitnami/postgresql:12.12.0
environment:
POSTGRESQL_DATABASE: airflow_db
POSTGRESQL_PASSWORD: 307ac85cc944958b
POSTGRESQL_USERNAME: airflow_db_user
volumes:
- airflow-postgresql-persistance:/bitnami
airflow-redis:
image: bitnamilegacy/redis:7.0.4
environment:
REDIS_PASSWORD: 14d0f6a7799d7dd7
volumes:
- airflow-redis-persistance:/bitnami
airflow-scheduler:
image: bitnamilegacy/airflow-scheduler:2.3.3
environment:
AIRFLOW_BASE_URL: https://airflow.your-domain.com/
AIRFLOW_DATABASE_HOST: airflow-db
AIRFLOW_DATABASE_NAME: airflow_db
AIRFLOW_DATABASE_PASSWORD: 307ac85cc944958b
AIRFLOW_DATABASE_USERNAME: airflow_db_user
AIRFLOW_EXECUTOR: CeleryExecutor
AIRFLOW_LOAD_EXAMPLES: no
AIRFLOW_SECRET_KEY: 5b3897804f8695e0
AIRFLOW_WEBSERVER_HOST: airflow
REDIS_HOST: airflow-redis
REDIS_PASSWORD: 14d0f6a7799d7dd7
volumes:
- airflow-dags:/opt/bitnami/airflow/dags
- airflow-requstxt:/bitnami/python/requirements.txt
depends_on:
- airflow
airflow-worker:
image: bitnamilegacy/airflow-worker:2.3.3
environment:
AIRFLOW_BASE_URL: https://airflow.your-domain.com/
AIRFLOW_DATABASE_HOST: airflow-db
AIRFLOW_DATABASE_NAME: airflow_db
AIRFLOW_DATABASE_PASSWORD: 307ac85cc944958b
AIRFLOW_DATABASE_USERNAME: airflow_db_user
AIRFLOW_EXECUTOR: CeleryExecutor
AIRFLOW_LOAD_EXAMPLES: no
AIRFLOW_SECRET_KEY: 5b3897804f8695e0
AIRFLOW_WEBSERVER_HOST: airflow
REDIS_HOST: airflow-redis
REDIS_PASSWORD: 14d0f6a7799d7dd7
volumes:
- airflow-data:/opt/bitnami/airflow/data
- airflow-dags:/opt/bitnami/airflow/dags
- airflow-requstxt:/bitnami/python/requirements.txt
depends_on:
- airflow
volumes:
airflow-dags: {}
airflow-requstxt: {}
airflow-postgresql-persistance: {}
airflow-redis-persistance: {}
airflow-data: {}
然后运行:
docker compose up -d
4
访问
启动后,访问您的应用: http://localhost:8080
为什么要自托管 Airflow?
完全数据所有权
数据保存在您的服务器上,无第三方访问。
无月费
只需支付服务器费用,无按席位或用量收费。
完全控制
按您的需求自定义、备份和扩展。
隐私与合规
满足 GDPR、HIPAA 或内部安全要求。
技术细节
Docker 镜像
bitnamilegacy/airflow:${airflow_version}想更快部署?
尝试 Senate!Senate 帮您处理 Docker、HTTPS、更新和监控。一条命令部署 300+ 应用。
在任意 Linux 服务器上安装:
sh -c "$(curl -sSL https://get.senate.sh)"