1. 集群环境搭建

1.1 集群规划

版本：flink-1.13.1-bin-scala_2.11.tgz

节点服务器	spark01	spark02	spark03
角色	JobManager、TaskManager	TaskManager	TaskManager

1.2 下载并解压安装包

[点击下载](Downloads | Apache Flink)将jar包上传到spark01节点服务器的/opt/module路径上。
使用tar -zxvf解压

1.3 修改集群配置

# 1. 修改flink-conf.yml文件
vi flink-conf.yml
# JobManager节点地址.
jobmanager.rpc.address: spark01
jobmanager.bind-host: 0.0.0.0
rest.address: spark01
rest.bind-address: 0.0.0.0
# TaskManager节点地址.需要配置为当前机器名
taskmanager.bind-host: 0.0.0.0
taskmanager.host: spark01

# 2. vi workers
spark01
spark02
spark03

# 3. vi masters
spark01:8081

1.4 分发安装目录

# 1. xsync 目录

# 2. 在spark02中操作 vi flink-conf.yaml
# TaskManager节点地址.需要配置为当前机器名
taskmanager.host: spark02

# 3. 在spark03中操作 vi flink-conf.yaml
# TaskManager节点地址.需要配置为当前机器名
taskmanager.host: spark03

1.5 启动集群

# 1. 在spark01服务器上执行start-cluster.sh启动flink集群
bin/start-cluster.sh
# 2. 查看启动情况
[root@spark01 flink-1.13.1]# xcall.sh jps
---------------spark01----------------
2960 StandaloneSessionClusterEntrypoint
3381 Jps
3308 TaskManagerRunner
---------------spark02----------------
2181 Jps
2123 TaskManagerRunner
---------------spark03----------------
2182 Jps
2125 TaskManagerRunner

1.6 访问web UI

[web ui](Apache Flink Web Dashboard)

1.7 向集群提交任务

wub ui提交任务

命令行提交任务

# 0. 开启nc
nc -lk 7777

# 1. 执行jar
bin/flink run -m spark01:8081 -c com.atguigu.wc.SocketStreamWordCount ./FlinkTutorial-1.0-SNAPSHOT.jar

# 2. 查看日志
cat flink-root-standalonesession-0-spark01.out
(hello,1)
(hello,2)
(flink,1)
(hello,3)
(scala,1)

2. docker环境搭建

2.1 准备工作

2.2 编写docker-compose文件

version: '3'

services:
  jobmanager:
    image: flink:1.18.0-scala_2.12
    command: jobmanager
    ports:
      - "8081:8081"
    volumes:
      - ./settings/:/settings
      - ./data/:/data
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager
    restart: always
  taskmanager:
    image: flink:1.18.0-scala_2.12
    command: taskmanager
    depends_on:
      - jobmanager
    scale: 1
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager
        taskmanager.numberOfTaskSlots: 20
    volumes:
      - ./settings/:/settings
      - ./data/:/data
    restart: always