二进制安装StarRocks 存算一体 3.4.0版本(单机版)基础环境系统环境软件版本IP地址FEBECPU指令集Ubuntu 22.04.1 LTSStarRocks-3.4.010.84.91.10LEADERBEAVX2安装之前须查看系统CPU是否支持AVX2指令集cat /proc/cpuinfo | grep avx2注意：StarRocks 依靠 AVX2 指令集充分发挥其矢量化能力,因此CPU指令集必须存在AVX2否则服务将无法正常工作。环境配置官网地址：https://docs.starrocks.io/zh/docs/deployment/environment_configurations/JDK环境配置安装JDK依赖apt install openjdk-11-jdk配置JDK环境变量export JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64/ >> /etc/profile export PATH=/usr/lib/jvm/java-1.11.0-openjdk-amd64/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin >> /etc/profile export LANG=en_US.UTF8 >> /etc/profile ##配置 LANG 变量 source /etc/profile禁用Transparent Huge Pages(Ubuntu默认就是madvise,可跳过)# 临时变更。 echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/defrag # 永久变更。 cat >> /etc/rc.d/rc.local << EOF if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo madvise > /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo madvise > /sys/kernel/mm/transparent_hugepage/defrag fi EOF禁用 Swap Space#临时关闭 Swap Space。 swapoff -a #永久关闭swapoff #从 /etc/fstab 文件中删除 Swap Space 信息。 [root@starrocks ~]# cat /etc/fstab ........................................................................... #/dev/mapper/rl-swap none swap defaults 0 0 #重启机器后确认 Swap Space 已关闭。 [root@starrocks ~]# free -h total used free shared buff/cache available Mem: 15Gi 583Mi 11Gi 8.0Mi 3.9Gi 14Gi Swap: 0B 0B 0B禁用 SELinux (Ubuntu默认没有SELinux,可跳过)# 临时变更。 setenforce 0 # 永久变更。 sed -i 's/SELINUX=.*/SELINUX=disabled/' /etc/selinux/config sed -i 's/SELINUXTYPE/#SELINUXTYPE/' /etc/selinux/config关闭防火墙(Ubuntu默认防火墙关闭的,可跳过)systemctl stop firewalld.service && systemctl disable firewalld.service内核参数优化cat > /etc/sysctl.conf << EOF vm.overcommit_memory = 1 vm.swappiness = 0 net.ipv4.tcp_abort_on_overflow=1 net.core.somaxconn=1024 vm.max_map_count = 262144 EOF sysctl -p #使修改生效vm.max_map_count = 262144 #高并发配置net.core.somaxconn=1024 #监听 Socket 队列的最大连接请求数为 1024net.ipv4.tcp_abort_on_overflow=1#网络配置vm.swappiness=0 #禁用 Swappiness时区配置timedatectl set-timezone "Asia/Shanghai" hwclock时间同步配置安装chronyapt install chrony配置chronycat > /etc/chrony.conf << EOF pool ntp1.aliyun.com iburst pool ntp2.aliyun.com iburst sourcedir /run/chrony-dhcp driftfile /var/lib/chrony/drift makestep 1.0 3 rtcsync keyfile /etc/chrony.keys ntsdumpdir /var/lib/chrony leapsectz right/UTC logdir /var/log/chrony EOF重启服务生效systemctl restart chronydulimit 设置cat >> /etc/security/limits.conf << EOF * soft nproc 65535 * hard nproc 65535 * soft nofile 655350 * hard nofile 655350 * soft stack unlimited * hard stack unlimited * hard memlock unlimited * soft memlock unlimited EOF cat >> /etc/security/limits.d/20-nproc.conf << EOF * soft nproc 65535 root soft nproc 65535 EOF检查端口是否被占用netstat -ntpl | grep "9060\|8040\|9050\|8060\|9070\|8030\|9020\|9030\|9010" be端口: 9060 8040 9050 8060 9070fe端口: 8030 9020 9030 9010其他echo 120000 > /proc/sys/kernel/threads-max echo 200000 > /proc/sys/kernel/pid_max安装StarRocks 3.4.0官方地址：https://docs.starrocks.io/zh/docs/deployment/deploy_manually/1. 下载并解压至工作目录curl -L https://releases.starrocks.io/starrocks/StarRocks-3.4.0-ubuntu-amd64.tar.gz | tar -xz -C /apps/ && \ mv /apps/StarRocks-3.4.0-ubuntu-amd64 /apps/StarRocks2. 创建FE和BE数据目录mkdir /data/3. 配置FE文件cat /apps/StarRocks/fe/conf/fe.conf LOG_DIR = ${STARROCKS_HOME}/log DATE = "$(date +%Y%m%d-%H%M%S)" JAVA_OPTS="-Dlog4j2.formatMsgNoLookups=true -Xmx32768m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:${LOG_DIR}/fe.gc.log.$DATE -XX:+PrintConcurrentLocks" JAVA_OPTS_FOR_JDK_11="-Dlog4j2.formatMsgNoLookups=true -Xmx30720m -XX:+UseG1GC -Xlog:gc*:${LOG_DIR}/fe.gc.log.$DATE:time -Djava.security.policy=${STARROCKS_HOME}/conf/udf_security.policy" sys_log_level = INFO meta_dir = /data/starrocks/fe/meta/ http_port = 8030 rpc_port = 9020 query_port = 9030 edit_log_port = 9010 mysql_service_nio_enabled = true priority_networks = 10.84.0.0/244. 创建 StarRocks FE 启动服务（systemd）cat /etc/systemd/system/starrocks-fe.service [Unit] Description=starrocks-fe daemon [Service] Type=simple ExecStart=/bin/bash /apps/StarRocks/fe/bin/start_fe.sh ExecStop=/bin/bash /apps/StarRocks/fe/bin/stop_fe.sh Restart=always [Install] WantedBy=multi-user.target5. 启动FE服务systemctl start starrocks-fe.service && systemctl enable starrocks-fe.service6. 配置BE文件cat /apps/StarRocks/be/conf/be.conf sys_log_level = INFO be_port = 9060 be_http_port = 8040 heartbeat_service_port = 9050 brpc_port = 8060 starlet_port = 9070 default_replication_num = 1 priority_networks = 10.84.0.0/24 storage_root_path = /data/starrocks/be/只想部署一个 BE 节点，必须在 FE 配置文件 fe/conf/fe.conf 中设置 default_replication_num 为 1。default_replication_num = 17. 创建 StarRocks BE 启动服务（systemd）cat /etc/systemd/system/starrocks-be.service [Unit] Description=starrocks-be daemon [Service] Type=simple ExecStart=/bin/bash /apps/StarRocks/be/bin/start_be.sh ExecStop=/bin/bash /apps/StarRocks/be/bin/stop_be.sh Restart=always [Install] WantedBy=multi-user.target8. 启动BE服务systemctl start starrocks-be.service && systemctl enable starrocks-be.service9. 验证服务是否正常启动netstat -ntpl | grep "9060\|8040\|9050\|8060\|9070\|8030\|9020\|9030\|9010" tcp 0 0 0.0.0.0:9050 0.0.0.0:* LISTEN 3568/starrocks_be tcp 0 0 0.0.0.0:9060 0.0.0.0:* LISTEN 3568/starrocks_be tcp 0 0 0.0.0.0:8060 0.0.0.0:* LISTEN 3568/starrocks_be tcp 0 0 0.0.0.0:8040 0.0.0.0:* LISTEN 3568/starrocks_be tcp6 0 0 :::9030 :::* LISTEN 35940/java tcp6 0 0 :::9070 :::* LISTEN 3568/starrocks_be tcp6 0 0 :::9010 :::* LISTEN 35940/java tcp6 0 0 :::9020 :::* LISTEN 35940/java tcp6 0 0 :::8030 :::* LISTEN 35940/java 部署后操作1. 安装MySQL客户端apt install mysql-client-core-8.02. 无密码登录StarRocksmysql -h 10.84.91.10 -P9030 -uroot --prompt="StarRocks"3. 设置root密码set PASSWORD=PASSWORD('J.K6%\;r*n#Z,5BW_xXp*');4. SQL查询优化SET GLOBAL group_concat_max_len = 2048000; set enable_query_cache=true; set GLOBAL pipeline_dop=3; set enable_profile = true; group_concat_max_len:全局设置group_concat 最大值enable_query_cache:是否开启 Query Cache,取值范围：true 和 false,true 表示开启，false 表示关闭（默认值）,开启该功能后，只有当查询满足Query Cache 所述条件时，才会启用 Query Cache。pipeline_dop:一个 Pipeline 实例的并行数量。可通过设置实例的并行数量调整查询并发度。默认值为 0，即系统自适应调整每个 pipeline 的并行度。您也可以设置为大于 0 的数值，通常为 BE 节点 CPU 物理核数的一半。从 3.0 版本开始，支持根据查询并发度自适应调节 pipeline_dop,有多少副本就配置多少。enable_profile：用于设置是否需要查看查询的 profile。默认为 false，即不需要查看 profile。默认情况下，只有在查询发生错误时，BE 才会发送 profile 给 FE，用于查看错误。正常结束的查询不会发送 profile。发送 profile 会产生一定的网络开销，对高并发查询场景不利。当用户希望对一个查询的 profile 进行分析时，可以将这个变量设为 true 后，发送查询。查询结束后，可以通过在当前连接的 FE 的 web 页面（地址：fe_host:fe_http_port/query）查看 profile。该页面会显示最近 100 条开启了 enable_profile 的查询的 profile。5. 添加be节点至集群单节点ALTER SYSTEM ADD BACKEND "10.84.91.10:9050"多节点ALTER SYSTEM ADD BACKEND "10.84.91.10:9050", "10.84.91.11:9050", "10.84.91.12:9050";6. 验证是否添加成功StarRocks> SHOW PROC '/backends'\G *************************** 1. row *************************** BackendId: 10002 IP: 10.84.91.10 HeartbeatPort: 9050 BePort: 9060 HttpPort: 8040 BrpcPort: 8060 LastStartTime: 2025-03-13 14:54:57 LastHeartbeat: 2025-03-17 10:33:40 Alive: true SystemDecommissioned: false ClusterDecommissioned: false TabletNum: 69 DataUsedCapacity: 0.000 B AvailCapacity: 746.239 GB TotalCapacity: 786.374 GB UsedPct: 5.10 % MaxDiskUsedPct: 5.10 % ErrMsg: Version: 3.4.0-e94580b Status: {"lastSuccessReportTabletsTime":"2025-03-17 10:33:39"} DataTotalCapacity: 746.239 GB DataUsedPct: 0.00 % CpuCores: 8 MemLimit: 50.866GB NumRunningQueries: 0 MemUsedPct: 0.30 % CpuUsedPct: 0.1 % DataCacheMetrics: Status: Normal, DiskUsage: 0B/580GB, MemUsage: 0B/0B Location: StatusCode: OK 1 row in set (0.02 sec)Alive: true:添加成功7. 添加fe节点至集群7.1、手动操作从节点启动时要指定LEADE的IP/apps/StarRocks/fe/bin/start_fe.sh --helper "fe_leader_host:edit_log_port" --daemon7.2、systemd 托管添加至集群方式cat /etc/systemd/system/starrocks-fe.service [Unit] Description=starrocks-fe daemon [Service] Type=simple ExecStart=/bin/bash /apps/StarRocks/fe/bin/start_fe.sh --helper "fe_leader_host:edit_log_port" --daemon ExecStop=/bin/bash /apps/StarRocks/fe/bin/stop_fe.sh Restart=always [Install] WantedBy=multi-user.target7.3、在LEADER节点添加从节点IP和PORT至集群中ALTER SYSTEM ADD FOLLOWER "10.84.91.11:9010","10.84.91.12:9010"因为此处是单机模式，默认自动会将本机fe节点添加进去并设置为LEADER,可跳过，如果是多节点可执行以上操作。8. 验证是否添加成功StarRocks> SHOW FRONTENDS\G; *************************** 1. row *************************** Name: 10.84.91.10_9010_1741848390717 IP: 10.84.91.10 EditLogPort: 9010 HttpPort: 8030 QueryPort: 9030 RpcPort: 9020 Role: LEADER ClusterId: 480437351 Join: true Alive: true ReplayedJournalId: 99143 LastHeartbeat: 2025-03-17 10:33:55 IsHelper: true ErrMsg: StartTime: 2025-03-17 10:32:36 Version: 3.4.0-e94580b 1 row in set (0.04 sec) ERROR: No query specifiedAlive: true 添加成功备份与恢复官方地址：https://docs.starrocks.io/zh/docs/administration/management/Backup_and_restore/StarRocks 支持在以下外部存储系统中备份数据：Apache™ Hadoop® （HDFS）集群AWS S3Google GCS阿里云 OSS腾讯云 COS华为云 OBSMinIOStarRocks 支持备份以下对象：内部数据库、表（所有类型和分区策略）和分区External Catalog 的元数据（自 v3.4.0 开始支持）同步物化视图和异步物化视图逻辑视图（自 v3.4.0 开始支持）UDF（自 v3.4.0 开始支持）部署外部存储(minio)依赖环境1.安装dockerapt install docker.io2.安装docker-composesudo curl -L https://github.com/docker/compose/releases/download/v2.33.1/docker-compose-linux-x86_64 -o /usr/bin/docker-compose && \ sudo chmod +x /usr/bin/docker-compose部署外部存储(minio)1. 部署miniocat > docker-compose.yaml << EOF services: minio: container_name: minio image: minio/minio:latest ports: - "9000:9000" - "9001:9001" environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: Sec@2024 command: minio server /minio_data --console-address ":9001" volumes: - /opt/minio/minio-data:/minio_data healthcheck: test: ["CMD", "curl", "-f", "http://minio:9000/minio/health/live"] interval: 5s timeout: 5s retries: 5 EOF2. 启动miniodocker-compose up -d3. 创建AK和SK3.1、下载Minio Client 工具curl -L https://dl.min.io/client/mc/release/linux-amd64/mc -o /usr/bin/mc && \ sudo chmod +x /usr/bin/mc3.2、设置别名因Minio Client工具的要求,只能用 alias（别名）访问服务端mc alias set StarRocks http://127.0.0.1:9000 minioadmin Sec@2024 && \ mc admin user svcacct add \ --access-key StarRocks \ --secret-key Sec@2024 \ StarRocks \ minioadmin3.3、验证是否创建成功列出 minioadmin 账号下的所有AKmc admin user svcacct list StarRocks minioadmin Access Key | Expiry StarRocks | no-expiry3.4、创建bucketmc mb StarRocks/starrocksmc mb：make bucket，用于创建 bucket；StarRocks：你通过 mc alias set 配置的别名；my-bucket：你要创建的 bucket 名称，可以改成你需要的名字。备份StarRocks数据库1. 创建备份仓库语法格式：https://docs.starrocks.io/zh/docs/sql-reference/sql-statements/backup_restore/CREATE_REPOSITORY/在 MinIO 存储空间 bucket_minio 中创建仓库 test_repo,s3的访问路径结尾不能以/结尾,否则会有报错,备份数据将在starrocks buckets中存放CREATE REPOSITORY test_repo WITH BROKER ON LOCATION "s3://starrocks/test_bak" PROPERTIES( "aws.s3.access_key" = "StarRocks", "aws.s3.secret_key" = "Sec@2024", "aws.s3.endpoint" = "http://10.84.91.10:9000" );2. 验证是否创建成功mysql> SHOW REPOSITORIES; +---------+----------------------------+---------------------+------------+-------------------------------------------+--------+--------+ | RepoId | RepoName | CreateTime | IsReadOnly | Location | Broker | ErrMsg | +---------+----------------------------+---------------------+------------+-------------------------------------------+--------+--------+ | 4561972 | test_repo | 2025-03-17 10:06:06 | false | s3://starrocks/test_bak | | NULL | +---------+----------------------------+---------------------+------------+-------------------------------------------+--------+--------+ 1 rows in set (0.00 sec)3. 备份数据库(全量备份)为数据库 sr_hub 创建数据快照 sr_hub_20240317 并备份至仓库 test_repo 中。-- 自 v3.4.0 起支持。 BACKUP DATABASE sr_hub SNAPSHOT sr_hub_20240317 TO test_repo;验证是否备份成功,备份是异步备份,需等备份完成后才能显示OKmysql> SHOW SNAPSHOT ON test_repo; +-----------------+-------------------------+--------+ | Snapshot | Timestamp | Status | +-----------------+-------------------------+--------+ | sr_hub_20250317 | 2025-03-17-14-45-53-143 | OK | +-----------------+---------------------------+------+ 1 rows in set (0.10 sec)如果不显示OK,可以通过 SHOW BACKUP 语句查看备份作业状态，或通过 CANCEL BACKUP 语句取消备份作业。官方地址：https://docs.starrocks.io/zh/docs/sql-reference/sql-statements/backup_restore/SHOW_BACKUP/mysql> SHOW BACKUP FROM test \G; *************************** 1. row *************************** JobId: 5654170 SnapshotName: sr_hub_20250317 DbName: sr_hub State: FINISHED BackupObjs: [sr_hub.access_people_online], [sr_hub.device_key_data] CreateTime: 2025-03-17 00:00:02 SnapshotFinishedTime: 2025-03-17 00:00:37 UploadFinishedTime: 2025-03-17 00:33:07 FinishedTime: 2025-03-17 00:33:41 UnfinishedTasks: Progress: TaskErrMsg: Status: [OK] Timeout: 86400 1 row in set (0.00 sec)备份作业当前所在阶段：PENDING：作业初始状态。SNAPSHOTING：正在进行快照操作。UPLOAD_SNAPSHOT：快照结束，准备上传。UPLOADING：正在上传快照。SAVE_META：正在本地生成元数据文件。UPLOAD_INFO：上传元数据文件和本次备份作业的信息。FINISHED：备份完成。CANCELLED：备份失败或被取消。4.备份分区(增量备份)4.1、查看分区ID备份分区前,要确定表已经按分区来存储数据了,否则将不支持备份分区示例：查看sr_hub库中的device_key_data分区SHOW PARTITIONS FROM sr_hub.device_key_data;执行结果：如果PartitionName列不是以 p日期 开头的,则表大概率没开启分区,不支持备份分区+-------------+---------------+----------------+---------------------+--------------------+--------+--------------+------------------------------------------------------------------------------------------------------+----------------------------+---------+----------------+---------------+---------------------+--------------------------+----------+------------+----------+-------------+--------------------+----------------+ | PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | VisibleVersionHash | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | LastConsistencyCheckTime | DataSize | IsInMemory | RowCount | DataVersion | VersionEpoch | VersionTxnType | | 5654248 | p20250317 | 2234 | 2025-03-17 14:40:54 | 0 | NORMAL | device_time | [types: [DATETIME]; keys: [2025-03-17 00:00:00]; ..types: [DATETIME]; keys: [2025-03-18 00:00:00]; ) | device_time, device_number | 2 | 1 | HDD | 9999-12-31 23:59:59 | NULL | 76.2MB | false | 9215872 | 2234 | 344570509234536449 | TXN_NORMAL | ......4.2、备份分区-- 自 v3.4.0 起支持。 BACKUP DATABASE sr_hub SNAPSHOT sr_hub_20240317 TO test_repo ON (TABLE sr_hub.device_key_data PARTITION (p20250317));恢复StarRocks数据库从快照还原数据时，必须指定快照的时间戳。数据恢复为异步操作可以通过 SHOW RESTORE 语句查看恢复作业状态，或通过 CANCEL RESTORE 语句取消恢复作业。官方地址：https://docs.starrocks.io/zh/docs/sql-reference/sql-statements/backup_restore/SHOW_RESTORE/恢复作业当前所在阶段：PENDING：作业初始状态。SNAPSHOTING：正在进行本地新建表的快照操作。DOWNLOAD：正在发送下载快照任务。DOWNLOADING：快照正在下载。COMMIT：准备生效已下载的快照。COMMITTING：正在生效已下载的快照。FINISHED：恢复完成。CANCELLED：恢复失败或被取消。1、获取快照时间戳mysql> SHOW SNAPSHOT ON test_repo; +------------------+-------------------------+--------+ | Snapshot | Timestamp | Status | +------------------+-------------------------+--------+ | sr_hub_20240317 | 2025-03-17-14-45-53-143 | OK | +------------------+-------------------------+--------+ 1 row in set (1.16 sec)2、恢复数据库以下示例将快照 sr_hub_20240317 中的数据库 sr_hub 还原到目标群集中的数据库 sr_hub。如果快照中不存在该数据库，系统将返回错误。如果目标群集中不存在该数据库，系统将自动创建该数据库。-- 自 v3.4.0 起支持。 RESTORE SNAPSHOT sr_hub_20240317 FROM test_repo DATABASE sr_hub PROPERTIES( "backup_timestamp" = "2025-03-17-14-45-53-143", "replication_num" = "1" -- 默认副本数为3 如果是单机要指定为1 否则恢复会报错 );3、恢复分区以下示例将快照 sr_hub_20240317 中数据库 sr_hub 下的表 device_key_data 的分区 p20250317 恢复到目标群集中数据库 sr_hub 下的表 device_key_data 的分区 p20250317 中。-- 自 v3.4.0 起支持。 RESTORE SNAPSHOT sr_hub_20240317 FROM test_repo DATABASE sr_hub ON (TABLE device_key_data PARTITION (p20250317)) PROPERTIES ( "backup_timestamp" = "2025-03-17-14-45-53-143", "replication_num" = "1" );4、验证是否恢复成功StarRocks:SHOW RESTORE \G; *************************** 1. row *************************** JobId: 11010 Label: sr_hub_20240317 Timestamp: 2025-03-17-14-45-53-143 DbName: sr_hub State: FINISHED -- 恢复完成 AllowLoad: false ReplicationNum: 1 ................................. ................................. .................................使用快照跨集群迁移数据前提条件有一台能正常工作的StarRocks服务1、创建仓库在新机器中创建仓库,需要注意的是；新机器中的创建仓库指令要和旧机器的创建仓库指令一致。否则会查不到快照信息CREATE REPOSITORY test_repo WITH BROKER ON LOCATION "s3://starrocks/test_bak" PROPERTIES( "aws.s3.access_key" = "StarRocks", "aws.s3.secret_key" = "Sec@2024", "aws.s3.endpoint" = "http://10.84.91.10:9000" );2、验证是否创建成功mysql> SHOW REPOSITORIES; +---------+----------------------------+---------------------+------------+-------------------------------------------+--------+--------+ | RepoId | RepoName | CreateTime | IsReadOnly | Location | Broker | ErrMsg | +---------+----------------------------+---------------------+------------+-------------------------------------------+--------+--------+ | 4561972 | test_repo | 2025-03-17 10:06:06 | false | s3://starrocks/test_bak | | NULL | +---------+----------------------------+---------------------+------------+-------------------------------------------+--------+--------+ 1 rows in set (0.00 sec)验证时的创建信息也会和旧机器的日期一致3、恢复数据库以下示例将快照 sr_hub_20240317 中的数据库 sr_hub 还原到目标群集中的数据库 sr_hub。如果快照中不存在该数据库，系统将返回错误。如果目标群集中不存在该数据库，系统将自动创建该数据库。-- 自 v3.4.0 起支持。 RESTORE SNAPSHOT sr_hub_20240317 FROM test_repo DATABASE sr_hub PROPERTIES( "backup_timestamp" = "2025-03-17-14-45-53-143", "replication_num" = "1" -- 默认副本数为3 如果是单机要指定为1 否则恢复会报错 );4、验证是否恢复成功StarRocks:SHOW RESTORE \G; *************************** 1. row *************************** JobId: 11010 Label: sr_hub_20240317 Timestamp: 2025-03-17-14-45-53-143 DbName: sr_hub State: FINISHED -- 恢复完成 AllowLoad: false ReplicationNum: 1 ................................. ................................. .................................
导出表数据至MiniOEXPORT TABLE users TO "s3://backup/" WITH BROKER ( "aws.s3.endpoint" = "http://192.168.8.197:9000", "aws.s3.access_key" = "backup", "aws.s3.secret_key" = "backup1111" );users: 是tables名字s3://backup/：Bucket名字aws.s3.endpoint ：对象存储地址aws.s3.access_key：访问对象存储时所用到的AK（Access Key）aws.s3.secret_key：访问对象存储时所用到的SK(Secret Key)从MiniO中导入表数据1、先创建表结构找研发要创建表结构的SQL语句2、使用LOAD BROKER方式导入minio中的CSV格式的表数据LOAD LABEL prod ( DATA INFILE("s3://greenpioneeriotuat/data_b38cbe2a-4a27-11ef-9277-0242ac180003_0_0.csv") INTO TABLE total_combine_active_energy ) WITH BROKER ( "aws.s3.endpoint" = "http://10.84.3.128:9000", "aws.s3.enable_ssl" = "false", "aws.s3.access_key" = "backup", "aws.s3.secret_key" = "backup1111", "aws.s3.use_aws_sdk_default_behavior" = "false", "aws.s3.use_instance_profile" = "false", "aws.s3.enable_path_style_access" = "true" ) PROPERTIES ( "timeout" = "72000" );3、检查是否导入成功或者导入进度SELECT * FROM information_schema.loads;total_combine_active_energy: 导入数据后生成的表名字aws.s3.endpoint ：对象存储地址DATA INFILE：数据源路径，上方例子是从对象存储的greenpioneeriotuat桶中导入data_b38cbe2a-4a27-11ef-9277-0242ac180003_0_0.csv文件aws.s3.enable_ssl：是否启用ssl加密方式传输aws.s3.access_key：访问对象存储时所用到的AK（Access Key）aws.s3.secret_key：访问对象存储时所用到的SK(Secret Key)aws.s3.use_aws_sdk_default_behavior：不使用AWS SDK的默认行为，AWS SDK有其预设的认证和配置机制aws.s3.use_instance_profile：是否使用EC2实例的IAM角色（Instance Profile）作为认证来源aws.s3.enable_path_style_access：使用路径样式访问S3timeout：导入数据时的超时时间，单位s从本地导入表数据curl --location-trusted -u root -H "label:label1" \ -H "Expect:100-continue" \ -H "column_separator:|" \ -H "max_filter_ratio:0.2" \ -T ./data_dir/lineorder.tbl -XPUT \ -H "timeout: 86400" \ http://192.168.8.198:8030/api/ssb/lineorder/_stream_load注意：本地导入，前提是表结构提前准备好--location-trusted：curl信任重定向，为了处理可能的重定向请求，通常在内部网络中使用。-u root: HTTP基本认证，其中root是用于认证的用户名。密码通常在-u之后用冒号分隔，或者curl会提示输入密码。-H "label:label1": HTTP头，其中label是流式加载任务的唯一标识符。它可以帮助追踪和管理导入任务的状态。-H "Expect:100-continue": 这个HTTP头告诉服务器客户端准备好发送数据体，等待服务器确认后再继续发送数据。这在发送大文件时有助于节省带宽和处理时间。-H "column_separator:|": 这个头指定了数据文件中列与列之间的分隔符。在这个例子中，列是用竖线|分隔的。-H "max_filter_ratio:0.2": 这个头设置了一个过滤比率，用于控制数据加载过程中数据过滤的程度。例如，0.2意味着最多可以过滤掉20%的数据行。-T ./data_dir/lineorder.tbl: 这个选项指定了要上传的本地文件。在这里，文件是./data_dir/lineorder.tbl。-XPUT: 这个选项指定了HTTP方法为PUT，这意味着客户端希望替换或更新服务器上的资源。-H "timeout: 86400": 这个头设置了请求的超时时间，单位是秒。86400秒等于一天，这意味着请求将在一天后超时。http://192.168.8.198:8030/api/ssb/lineorder/_stream_load: 这是目标URL，指向数据导入的API端点。在这个例子中，数据将被发送到IP地址192.168.8.198上的8030端口，具体到/api/ssb/lineorder/_stream_load路径，这通常是用于处理SSB（Star Schema Benchmark）数据的流式加载接口。
基础环境系统环境软件版本IP地址FECNCPU指令集Rocky Linux release 9.4StarRocks-3.3.0192.168.8.198LEADERCNAVX2Rocky Linux release 9.4StarRocks-3.3.0192.168.8.195FOLLOWERCNAVX2Rocky Linux release 9.4StarRocks-3.3.0192.168.8.194FOLLOWERCNAVX2Ubuntu 20.04minio:latest192.168.8.197nonenonenone安装之前须查看系统CPU是否支持AVX2指令集cat /proc/cpuinfo | grep avx2注意：StarRocks 依靠 AVX2 指令集充分发挥其矢量化能力,因此CPU指令集必须存在AVX2否则服务将无法正常工作。环境配置官网地址：https://docs.starrocks.io/zh/docs/deployment/environment_configurations/JDK环境配置dnf install java-11-openjdk-devel echo "export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-11.0.23.0.9-3.el9.x86_64/" >> /etc/profile echo "export PATH=/usr/lib/jvm/java-11-openjdk-11.0.23.0.9-3.el9.x86_64/bin/:$PATH" >> /etc/profile echo "export LANG=en_US.UTF8" >> /etc/profile #配置 LANG 变量 source /etc/profile禁用Transparent Huge Pages# 临时变更。 echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/defrag # 永久变更。 cat >> /etc/rc.d/rc.local << EOF if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo madvise > /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo madvise > /sys/kernel/mm/transparent_hugepage/defrag fi EOF禁用 Swap Space#临时关闭 Swap Space。 swapoff -a #永久关闭swapoff #从 /etc/fstab 文件中删除 Swap Space 信息。 [root@starrocks ~]# cat /etc/fstab ........................................................................... #/dev/mapper/rl-swap none swap defaults 0 0 #重启机器后确认 Swap Space 已关闭。 [root@starrocks ~]# free -h total used free shared buff/cache available Mem: 15Gi 583Mi 11Gi 8.0Mi 3.9Gi 14Gi Swap: 0B 0B 0B禁用 SELinux# 临时变更。 setenforce 0 # 永久变更。 sed -i 's/SELINUX=.*/SELINUX=disabled/' /etc/selinux/config sed -i 's/SELINUXTYPE/#SELINUXTYPE/' /etc/selinux/config关闭防火墙systemctl stop firewalld.service && systemctl disable firewalld.service内核参数优化cat > /etc/sysctl.conf << EOF vm.overcommit_memory = 1 vm.swappiness = 0 net.ipv4.tcp_abort_on_overflow=1 net.core.somaxconn=1024 vm.max_map_count = 262144 EOF #使修改生效 sysctl -pvm.max_map_count = 262144 #高并发配置net.core.somaxconn=1024 #监听 Socket 队列的最大连接请求数为 1024net.ipv4.tcp_abort_on_overflow=1 #网络配置vm.swappiness=0 #禁用 Swappiness时区配置timedatectl set-timezone "Asia/Shanghai" hwclock时间同步配置cat > /etc/chrony.conf << EOF pool ntp1.aliyun.com iburst pool ntp2.aliyun.com iburst sourcedir /run/chrony-dhcp driftfile /var/lib/chrony/drift makestep 1.0 3 rtcsync keyfile /etc/chrony.keys ntsdumpdir /var/lib/chrony leapsectz right/UTC logdir /var/log/chrony EOF systemctl restart chronydulimit 设置cat >> /etc/security/limits.conf << EOF * soft nproc 65535 * hard nproc 65535 * soft nofile 655350 * hard nofile 655350 * soft stack unlimited * hard stack unlimited * hard memlock unlimited * soft memlock unlimited EOF cat >> /etc/security/limits.d/20-nproc.conf << EOF * soft nproc 65535 root soft nproc 65535 EOF其他echo 120000 > /proc/sys/kernel/threads-max echo 200000 > /proc/sys/kernel/pid_max建议环境设置好后，重启下机器，然后再进行部署！！！部署StarRocksMinio1.安装dockerapt install docker.io docker-compose -y2.部署miniocat > minio.yaml << EOF version: "3" services: minio: container_name: minio environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: Sec@2024 image: minio/minio:latest ports: - "9001:9001" - "9000:9000" command: minio server /minio_data --console-address ":9001" volumes: - "/opt/minio/minio-data/:/minio_data" healthcheck: test: ["CMD", "curl", "-f", "http://minio:9000/minio/health/live"] interval: 5s timeout: 5s retries: 5 EOF3.创建ak、sk{"url":"http://10.84.3.128:9001/api/v1/service-account-credentials","accessKey":"starrocks","secretKey":"Sec@2024","api":"s3v4","path":"auto"}FE1. 下载并解压至工作目录wget https://releases.starrocks.io/starrocks/StarRocks-3.3.0.tar.gz && tar zxvf StarRocks-3.3.0.tar.gz -C /opt/2. 创建FE元数据目录mkdir /apps/data/fe/meta -p3. 配置FE文件cat > /opt/StarRocks-3.3.0/fe/conf/fe.conf << EOF run_mode = shared_data priority_networks = 192.168.0.0/24 cloud_native_meta_port = 6090 enable_load_volume_from_conf = true cloud_native_storage_type = S3 meta_dir = /apps/data/fe/meta #FE元数据目录 aws_s3_path = starrocks #Bucket名称 aws_s3_endpoint = http://192.168.8.197:9000 #对象存储IP:PORT aws_s3_region = starrocks aws_s3_access_key = starrocks #AK aws_s3_secret_key = Sec@2024 #SK LOG_DIR = ${STARROCKS_HOME}/log DATE = "$(date +%Y%m%d-%H%M%S)" JAVA_OPTS="-Dlog4j2.formatMsgNoLookups=true -Xmx8192m -XX:+UseG1GC -Xlog:gc*:${LOG_DIR}/fe.gc.log.$DATE:time -XX:ErrorFile=${LOG_DIR}/hs_err_pid%p.log -Djava.security.policy=${STARROCKS_HOME}/conf/udf_security.policy" sys_log_level = INFO http_port = 8030 rpc_port = 9020 query_port = 9030 edit_log_port = 9010 mysql_service_nio_enabled = true # 最小的Compaction score，低于该值的 Partition 不会发起Compaction任务 lake_compaction_score_selector_min_score = 10.0 # FE 上可同时发起的 Compaction Task 数量 # 默认值为-1，即FE会根据系统中 BE 数量自动计算 # 如果设置为0，则 FE 不会发起任何 Compaction 任务 lake_compaction_max_tasks = -1 # 控制show proc '/compactions' 显示的结果数量，默认为12 lake_compaction_history_size = 12 lake_compaction_fail_history_size = 12 EOF4. 启动主节点FE/opt/StarRocks-3.3.0/fe/bin/start_fe.sh --daemon注意：从节点和主节点是一样的配置，但是从节点配置完先不启动，只启动主节点，等配置完成后，启动从节点。 主节点执行 1-4，从节点执行 1-35. 验证是否启动成功[root@starrocks01 ~]# cat /opt/StarRocks-3.3.0/fe/log/fe.log | grep thrift 2024-07-15 09:14:16.241+08:00 INFO (UNKNOWN 192.168.8.198_9010_1721006047505(-1)|1) [FrontendThriftServer.start():65] thrift server started with port 9020.CN1. 创建CN元数据目录mkdir -p /apps/data/cn2. 配置CN文件cat > /opt/StarRocks-3.3.0/be/conf/cn.conf << EOF sys_log_level = INFO be_port = 9060 be_http_port = 8040 heartbeat_service_port = 9050 brpc_port = 8060 starlet_port = 9070 priority_networks = 192.168.0.0/24 storage_root_path = /apps/data/cn cumulative_compaction_num_threads_per_disk = 4 base_compaction_num_threads_per_disk = 2 cumulative_compaction_check_interval_seconds = 2 # 控制 BE/CN 上同时执行 Compaction 任务的线程数，默认值为4 # 也即 BE 上可同时为多少个 Tablet进行 Compaction compact_threads = 4 # BE 上 Compaction任务队列大小，控制可接收来自FE的最大Compaction 任务数 # 默认值为100 compact_thread_pool_queue_size = 100 # 单次 Compaction 任务最多合并的数据文件数量，默认为1000 # 在实践中我们建议将该值调整为100，这样，每个 Compaction Task 可以更快速地结束 # 且消耗更少的资源 max_cumulative_compaction_num_singleton_deltas=100 EOF3. 启动CN/opt/StarRocks-3.3.0/be/bin/start_cn.sh --daemon4. 验证是否启动成功[root@starrocks01 ~]# cat /opt/StarRocks-3.3.0/be/log/cn.INFO | grep heartbeat I0715 09:22:45.866142 22959 thrift_server.cpp:383] heartbeat has started listening port on 9050 I0715 09:22:45.866148 22959 starrocks_be.cpp:282] CN start step 13: start heartbeat server successfully其他节点安装CN可重复1-4步骤，CN三台可一起启动，无先后顺序搭建集群1. 使用MySQL客户端连接FE#安装客户端 apt install mysql-client #连接FE mysql -uroot -P9030 -h 127.0.0.12. 配置 FE 高可用mysql> ALTER SYSTEM ADD FOLLOWER "192.168.8.195:9010"; Query OK, 0 rows affected (0.03 sec) mysql> ALTER SYSTEM ADD FOLLOWER "192.168.8.194:9010"; Query OK, 0 rows affected (0.02 sec)3. 启动FE从节点/opt/StarRocks-3.3.0/fe/bin/start_fe.sh --helper 192.168.8.198:9010 --daemon注意：如果在配置高可用之前启动了从节点FE,配置完高可用后重启时会报错192.168.8.195_9010_1721022086495 weren't! UNEXPECTED_STATE: Unexpected internal state, may have side effects.解决办法：# 清除从FE的meta目录 rm -rf /apps/data/fe/meta/* #重新启动从节点FE /opt/StarRocks-3.3.0/fe/bin/start_fe.sh --helper 192.168.8.198:9010 --daemo4. 查看FE状态mysql> SHOW FRONTENDS\G; *************************** 1. row *************************** Name: 192.168.8.198_9010_1721006047505 IP: 192.168.8.198 EditLogPort: 9010 HttpPort: 8030 QueryPort: 9030 RpcPort: 9020 Role: LEADER ClusterId: 463766615 Join: true Alive: true ReplayedJournalId: 5042 LastHeartbeat: 2024-07-15 13:54:43 IsHelper: true ErrMsg: StartTime: 2024-07-15 09:22:27 Version: 3.3.0-19a3f66 *************************** 2. row *************************** Name: 192.168.8.194_9010_1721021656623 IP: 192.168.8.194 EditLogPort: 9010 HttpPort: 8030 QueryPort: 9030 RpcPort: 9020 Role: FOLLOWER ClusterId: 463766615 Join: true Alive: true ReplayedJournalId: 5040 LastHeartbeat: 2024-07-15 13:54:43 IsHelper: true ErrMsg: StartTime: 2024-07-15 13:53:49 Version: 3.3.0-19a3f66 *************************** 3. row *************************** Name: 192.168.8.195_9010_1721021647617 IP: 192.168.8.195 EditLogPort: 9010 HttpPort: 8030 QueryPort: 9030 RpcPort: 9020 Role: FOLLOWER ClusterId: 463766615 Join: true Alive: true ReplayedJournalId: 5040 LastHeartbeat: 2024-07-15 13:54:43 IsHelper: true ErrMsg: StartTime: 2024-07-15 13:51:06 Version: 3.3.0-19a3f66 3 rows in set (0.02 sec)如果字段 Role 为 FOLLOWER，说明该 FE 节点有资格被选为 Leader FE 节点。如果字段 Role 为 LEADER，说明该 FE 节点为 Leader FE 节点。5. 添加CN节点mysql> ALTER SYSTEM ADD COMPUTE NODE "192.168.8.198:9050"; Query OK, 0 rows affected (0.04 sec) mysql> ALTER SYSTEM ADD COMPUTE NODE "192.168.8.195:9050"; Query OK, 0 rows affected (0.03 sec) mysql> ALTER SYSTEM ADD COMPUTE NODE "192.168.8.194:9050"; Query OK, 0 rows affected (0.02 sec)6. 查看CN状态mysql> SHOW PROC '/compute_nodes'\G *************************** 1. row *************************** ComputeNodeId: 12600 IP: 192.168.8.194 HeartbeatPort: 9050 BePort: 9060 HttpPort: 8040 BrpcPort: 8060 LastStartTime: 2024-07-15 13:12:56 LastHeartbeat: 2024-07-15 14:09:38 Alive: true SystemDecommissioned: false ClusterDecommissioned: false ErrMsg: Version: 3.3.0-19a3f66 CpuCores: 40 NumRunningQueries: 0 MemUsedPct: 0.47 % CpuUsedPct: 0.0 % DataCacheMetrics: Status: Normal, DiskUsage: 0B/0B, MemUsage: 0B/0B HasStoragePath: true StarletPort: 9070 WorkerId: 9 WarehouseName: default_warehouse TabletNum: 17 *************************** 2. row *************************** ComputeNodeId: 12599 IP: 192.168.8.195 HeartbeatPort: 9050 BePort: 9060 HttpPort: 8040 BrpcPort: 8060 LastStartTime: 2024-07-15 13:09:46 LastHeartbeat: 2024-07-15 14:09:38 Alive: true SystemDecommissioned: false ClusterDecommissioned: false ErrMsg: Version: 3.3.0-19a3f66 CpuCores: 20 NumRunningQueries: 0 MemUsedPct: 0.97 % CpuUsedPct: 0.0 % DataCacheMetrics: Status: Normal, DiskUsage: 0B/0B, MemUsage: 0B/0B HasStoragePath: true StarletPort: 9070 WorkerId: 8 WarehouseName: default_warehouse TabletNum: 21 *************************** 3. row *************************** ComputeNodeId: 12521 IP: 192.168.8.198 HeartbeatPort: 9050 BePort: 9060 HttpPort: 8040 BrpcPort: 8060 LastStartTime: 2024-07-15 13:04:21 LastHeartbeat: 2024-07-15 14:09:38 Alive: true SystemDecommissioned: false ClusterDecommissioned: false ErrMsg: Version: 3.3.0-19a3f66 CpuCores: 40 NumRunningQueries: 0 MemUsedPct: 0.12 % CpuUsedPct: 0.0 % DataCacheMetrics: Status: Normal, DiskUsage: 0B/0B, MemUsage: 0B/0B HasStoragePath: true StarletPort: 9070 WorkerId: 1 WarehouseName: default_warehouse TabletNum: 20 3 rows in set (0.00 sec)7. 停止FE/opt/StarRocks-3.3.0/fe/bin/stop_fe.sh --daemon8. 停止CN/opt/StarRocks-3.3.0/be/bin/stop_cn.sh --daemon
环境检查：部署的前提条件：https://docs.starrocks.io/zh/docs/deployment/deployment_prerequisites/StarRocks 依靠 AVX2 指令集充分发挥其矢量化能力。因此，在生产环境中，必须保证AVX2 指令集存在检查AVX2 指令集：root@starrocks002:~# cat /proc/cpuinfo | grep avx2 flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt c lwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves wbnoinvd arat avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid md_clear flush_l1d arch_capabilities flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt c lwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves wbnoinvd arat avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid md_clear flush_l1d arch_capabilities 。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。注意：如果执行后，无打印，说明机器没有AVX2 指令集，运行不了starrocksdocker-compose版本root@starrocks002:~# docker-compose -v Docker Compose version v2.27.2 编排文件cat > docker-compose.yaml << EOF version: "3" services: minio: container_name: minio environment: MINIO_ROOT_USER: miniouser MINIO_ROOT_PASSWORD: miniopassword image: registry.cn-shanghai.aliyuncs.com/qwx_images/minio:latest ports: - "9001:9001" - "9000:9000" command: minio server /minio_data --console-address ":9001" healthcheck: test: ["CMD", "curl", "-f", "http://minio:9000/minio/health/live"] interval: 5s timeout: 5s retries: 5 minio_mc: image: registry.cn-shanghai.aliyuncs.com/qwx_images/mc:latest entrypoint: - sh - -c - | until mc alias set myminio http://minio:9000 miniouser miniopassword; do echo "Waiting for minio to be ready..." sleep 3 done echo "Minio is ready, setting up credentials..." if ! mc admin user svcacct info myminio AAAAAAAAAAAAAAAAAAAA > /dev/null 2>&1; then mc admin user svcacct add --access-key AAAAAAAAAAAAAAAAAAAA \ --secret-key BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB \ myminio \ miniouser fi depends_on: minio: condition: service_healthy starrocks-fe: image: registry.cn-shanghai.aliyuncs.com/qwx_images/fe-ubuntu:3.2-latest hostname: starrocks-fe container_name: starrocks-fe user: root command: - /bin/bash - -c - | echo "# enable shared data, set storage type, set endpoint" >> /opt/starrocks/fe/conf/fe.conf echo "run_mode = shared_data" >> /opt/starrocks/fe/conf/fe.conf echo "cloud_native_storage_type = S3" >> /opt/starrocks/fe/conf/fe.conf echo "aws_s3_endpoint = minio:9000" >> /opt/starrocks/fe/conf/fe.conf echo "# set the path in MinIO" >> /opt/starrocks/fe/conf/fe.conf echo "aws_s3_path = starrocks" >> /opt/starrocks/fe/conf/fe.conf echo "# credentials for MinIO object read/write" >> /opt/starrocks/fe/conf/fe.conf echo "aws_s3_access_key = AAAAAAAAAAAAAAAAAAAA" >> /opt/starrocks/fe/conf/fe.conf echo "aws_s3_secret_key = BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB" >> /opt/starrocks/fe/conf/fe.conf echo "aws_s3_use_instance_profile = false" >> /opt/starrocks/fe/conf/fe.conf echo "aws_s3_use_aws_sdk_default_behavior = false" >> /opt/starrocks/fe/conf/fe.conf echo "# Set this to false if you do not want default" >> /opt/starrocks/fe/conf/fe.conf echo "# storage created in the object storage using" >> /opt/starrocks/fe/conf/fe.conf echo "enable_load_volume_from_conf = true" >> /opt/starrocks/fe/conf/fe.conf /opt/starrocks/fe/bin/start_fe.sh ports: - 8030:8030 - 9020:9020 - 9030:9030 healthcheck: test: 'mysql -u root -h starrocks-fe -P 9030 -e "show frontends\G" |grep "Alive: true"' interval: 10s timeout: 5s retries: 3 depends_on: minio: condition: service_healthy starrocks-cn: image: registry.cn-shanghai.aliyuncs.com/qwx_images/cn-ubuntu:3.2-latest command: - /bin/bash - -c - | sleep 200s; ulimit -u 65535; ulimit -n 65535; mysql --connect-timeout 2 -h starrocks-fe -P9030 -uroot -e "ALTER SYSTEM ADD COMPUTE NODE \"starrocks-cn:9050\";" /opt/starrocks/cn/bin/start_cn.sh ports: - 8040:8040 hostname: starrocks-cn container_name: starrocks-cn user: root depends_on: starrocks-fe: condition: service_healthy minio: condition: service_healthy healthcheck: test: 'mysql -u root -h starrocks-fe -P 9030 -e "SHOW COMPUTE NODES\G" |grep "Alive: true"' interval: 10s timeout: 5s retries: 3 EOF创建存储桶浏览器打开minio的在线地址http://10.84.3.129:9001注意：需要在Buckets中创建starrocks否则，starrocks-cn将报错，可到starrocks-cn容器中的/opt/starrocks/cn/log/cn.INFO查看报错信息示例：I0703 06:51:10.169481 1181 agent_server.cpp:436] Submit task success. type=CREATE, signature=10339, task_count_in_queue=10 W0703 06:51:10.174785 1036 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6B127DB7][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/meta/000000000000285C_0000000000000001.me ta error: The specified bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() be/src/storage/lake/tablet_manager.cpp:198 file.save(*metadata), signature: 10332 W0703 06:51:10.175204 1466 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6B140674][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/SCHEMA_0000000000002859 error: The specif ied bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() be/src/storage/lake/tablet_manager.cpp:513 file.save(schema_pb) be/src/storage/lake/tablet_manager.cpp:177 create_schema_file(req.tablet_id, tablet_metadata_pb->schema()), signature: 10330 W0703 06:51:10.175679 1467 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6B1E0F64][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/meta/000000000000285B_0000000000000001.me ta error: The specified bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() be/src/storage/lake/tablet_manager.cpp:198 file.save(*metadata), signature: 10331 W0703 06:51:10.178843 1466 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6B5E672D][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/meta/000000000000285D_0000000000000001.me ta error: The specified bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() be/src/storage/lake/tablet_manager.cpp:198 file.save(*metadata), signature: 10333 W0703 06:51:10.179836 1467 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6B6C9D42][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/meta/000000000000285F_0000000000000001.me ta error: The specified bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() be/src/storage/lake/tablet_manager.cpp:198 file.save(*metadata), signature: 10335 W0703 06:51:10.181123 1036 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6B7CFA0E][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/meta/000000000000285E_0000000000000001.me ta error: The specified bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() be/src/storage/lake/tablet_manager.cpp:198 file.save(*metadata), signature: 10334 W0703 06:51:10.181665 1466 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6B8AB304][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/meta/0000000000002860_0000000000000001.me ta error: The specified bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() be/src/storage/lake/tablet_manager.cpp:198 file.save(*metadata), signature: 10336 W0703 06:51:10.182467 1467 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6B9A7B06][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/meta/0000000000002861_0000000000000001.me ta error: The specified bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() be/src/storage/lake/tablet_manager.cpp:198 file.save(*metadata), signature: 10337 W0703 06:51:10.183777 1036 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6BAC5B5C][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/meta/0000000000002862_0000000000000001.me ta error: The specified bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() be/src/storage/lake/tablet_manager.cpp:198 file.save(*metadata), signature: 10338 W0703 06:51:10.183971 1466 agent_task.cpp:232] create table failed. status: Not found: starlet err [RequestID=17DEA14F6BAD7196][StatusCode=404]Put object s3://starrocks/0a861f1d-511e-4d23-9ddd-555f9b504b86/db10004/10328/10327/meta/0000000000002863_0000000000000001.me ta error: The specified bucket does not exist be/src/storage/protobuf_file.cpp:147 output_file->close() 相关容器目录持久化FE的相关目录：meta目录：/opt/starrocks/fe/meta/log目录：/opt/starrocks/fe/log/配置文件目录：/opt/starrocks/fe/conf/CN的相关目录：配置文件目录：/opt/starrocks/cn/conf/log目录：/opt/starrocks/cn/log/MiniO的相关目录：数据目录：/minio_data/