分布式存储Minio集群搭建
Minio是一个开源的分布式文件存储系统，它基于 Golang 编写,虽然轻量，却拥有着不错的高性能，可以将图片、视频、音乐、pdf这些文件存储到多个主机，可以存储到多个Linux，或者多个Windows，或者多个Mac，Minio中存储最大文件可以达到5TB。任何类型的文件都是支持的，主要应用在微服务系统中.
一、准备机器
获取最新更新以及文章用到的软件包，请移步点击：查看更新
1、准备四台机器，（minio集群最少四台）。
192.168.223.131 minio-1192.168.223.128 minio-2192.168.223.129 minio-3192.168.223.130 minio-4
2、编辑hosts文件，将以上内容添加到hosts中
vim /etc/hosts
部署(所有机器均执行)
以下的操作都需要在四台机器上执行
3、创建挂载磁盘路径
mkdir -p /data/minio_data/
4、挂载磁盘路径到文件系统
注意：需要将新建的目录挂在到对应的磁盘下,磁盘不挂载好，集群启动会报错，还需要注意的是挂载的文件系统至少要1G不然无法初始化导致集群报错
文件系统 容量 已用 可用 已用% 挂载点
[root@minio-1 minio]# df -h文件系统 容量 已用 可用 已用% 挂载点devtmpfs 470M 0 470M 0% /devtmpfs 487M 0 487M 0% /dev/shmtmpfs 487M 8.4M 478M 2% /runtmpfs 487M 0 487M 0% /sys/fs/cgroup/dev/mapper/centos_hadoop--master-root 47G 12G 36G 25% //dev/sda1 1014M 240M 775M 24% /boottmpfs 98M 0 98M 0% /run/user/0tmpfs 98M 12K 98M 1% /run/user/42————————————————
5、将上面挂载磁盘路径挂载到相应的文件系统上
mount /dev/sda1 /data/minio_data/
6、查看挂载信息
7、创建minio目录
cd /data/minio_data/
8、下载安装包
wget http://dl.minio.org.cn/server/minio/release/linux-amd64/miniowget https://dl.min.io/client/mc/release/linux-amd64/mc
9、赋执行权限（根据情况，这里赋全部权限）
chmod +x minio mc
10、创建启动脚本，编辑run.sh文件
mkdir /data/minio_data && cd /data/minio_data
内容如下：
cat > run.sh <<EOF#!/bin/bashexport MINIO_ACCESS_KEY=minioexport MINIO_SECRET_KEY=Leo825#20210423/usr/local/bin/minio server --address=192.168.81.235:9000 http://192.168.81.235/data/minio_data/data1 http://192.168.81.236/data/minio_data/data1 http://192.168.81.237/data/minio_data/data1 http://192.168.81.234/data/minio_data/data1EOF
11、赋执行权限（根据情况，这里赋全部权限）
chmod 777 /data/minio_data/run.sh
12、创建启动服务，创建minio.service启动脚本
vim /usr/lib/systemd/system/minio.service
内容如下：
[Unit]Description=Minio serviceDocumentation=https://docs.minio.io/[Service]#安装包路径WorkingDirectory=/data/minio_data#启动命令路径ExecStart=/data/minio_data/run.shRestart=on-failureRestartSec=5[Install]WantedBy=multi-user.target
复制代码
13、启动测试（所有机器执行）
复制代码
重新加载服务的配置文件
systemctl daemon-reload
启动minio服务
systemctl start minio
查看minio状态
systemctl status minio[root@minio-2 ~]# systemctl status minio● minio.service - Minio serviceLoaded: loaded (/usr/lib/systemd/system/minio.service; disabled; vendor preset: disabled)Active: active (running) since 日 2021-01-31 17:22:54 CST; 17s agoDocs: https://docs.minio.io/Main PID: 2036 (run.sh)Tasks: 8CGroup: /system.slice/minio.service├─2036 /bin/bash /data/minio_data/run.sh└─2039 /data/minio_data server http://192.168.223.232/data/minio_data/data1 http://192.168.223.233/data/minio_data/data1
关闭minio服务
systemctl stop minio
复制代码
14、访问地址
集群中的任何一台机器都可以访问：
http://192.168.223.132:9000/http://192.168.223.133:9000/http://192.168.223.134:9000/http://192.168.223.135:9000/
15、创建测试桶
16、上传测试
17、主机上可以查看到上传的文件
二、nginx配置文件服务器访问
1、执行命令
mc alias set minio http://192.168.223.132:9000/ minio Leo825#20210423 --api S3v4开启匿名访问mc policy set public minio/sy01
2、web页面开启匿名访问
3、http访问，sy01是桶名称，方便浏览器访问。
upstream minio-server{ server 192.168.6.124:9000 weight=25 max_fails=2 fail_timeout=30s; server 192.168.6.125:9000 weight=25 max_fails=2 fail_timeout=30s; server 192.168.6.126:9000 weight=25 max_fails=2 fail_timeout=30s; server 192.168.6.128:9000 weight=25 max_fails=2 fail_timeout=30s;}server { listen 8888; server_name 192.168.6.120; #To allow special characters in headers ignore_invalid_headers off; # Allow any size file to be uploaded. # Set to a value such as 1000m; to restrict file size to a specific value client_max_body_size 0; # To disable buffering proxy_buffering off; location /sy01/ { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Host $http_host; proxy_connect_timeout 300; # Default is HTTP/1, keepalive is only enabled in HTTP/1.1 proxy_http_version 1.1; proxy_set_header Connection ""; chunked_transfer_encoding off; proxy_pass http://minio-server; } location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Host $http_host; proxy_connect_timeout 300; # Default is HTTP/1, keepalive is only enabled in HTTP/1.1 proxy_http_version 1.1; proxy_set_header Connection ""; chunked_transfer_encoding off; proxy_pass http://minio-server; # If you are using docker-compose this would be the hostname i.e. minio # Health Check endpoint might go here. See https://www.nginx.com/resources/wiki/modules/healthcheck/ # /minio/health/live; }}
4、https访问，minio是负载minio服务，sy01是桶名称，方便浏览器访问。
upstream minio-server{ server 192.168.6.124:9000 weight=25 max_fails=2 fail_timeout=30s; server 192.168.6.125:9000 weight=25 max_fails=2 fail_timeout=30s; server 192.168.6.126:9000 weight=25 max_fails=2 fail_timeout=30s; server 192.168.6.128:9000 weight=25 max_fails=2 fail_timeout=30s;}server { listen 443 ssl; server_name 192.168.6.120; ssl_certificate /etc/nginx/ssl/192.168.6.120.crt; ssl_certificate_key /etc/nginx/ssl/192.168.6.120.key; location /sy01/ { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Host $http_host; proxy_connect_timeout 300; proxy_http_version 1.1; proxy_set_header Connection ""; chunked_transfer_encoding off; proxy_pass http://minio-server; } location /minio { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Host $http_host; proxy_connect_timeout 300; proxy_http_version 1.1; proxy_set_header Connection ""; chunked_transfer_encoding off; proxy_pass http://minio-server; } }
或者换成tcp代理也可以实现，放在http层外，如果服务器在内网，访问在外网，中间经过了代理，或者隧道之内的，必须要要tcp，否则会http头部hred验证会失败。
upstream minio-server { server 192.168.178.40:9000 max_fails=3 fail_timeout=30s; server 192.168.178.41:9000 max_fails=3 fail_timeout=30s; server 192.168.178.42:9000 max_fails=3 fail_timeout=30s; server 192.168.178.43:9000 max_fails=3 fail_timeout=30s;}server { listen 9000; proxy_connect_timeout 2s; proxy_timeout 900s; proxy_pass minio-server;}