Elasticsearch Snapshot to AWS S3

Es 核心存储库插件

es核心存储库插件是：

S3存储库: S3存储库插件添加了对使用S3作为存储库的支持。
Azure存储库: Azure存储库插件添加了对使用Azure作为存储库的支持。
HDFS存储库: Hadoop HDFS Repository插件增加了对使用HDFS作为存储库的支持。
Google云端存储库: GCS存储库插件添加了对使用Google Cloud Storage服务作为存储库的支持。

es 安装 aws s3 存储插件

#推荐所有es节点均安装该插件
usr/local/elasticsearch-node-1/bin/elasticsearch-plugin install repository-s3
# 重启es
pkill java
su - es -c "/usr/local/elasticsearch-node-1/bin/elasticsearch -d "
su - es -c "/usr/local/elasticsearch-node-2/bin/elasticsearch -d "

# es 输入 aws key
#ACCESS-KEY
/usr/local/elasticsearch-node-2/bin/elasticsearch-keystore add s3.client.default.access_key
#SECRET-KEY
/usr/local/elasticsearch-node-2/bin/elasticsearch-keystore add s3.client.default.secret_key

创建aws role 并添加到es所在的节点

# 推荐的s3权限，如有疑问可以移步官网：https://www.elastic.co/guide/en/elasticsearch/plugins/6.5/repository-s3-repository.html
{
  "Statement": [
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation",
        "s3:ListBucketMultipartUploads",
        "s3:ListBucketVersions"
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::snap.example.com"
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts"
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::snap.example.com/*"
  "Version": "2012-10-17"
#注意：该策略是允许所挂载的ec2节点对snap.example.com存储桶的访问列表及写入权限。
将创建好的角色挂载到对应的es节点上。
创建ES在Amazon S3存储库实例
# 首先需要创建对应的aws s3 桶，可以自行参考aws官网

#创建ES在Amazon S3存储库实例
curl -XPUT 'http://localhost:9200/_snapshot/backup' -H 'Content-Type: application/json' -d'
    "type": "s3",
    "settings": {
        "bucket": "snap.example.com",
        "region": "ap-northeast-1"
# 确认备份仓库是否创建成功
curl -XPOST http://localhost:9200/_snapshot/backup/_verify?pretty
#查看创建的存储仓库
curl -XGET localhost:9200/_snapshot/backup?pretty
#注意：这个backup其实就是存储库（自己理解，如有错误请指出）。存储桶需要存在以注册快照的存储库。如果您未创建存储桶，则存储库注册将失败。
一些常用命令
#备份所有索引
curl -XPUT http:///localhost:9200/_snapshot/backup/snapshot_all
#备份部分索引
curl -XPUT 'http://localhost:9200/_snapshot/backup/index-201807' -H 'Content-Type: application/json' -d '{ "indices": "index-201807" }'
备份多个索引：
   "indices": "products,index_1,index_2",
   "ignore_unavailable": true,
   "include_global_state": false
#查看快照信息
curl -XGET 'http://localhost:9200/_snapshot/backup/_all'?pretty
#恢复索引数据：
curl -XPOST 'http://localhost:9200/_snapshot/backup/index-201807/_restore
#查看恢复状态：
GET http://localhost:9200/_recovery/index-201807
GET http://localhost:9200/_recovery/
#删除一个快照存储桶：
curl -XDELETE localhost:9200/_snapshot/backup/index-201807?pretty
如有其它问题请参考：http://www.voidcn.com/article/p-srbgjcjr-bro.html
或者移步：
https://medium.com/@federicopanini/elasticsearch-backup-snapshot-and-restore-on-aws-s3-f1fc32fbca7f
https://www.elastic.co/guide/cn/elasticsearch/guide/current/backing-up-your-cluster.html#backing-up-your-cluster