Elastic Stack最佳实践：在腾讯云上通过docker运行esrally

原创

点火三周

发布于 2022-03-19 20:24:54

1.4K0

发布于 2022-03-19 20:24:54

文章被收录于专栏：Elastic Stack专栏Elastic Stack专栏

如果要在腾讯云上运行es_rally, 在我看来，用docker是最方便的，可以省去很多麻烦事。如果不想用docker，要自己安装的话，可以参考以下几篇文章：

/developer/article/1891569

/developer/article/1892065

https://developer.aliyun.com/article/851848

所以本文主要是简要说明一下docker上使用rally的踩坑要点：

前提 1：已经在服务器上安装了docker。因为现在新申请的云服务器很多已经默认安装了docker，所以这部分省略
前提 2：你的服务器可以访问internet，一般的云服务器都有公网ip，可以访问网络，所以这部分也省略
github的问题。因为esrally需要通过访问github来下载数据，因此，对于git的连通性是必须要解决的 - 修改 /etc/hosts, 添加内容：140.82.112.3 github.com 199.232.69.194 github.global.ssl.fastly.net
esrally配置文件的问题。esrally配置文件位于/rally/.rally/rally.ini 需要自定义 Rally，强烈建议对容器中的/rally/.rally 目录使用本地绑定挂载（或命名卷）。这将确保您在调用中具有持久性，并且可以重复使用下载和提取的任何曲目，从而减少启动时间。具体操作为：

mkdir myrally
sudo chgrp 0 myrally
sudo chmod -R 777 myrally
# First run will also generate the rally.ini
docker run --rm -v $PWD/myrally:/rally/.rally elastic/rally race --track=nyc_taxis --test-mode --pipeline=benchmark-only --target-hosts=es01:9200

...

# inspect results
$ tree myrally/benchmarks/races/
myrally/benchmarks/races/
└── 1d81930a-4ebe-4640-a09b-3055174bce43
    └── race.json

1 directory, 1 file

注意，如果下载在.rally目录中的新创建的tracks没有权限的话，需要重新给与docker应用该目录的写权限

可以通过下载脚本，单独下载某个track所需要的文件，操作为：
- 1 下载脚本：

curl -O https://raw.githubusercontent.com/elastic/rally-tracks/master/download.sh
chmod u+x download.sh

2 修改脚本：

vi download.sh
# 修改ROOT为
readonly ROOT="myrally/benchmarks"

3 使用脚本下载数据： ./download.sh geonames
运行时链接git的问题。通常会抛出如下问题：

docker run --privileged=true --rm -v $PWD/myrally:/rally/.rally elastic/rally race --track=geonames  --user-tag="ece:7.14.2" --pipeline=benchmark-only --target-hosts="https://es-i4hx8bxq.public.tencentelasticsearch.com:9200" --client-options="use_ssl:true,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'Elastic@1234#'"

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[WARNING] No Internet connection detected. Automatic download of track data sets etc. is disabled.
[INFO] Race id is [e9941039-d45a-4a7a-bb34-6c5d30256e17]
[ERROR] Cannot race. Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/esrally/utils/repo.py", line 69, in update
    git.checkout(self.repo_dir, branch=branch)
  File "/usr/local/lib/python3.8/site-packages/esrally/utils/git.py", line 37, in probe
    return f(src, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/esrally/utils/git.py", line 67, in checkout
    raise exceptions.SupplyError("Could not checkout [%s]. Do you have uncommitted changes?" % branch)
esrally.exceptions.SupplyError: Could not checkout [7.14]. Do you have uncommitted changes?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/esrally/actor.py", line 92, in guard
    return f(self, msg, sender)
  File "/usr/local/lib/python3.8/site-packages/esrally/racecontrol.py", line 110, in receiveMsg_Setup
    self.coordinator.setup(sources=msg.sources)
  File "/usr/local/lib/python3.8/site-packages/esrally/racecontrol.py", line 194, in setup
    self.current_track = track.load_track(self.cfg)
  File "/usr/local/lib/python3.8/site-packages/esrally/track/loader.py", line 191, in load_track
    repo = track_repo(cfg)
  File "/usr/local/lib/python3.8/site-packages/esrally/track/loader.py", line 284, in track_repo
    return GitTrackRepository(cfg, fetch, update)
  File "/usr/local/lib/python3.8/site-packages/esrally/track/loader.py", line 327, in __init__
    self.repo.update(distribution_version)
  File "/usr/local/lib/python3.8/site-packages/esrally/utils/repo.py", line 113, in update
    raise exceptions.DataError("Cannot update %s in [%s] (%s)." % (self.resource_name, self.repo_dir, e.message)).with_traceback(tb)
  File "/usr/local/lib/python3.8/site-packages/esrally/utils/repo.py", line 69, in update
    git.checkout(self.repo_dir, branch=branch)
  File "/usr/local/lib/python3.8/site-packages/esrally/utils/git.py", line 37, in probe
    return f(src, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/esrally/utils/git.py", line 67, in checkout
    raise exceptions.SupplyError("Could not checkout [%s]. Do you have uncommitted changes?" % branch)
esrally.exceptions.DataError: Cannot update tracks in [/rally/.rally/benchmarks/tracks/default] (Could not checkout [7.14]. Do you have uncommitted changes?).

使用--offline只是不会下载一些内容，但是连接git的操作依然会执行，通过 --track-path=/rally/.rally/benchmarks/tracks/default/${your_track} 重新指定本地的track则可以绕过git的连接，如果使用 --track=geonames 这个参数，将会一直去连接git找到这个最新的track信息；

比如：

docker run --privileged=true --rm -v $PWD/myrally:/rally/.rally elastic/rally race --offline --track-path=/rally/.rally/benchmarks/tracks/default/geonames --user-tag="ece:7.14.2" --pipeline=benchmark-only --target-hosts="https://es-i4hx8bxq.public.tencentelasticsearch.com:9200" --client-options="use_ssl:true,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:xxxxxx'"

原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。

如有侵权，请联系 cloudcommunity@tencent.com 删除。

容器镜像服务