• 欢迎访问LLYCLOUD information technical share center (ITSC),Linux 安全,Linux 系统,推荐使用最新版火狐浏览器和Chrome浏览器访问本网站,欢迎加入LLYCLOUD QQ群
  • 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏LLYCLOUD ITSC吧

Elastic Stack 6.7 安装部署尝鲜,基于centos7.6最新系统。使用docker安装。ELK 6.7

Linux Akide_Liu 8个月前 (04-01) 626次浏览 1个评论
文章目录[隐藏]

Elastic Stack 6.7.0尝鲜
基于CentOS7.6, Docker-CE 18.09.4, Docker-web-Ui, Elasticsearch 6.7, Kibana 6.7, Logstash6.7,X-Pack

Akide_Liu

Elastic Stack 6.7 安装部署尝鲜,基于centos7.6最新系统。使用docker安装。ELK 6.7

安装顺序:

Elasticsearch (install instructions)主要搜索引擎

Kibana (install)图形化web界面

Logstash (install)log的汇总与收集(server端)

Beats (install instructions)log收集(client端)

Elasticsearch Hadoop (install instructions)(Hadoop大文件系统,本次不用。)

在安装手册上面有多重方法包括,源码,deb,rpm等等,在这里我们使用源码基于centos7.6.

官网推荐使用java版本Oracle JDK version 1.8.0_131

必须有一个专用的用户运行。

Reference:https://www.elastic.co/guide/en/elasticsearch/reference/6.7/docker.html

docker Images are available for running Elasticsearch as Docker containers. They may be downloaded from the Elastic Docker Registry.

Install Elasticsearch with Docker

Docker安装ELK:

Reference:https://elk-docker.readthedocs.io/

Docker hub:https://hub.docker.com/r/sebp/elk/

环境准备

安装最新版docker-cehttps://docs.docker.com/install/linux/docker-ce/centos/

卸载老版本:
yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine

安装docker-ce依赖

sudo yum install -y yum-utils \
device-mapper-persistent-data \
lvm2

添加yum repo

sudo yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo

启用repo

sudo yum-config-manager --enable docker-ce

开始安装docker-ce

sudo yum install docker-ce docker-ce-cli containerd.io

启动docker服务并开机自启动:

sudo systemctl start docker

sudo systemctl enable docker

可以安装一个docker-ui(对于不太会用docker的,比如我)

https://hub.docker.com/r/uifd/ui-for-docker/

复制粘贴:

Quickstart

 	Run: docker run -d -p 9000:9000 --privileged -v /var/run/docker.sock:/var/run/docker.sock uifd/ui-for-docker
	
 	Open your browser to http://<dockerd host ip>:9000

最小内存要求4GB for docker
Mmap counts高于262,144(不太清楚具体是啥,应该是虚拟内存类似的,有兴趣自己看原文。)

sysctl -w vm.max_map_count=262144

设置防火墙允许相关端口通讯,至少保证5044.推荐关闭防火墙(内网)

systemctl stop firewalld

systemctl status firewalld

● firewalld.service - firewalld - dynamic firewall daemon

Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)

Active: inactive (dead)

Docs: man:firewalld(1)

下载ELK-docker

docker pull sebp/elk

Elastic Stack 6.7 安装部署尝鲜,基于centos7.6最新系统。使用docker安装。ELK 6.7

当然可以选取所需要的版本:

For instance, the image containing Elasticsearch 1.7.3, Logstash 1.5.5, and Kibana 4.1.2 (which is the last image using the Elasticsearch 1.x and Logstash 1.x branches) bears the tag E1L1K4, and can therefore be pulled using sudo docker pull sebp/elk:E1L1K4.

现在的least版本也就是default版本是ELK 6.7

启动docker of ELK

sudo docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -p 9300:9300 -it --name elk sebp/elk &

想看安装过程的可以去ui-docker里面看,但是如果不加&后面log输出太多会导致一旦中断命令就会关闭ELK Stack。

使用的端口:

5601 Kibana web界面
9200 Elasticsearch JSON 接口
5044 Logstash Beats 接口

Elastic Stack 6.7 安装部署尝鲜,基于centos7.6最新系统。使用docker安装。ELK 6.7

    1. Access Kibana’s web interface by browsing to http://<your-host>:5601, where <your-host> is the hostname or IP address of the host Docker is running on (see note), e.g. localhost if running a local native version of Docker, or the IP address of the virtual machine if running a VM-hosted version of Docker (see note).

(访问http://主机名:5601到Kibana web)

Elastic Stack 6.7 安装部署尝鲜,基于centos7.6最新系统。使用docker安装。ELK 6.7

首先需要打开x-pack(安全模块,才可以设置密码。)

打开kibana ---> License Management ---> Start a 30-day trial

进入ELK-docker配置默认用户的密码:

Reference:https://www.elastic.co/guide/en/elasticsearch/reference/6.7/configuring-security.html

docker exec -it elk /bin/bash

/opt/elasticsearch/bin/elasticsearch-setup-passwords interactive

Unexpected response code [500] from calling GET http://172.17.0.3:9200/_xpack/security/_authenticate?pretty

It doesn't look like the X-Pack security feature is enabled on this Elasticsearch node.

Please check if you have enabled X-Pack security in your elasticsearch.yml configuration file.

ERROR: X-Pack Security is disabled by configuration.

[email protected]:/# curl http://127.0.0.1:9200/_cat/health

1554113235 10:07:15 elasticsearch yellow 1 1 12 12 0 0 4 0 - 75.0%

[email protected]:/# curl http://172.17.0.3:9200/_xpack/security/_authenticate?pretty

{

"error" : {

"root_cause" : [

{

"type" : "exception",

"reason" : "Security must be explicitly enabled when using a trial license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."

}

],

"type" : "exception",

"reason" : "Security must be explicitly enabled when using a trial license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."

},

"status" : 500

}

[xpack.security.enabled] to [true]

vim /etc/elasticsearch/elasticsearch.yml
[[email protected] conf.d]# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

c33dd5a85ae8  sebp/elk "/usr/local/bin/star…" 14 minutes ago Up 11 minutes 0.0.0.0:5044->5044/tcp, 0.0.0.0:5601->5601/tcp, 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp elk

6552e53ec2ee uifd/ui-for-docker "/ui-for-docker" 2 hours ago Up 2 hours 0.0.0.0:9000->9000/tcp stupefied_mclaren

[[email protected] conf.d]# docker restart c33dd5a85ae8

c33dd5a85ae8
/opt/elasticsearch/bin/elasticsearch-setup-passwords interactive

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.

You will be prompted to enter passwords as the process progresses.

Please confirm that you would like to continue [y/N]y

Enter password for [elastic]:

Reenter password for [elastic]:

Enter password for [apm_system]:

Reenter password for [apm_system]:

Enter password for [kibana]:

Reenter password for [kibana]:

Enter password for [logstash_system]:

Reenter password for [logstash_system]:

Enter password for [beats_system]:

Reenter password for [beats_system]:

Enter password for [remote_monitoring_user]:

Reenter password for [remote_monitoring_user]:

Changed password for user [apm_system]

Changed password for user [kibana]

Changed password for user [logstash_system]

Changed password for user [beats_system]

Changed password for user [remote_monitoring_user]

Changed password for user [elastic]

在这里设置了很多的用户,关于相关用于的专属功能原文:

https://www.elastic.co/guide/en/elastic-stack-overview/6.7/built-in-users.html

因为启用了x-pack,所以相关组件需要设置用户名密码才能登陆。我们需要设置原来能够使用kibana, logstash

[email protected]:/# vim /opt/kibana/config/kibana.yml

#elasticsearch.username: "elastic"

#elasticsearch.password: "yourpass"

[email protected]:/# /etc/init.d/kibana restart

* Stopping Kibana5 [ OK ]

* Starting Kibana5 [ OK ]

[email protected]:/# vim /etc/logstash/conf.d/30-output.conf

#user => "elastic"

#password => "yourpassword"

(#要去掉)

[email protected]:/# vim /etc/logstash/conf.d/02-beats-input.conf

#ssl => true

#ssl_certificate => "/etc/pki/tls/certs/logstash-beats.crt"

#ssl_key => "/etc/pki/tls/private/logstash-beats.key"

(我这里去掉了ssl设置,因为是纯粹内网环境,安全完全没有问题)

[email protected]:/# /etc/init.d/logstash restart

Killing logstash (pid 368) with SIGTERM

Waiting for logstash (pid 368) to die...

Waiting for logstash (pid 368) to die...

logstash stop failed; still running.

logstash started.

(如果不行的话手动kill process然后使用启动脚本)

Log位置:

tail -f /var/log/logstash/logstash-plain.log

使用logging添加系统的system logs

Elastic Stack 6.7 安装部署尝鲜,基于centos7.6最新系统。使用docker安装。ELK 6.7

点开就有怎么安装的流程,但是注意这个是直接连接elasticsearch,我们要先连接elasticsearch来load visualize和dashboard。

Download and install Filebeat

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.7.0-x86_64.rpm sudo

rpm -vi filebeat-6.7.0-x86_64.rpm

Edit the configuration

Modify filebeat.yml to set the connection information:

output.elasticsearch:

hosts: ["<es_url>"]

username: "elastic"

password: "<password>"

setup.kibana:

host: "<kibana_url>"

Enable and configure the system module

sudo filebeat modules enable system

Start Filebeat
sudo filebeat setup

sudo service filebeat start




[[email protected] ~]# sudo filebeat setup

Loaded index template

Loading dashboards (Kibana must be running and reachable)

Loaded dashboards

Loaded machine learning job configurations

[[email protected] ~]# service metricbeat start

Redirecting to /bin/systemctl start metricbeat.service

Failed to start metricbeat.service: Unit not found.

[[email protected] ~]# service filebeat start

Starting filebeat (via systemctl): [ OK ]

output.logstash:

[[email protected] ~]# vim /etc/filebeat/filebeat.yml

#setup.kibana:

# Kibana Host

# Scheme and port can be left out and will be set to the default (http and 5601)

# In case you specify and additional path, the scheme is required: http://localhost:5601/path

# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601

# host: "192.168.10.102:5601"

#output.elasticsearch:

# Array of hosts to connect to.

# hosts: ["192.168.10.102:9200"]

# Enabled ilm (beta) to use index lifecycle management instead daily indices.

#ilm.enabled: false

# Optional protocol and basic auth credentials.

#protocol: "https"

# username: "elastic"

# password: "yourpasswd"

output.logstash:

# The Logstash hosts

hosts: ["192.168.10.102:5044"]

(只这部分enable,剩下全部#)

配置Metrics

下载安装

curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-6.7.0-x86_64.rpm

sudo rpm -vi metricbeat-6.7.0-x86_64.rpm

修改配置文件/etc/metricbeat/metricbeat.yml

output.elasticsearch:

hosts: ["<es_url>"]

username: "elastic"

password: "<password>"

setup.kibana:

host: "<kibana_url>"

启动模块

sudo metricbeat modules enable system

启动metricbeat

sudo metricbeat setup sudo service metricbeat start

最终效果截图,我使用了Frp和nginx代理了kibana,https://elk.llycloud.com

Elastic Stack 6.7 安装部署尝鲜,基于centos7.6最新系统。使用docker安装。ELK 6.7


Llycloud ITSC , 版权所有丨如未注明 , 均为原创丨本网站采用BY-NC-SA协议进行授权
转载请注明原文链接:Elastic Stack 6.7 安装部署尝鲜,基于centos7.6最新系统。使用docker安装。ELK 6.7
喜欢 (1)
[]
分享 (0)
发表我的评论
取消评论

表情 贴图 加粗 删除线 居中 斜体 签到

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
(1)个小伙伴在吐槽
  1. I have read so many articles or reviews about the blogger lovers except this article is genuinely a pleasant post, keep it up.
    Buy Cialis2019-10-30 02:50 回复 Windows 10 | Firefox浏览器 21.0