使用 Host sFlow 部署端口镜像功能实现腾讯云服务器流量旁路监控/数据分析

原创

说云时间

修改于 2017-11-10 14:38:55

7.3K1

修改于 2017-11-10 14:38:55

文章被收录于专栏：Stone.L@Tencent的专栏Stone.L@Tencent的专栏

为了方便的对网站流量进行数据监控和分析,一般会用到交换机的端口镜像功能,将数据复制一份发送到监控平台或分析系统;而如果将网站部署到腾讯云平台,是否也可以将服务器流量进行复制转发呢?答案是可以的,实现的方法就是通过Host sFlow这个流量监控利器,下面以CentOS7为例,介绍其部署和验证方法:

部署架构

操作系统: CentOS 7.3

为Web服务器添加双网卡-弹性网卡检查主机网卡配置和路由: ifconfig route -n
下载并安装Host sFlow

所有版本下载地址:http://www.sflow.net/downloads.php Github:https://github.com/sflow/host-sflow/releases

以CentOS7为例,运行:

sudo yum install hsflowd-2.0.11-1.x86_64.rpm #当前版本
sudo systemctl enable hsflowd # or chkconfig --add hsflowd /启用
sudo vi /etc/hsflowd.conf #编辑配置文件

配置文件示例:(详细配置说明见注意事项)

sflow {
    #采样比1/1
    sampling = 1
    #截取包大小为256
    headerBytes = 256
    #设置收集器地址和端口
    collector { ip=10.11.11.7 udpport=6343 }
    #设置采样的网卡
    pcap { dev = eth0 }
    }

启动采样:

sudo service hsflowd start

下载并安装收集分析器,以sFlowTrend为例(支持sFlow协议的软件都可以)

下载页面:http://www.inmon.com/products/sFlowTrend.php 或在CentOS服务器运行:

wget http://www.inmon.com/products/sFlowTrend/downloads/sFlowTrend-linux-6_6.rpm

安装并运行: yum install java -y #sFlowTrend依赖java环境 rpm -i sFlowTrend-linux-6_6.rpm #rpm包安装 systemctl status sflowtrend-server.service #查看状态 systemctl start sflowtrend-server.service #启动

使用web portal查看分析数据:

http://<hostname>:8087/sflowtrend #浏览器中打开http
https://<hostname>:8443/sflowtrend #浏览器中打开https

正常的话很快就可以看到以下内容:

注意事项:

安全组放通相应监听和传输端口
windows等其他版本下载相应安装包即可
Host sFlow 详细配置说明 http://sflow.net/host-sflow-linux-config.php cat /etc/hsflowd.conf # hsflowd configuration file # http://sflow.net/host-sflow-linux-config.php

    sflow {
  # ======  Agent IP selection ======
  # Selection is automatic, unless:
  # (1) override with preferred CIDR:
  #   agent.cidr = 192.168.0.0/16
  # (2) Override with interface:
  #   agent = eth0

  # ====== Sampling/Polling/Collectors ======
  # EITHER: automatic (DNS SRV+TXT from _sflow._udp):
  #   DNS-SD { }
  # OR: manual:
  #   Counter Polling:
  #     polling = 30
  #   default sampling N:
  #     sampling = 400 采样比默认为1/400, 可设置分母为1,表示全采样
  sampling = 1
  #  截取包大小默认为256, 可以修改为1500(需重新编译), 尽量不要设置太大,否则会产生很多分片包,影响性能
  headerBytes = 256
  #   sampling N on interfaces with ifSpeed:
  #     sampling.100M = 100
  #     sampling.1G = 1000
  #     sampling.10G = 10000
  #     sampling.40G = 40000
  #   sampling N for apache, nginx:
  #     sampling.http = 50
  #   sampling N for application (requires json):
  #     sampling.app.myapp = 100
  #   collectors: 收集器地址,可修改为数据分析系统所在的ip地址,收集器可以设置多个
  collector { ip=10.11.11.7 udpport=6343 }
  #   add additional collectors here

  # ====== Local configuration ======
  # listen for JSON-encoded input:
  #   json { UDPport = 36343 }
  # PCAP+BPF packet-sampling:
  #   Bridge example:
  #     pcap { dev = docker0 }
  #   NIC example: 设置要采样的网卡
  pcap { dev = eth0 }
  #     pcap { dev = eth0 }
  #     pcap { dev = eth1 }
  #   All NICs example:
  #     pcap { speed=1G-1T }
  # NFLOG packet-sampling:
  #   nflog { group = 5  probability = 0.0025 }
  # ULOG packet-sampling:
  #   ulog { group = 1  probability = 0.0025 }
  # Nvidia NVML GPU monitoring:
  #   nvml { }
  # Xen hypervisor and VM monitoring:
  #   xen { }
  # Open vSwitch sFlow configuration:
  #   ovs { }
  # KVM (libvirt) hypervisor and VM monitoring:
  #   kvm { }
  # Docker container monitoring:
  #   docker { }
  # TCP round-trip-time/loss/jitter (requires pcap/nflog/ulog)
  #   tcp { }
  # monitoring of systemd cgroups
  #   systemd { }
  # DBUS agent
  #   dbus { }
  # Learn config from Arista EAPI
  #   eapi { }
}