外包技能--Ingresn进阶之ingress-nginx

原创

Godev

发布于 2023-08-02 18:40:39

5950

发布于 2023-08-02 18:40:39

文章被收录于专栏：GodevGodev

nodeport的方式去暴露端口，那你需要得去判断暴露的端口有没有被占用，再创建新的应用会判断端口有没有被分配出去 nodeport本身是基于默认的iptables的代理模式做的网络转发，也就是SANT,DANT,基于四层的，做七层是做不了的，性能差一点，因为它需要防火墙的转发和过滤。

外部访问应用方式

Pod与Ingress的关系
通过Service相关联
通过Ingress Controller实现Pod的负载均衡
支持TCP/UDP 4层和HTTP 7层

简单示例

Ingress之Controller

controller类似装的k8s组件，时常的要去api去交互，时常去获取api的相关的信息，刷新自己的规则，类似与其他控制器 ingress，k8s设计了一个比较全局性的负载均衡器，准确的来说Ingress它是k8s中的一个规则，实现这个规则就是使用的这个控制器，一般称为ingress控制器

ingress控制器主要做的工作就是，它访问到这个控制器，它帮你转发的具体pod,也就是集群池，关联的哪些应用，哪些pod的IP，会帮你关联出来，暴露的端口80,443

部署Ingress Controlle

部署文档： https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md

创建Ingress规则，为你的应用暴露一个端口，暴露一个域名，让用户去访问这个ingress controller控制器就可以了

控制器选择类型

https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/ 注意事项： ? 镜像地址修改成国内的： nginx-ingress-controller:0.30.0 ? 使用宿主机网络：hostNetwork: true

ingress获取

github版本 https://github.com/kubernetes/ingress-nginx/tree/nginx-0.30.0

修改yaml文件

    spec:          //这是212行的spec字段
      hostNetwork: true            //添加此行，表示使用主机网络
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        Ingress: nginx            //设置节点的标签选择器，指定在哪台节点上运行

部署yaml

[root@master static]# kubectl apply -f mandatory.yaml 
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
Warning: rbac.authorization.k8s.io/v1beta1 Role is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 Role
role.rbac.authorization.k8s.io/nginx-ingress-role created
Warning: rbac.authorization.k8s.io/v1beta1 RoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 RoleBinding
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBinding
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created
[root@master static]#

等待ingress-controller为Running状态

[root@master ingress]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-65886f4f5d-zd2nd   1/1     Running   0          92s
[root@master ingress]#

部署示例Pod（Deployment）、svc

[root@master ingress]# kubectl apply -f deployment_svc.yaml 
deployment.apps/myapp-deploy1 created
service/myapp-clusterip1 created
[root@master ingress]# 
[root@master ingress]# cat deployment_svc.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deploy1
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
      release: v1
  template:
    metadata:
      labels:
        app: myapp
        release: v1
        env: test
    spec:
      containers:
      - name: myapp
        image: nginx
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: myapp-clusterip1
  namespace: default
spec:
  type: ClusterIP  # 默认类型
  selector:
    app: myapp
    release: v1
  ports:
  - name: http
    port: 80
    targetPort: 80

[root@master ingress]#

查看ingress-controller

ingress-controller，我们可以用netstat去查看我们监听的端口80/443 一般我们还可以在这样DaemonSet控制器前面再跑两个基于4层的负载均衡器 User–>lb(vm-nginx/lvs/haproxy)—>node1/node2的IP，再使用算法，进行轮询，---->pod

[root@node02 ~]#  netstat -anpt |grep 80
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      30176/nginx: master 
tcp        0      0 127.0.0.1:10246         127.0.0.1:56804         TIME_WAIT   -                   
tcp        0      0 192.168.2.191:58020     10.96.0.1:443           ESTABLISHED 2829/flanneld       
tcp        0      0 127.0.0.1:10246         127.0.0.1:56808         TIME_WAIT   -                   
tcp        0      0 127.0.0.1:10246         127.0.0.1:56780         TIME_WAIT   -                   
tcp        0      0 127.0.0.1:10246         127.0.0.1:56806         TIME_WAIT   -                   
tcp        0      0 127.0.0.1:10246         127.0.0.1:56802         TIME_WAIT   -                   
[root@node02 ~]# netstat -anpt |grep 443
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      30176/nginx: master 
tcp        0      0 192.168.2.191:51978     192.168.2.137:6443      ESTABLISHED 2085/kubelet        
tcp        0      0 192.168.2.191:58020     10.96.0.1:443           ESTABLISHED 2829/flanneld       
tcp        0      0 192.168.2.191:34026     10.96.0.1:443           ESTABLISHED 30129/nginx-ingress 
tcp        0      0 192.168.2.191:51976     192.168.2.137:6443      ESTABLISHED 2465/kube-proxy     
[root@node02 ~]#

Http 配置

[root@master ingress]#  kubectl get pod -n ingress-nginx -o wide
NAME                                        READY   STATUS    RESTARTS   AGE    IP           NODE     NOMINATED NODE   READINESS GATES
nginx-ingress-controller-65886f4f5d-zd2nd   1/1     Running   0          3m2s   10.244.2.2   node02   <none>           <none>
[root@master ingress]# 
[root@k8s-master demo]# cat http.yaml 
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: docs-http
  namespace: default
spec:
  rules:
    - host: docs.27ops.com
      http:
        paths:
        - path: /usr/share/nginx/html
          backend:
            serviceName: myapp-clusterip1
            servicePort: 80
[root@master ingress]# 
[root@master ingress]# kubectl apply  -f http.yaml 
Warning: networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
ingress.networking.k8s.io/docs-http created
[root@master ingress]# kubectl get ingress -o wide
NAME        CLASS    HOSTS            ADDRESS   PORTS   AGE
docs-http   <none>   docs.27ops.com             80      28s
[root@master ingress]#

测试访问，这里我是写到了我的hosts文件中，要是做域名解析的话也是解析我们ingress的IP

Https配置

cfssl生成证书

[root@master ingress]#  cat cfssl.sh
curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo
chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo
[root@master ingress]#  sh cfssl.sh
[root@master ingress]#  ls
certs.sh  cfssl.sh
[root@master ingress]#  chmod +x certs.sh 
[root@master ingress]#  sh certs.sh

查看证书

[root@master *.27ops.com]# ls
*.27ops.com.cer  *.27ops.com.conf  *.27ops.com.csr  *.27ops.com.csr.conf  *.27ops.com.key  ca.cer  fullchain.cer
[root@master *.27ops.com]#

创建证书secret

[root@master *.27ops.com]# kubectl create secret tls tls-docs --cert=fullchain.cer --key=*.27ops.com.key
secret/tls-docs created
[root@master *.27ops.com]# kubectl get secret
NAME                  TYPE                                  DATA   AGE
default-token-sdxdw   kubernetes.io/service-account-token   3      84m
tls-docs              kubernetes.io/tls                     2      26s
[root@master *.27ops.com]# 


[root@master ingress]# kubectl apply -f https.yaml 
Warning: networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
ingress.networking.k8s.io/docs-https created
[root@master ingress]# cat https.yaml 
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: docs-https
  namespace: default
spec:
  tls:
    - hosts:
      - docs.27ops.com
      secretName: tls-docs
  rules:
    - host: docs.27ops.com
      http:
        paths:
        - path: /
          backend:
            serviceName: myapp-clusterip1
            servicePort: 80
[root@master ingress]# kubectl get ingress
NAME         CLASS    HOSTS            ADDRESS         PORTS     AGE
docs-http    <none>   docs.27ops.com   10.98.174.230   80        16m
docs-https   <none>   docs.27ops.com   10.98.174.230   80, 443   28s
[root@master ingress]#

Nginx