以前AES2.Xのインストールにトライしたが断念。
k8s clusterを作り直したので、AES3.Xのインストールにリトライ。
今回は下記バージョンのインストールを試してみる。
helm search repo edge-stack
NAME CHART VERSION APP VERSION DESCRIPTION
datawire/edge-stack 8.5.1 3.5.1 A Helm chart for Ambassador Edge StackAES-CRDS適用
AESインストールする前に、getambassador.io/v3alpha1, getambassador.io/v2用CRDSを適用する。
kubectl apply -f https://app.getambassador.io/yaml/edge-stack/3.5.1/aes-crds.yamlこれを適用すると、emissary-systemというnamespace, emissary-apiextというdeploymentが作成される。
namespace/emissary-system created
serviceaccount/emissary-apiext created
clusterrole.rbac.authorization.k8s.io/emissary-apiext created
clusterrolebinding.rbac.authorization.k8s.io/emissary-apiext created
role.rbac.authorization.k8s.io/emissary-apiext created
rolebinding.rbac.authorization.k8s.io/emissary-apiext created
service/emissary-apiext created
deployment.apps/emissary-apiext createdこれは、getambassador.io/v3alpha1とgetambassador.io/v2間のAmbassador Edge Stack CRDの変換をサポートするAPIサーバー拡張で、常に起動しておく必要がある。
kubectl -n emissary-system get all ✘ 1
NAME READY STATUS RESTARTS AGE
pod/emissary-apiext-5fd5ff7d5b-8hzt9 1/1 Running 0 52s
pod/emissary-apiext-5fd5ff7d5b-jf27q 1/1 Running 0 52s
pod/emissary-apiext-5fd5ff7d5b-llsbt 1/1 Running 0 52s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/emissary-apiext ClusterIP 10.245.201.63 <none> 443/TCP 53s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/emissary-apiext 3/3 3 3 53s
NAME DESIRED CURRENT READY AGE
replicaset.apps/emissary-apiext-5fd5ff7d5b 3 3 3 53s
Install Ambasador Edge Stack
helmでAESをインストール
helm install edge-stack datawire/edge-stack -n ambassador --create-namespace -f ambassador-values.yaml# Resource requests/limits
resources:
limits:
cpu: 1000m
memory: 600Mi
requests:
cpu: 200m
memory: 200Mi
状態確認
kubectl get all -n ambassadorこんな感じで出力される。
kubectl -n ambassador get all ✘ 130
NAME READY STATUS RESTARTS AGE
pod/edge-stack-85d8669ff5-484dl 1/1 Running 0 7d7h
pod/edge-stack-85d8669ff5-7dvz2 1/1 Running 0 7d7h
pod/edge-stack-85d8669ff5-xkrcj 1/1 Running 0 7d7h
pod/edge-stack-agent-7679947fc9-mb2hj 1/1 Running 0 7d7h
pod/edge-stack-redis-6b85454bd6-55w5c 1/1 Running 0 7d7h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/edge-stack LoadBalancer 10.245.177.249 188.166.204.136 80:32406/TCP,443:32484/TCP 14d
service/edge-stack-admin ClusterIP 10.245.167.147 <none> 8877/TCP,8005/TCP 14d
service/edge-stack-agent ClusterIP 10.245.65.59 <none> 80/TCP 14d
service/edge-stack-redis ClusterIP 10.245.100.123 <none> 6379/TCP 14d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/edge-stack 3/3 3 3 14d
deployment.apps/edge-stack-agent 1/1 1 1 14d
deployment.apps/edge-stack-redis 1/1 1 1 14d
NAME DESIRED CURRENT READY AGE
replicaset.apps/edge-stack-85d8669ff5 3 3 3 7d7h
replicaset.apps/edge-stack-agent-7679947fc9 1 1 1 7d7h
replicaset.apps/edge-stack-redis-6b85454bd6 1 1 1 7d7hステータス確認
Helm
helm ls -n ambassador
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
edge-stack ambassador 1 2023-03-23 10:52:33.18849 +0900 JST deployed edge-stack-8.5.1 3.5.1doctl
doctl compute load-balancer list --format IP,ID,Name,Status
IP ID Name Status
163.47.10.103 f9ab559d-9999-9999-9999-9999999999999 ac5573ab1ff154b648ac1c4d9cccf475 activek8s svc
kubectl -n ambassador get svc ✘ 1
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
edge-stack LoadBalancer 10.245.221.238 163.47.10.103 80:30033/TCP,443:32622/TCP 2m52s
edge-stack-admin ClusterIP 10.245.61.110 <none> 8877/TCP,8005/TCP 2m52s
edge-stack-agent ClusterIP 10.245.208.39 <none> 80/TCP 2m52s
edge-stack-redis ClusterIP 10.245.111.224 <none> 6379/TCP 2m52sAES Listener設定
Listener設定のマニフェストを適用する
kubectl apply -f ambassador_listener.yaml---
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
name: http-listener
spec:
port: 8080
protocol: HTTPS
# protocolStack:
# - PROXY
# - HTTP
# - TCP
securityModel: XFP
statsPrefix: http-listener
hostBinding:
namespace:
from: ALL
---
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
name: https-listener
spec:
port: 8443
protocol: HTTPS
# protocolStack:
# - PROXY
# - TLS
# - HTTP
# - TCP
securityModel: XFP
statsPrefix: https-listener
hostBinding:
namespace:
from: ALL
Listenerが作成される
kubectl get listeners.getambassador.io
NAME PORT PROTOCOL STACK STATSPREFIX SECURITY L7DEPTH
http-listener 8080 HTTPS http-listener XFP
https-listener 8443 HTTPS https-listener XFPDNS登録
AES用にDNS登録する。
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
edge-stack LoadBalancer 10.245.221.238 163.47.10.103 80:30033/TCP,443:32622/TCP 2m52s上の”EXTERNAL-IP”フィールドに表示されているIPアドレスをAレコードに設定する。
Hostリソース追加
kubectl apply -f amb-host.yaml---
apiVersion: getambassador.io/v3alpha1
kind: Host
metadata:
name: wp
namespace: ambassador
spec:
hostname: wp.dok8s.net
acmeProvider:
authority: https://acme-v02.api.letsencrypt.org/directory
email: wp@dok8s.net
tlsSecret:
name: tls2-cert
requestPolicy:
insecure:
action: Redirect
additionalPort: 8080kubectl -n ambassador get host wp
NAME HOSTNAME STATE PHASE COMPLETED PHASE PENDING AGE
wp wp.dok8s.net Ready STATEがReadyになればOK
Mappingリソース追加
kubectl apply -f amb-map.yaml---
apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
name: wp
namespace: ambassador
spec:
prefix: /
host: wp.dok8s.net
service: http://wordpress.wp/kubectl -n ambassador get mapping wp ✘ 1
NAME SOURCE HOST SOURCE PREFIX DEST SERVICE STATE REASON
wp wp.dok8s.net / http://wordpress.wp/Proxy Protocol有効化
DigitalOcean Load-BalancerのProxy Protocol設定を有効化する。
AES LoadBalancerにannotation追加
Chartファイルに下記を追加し、helm upgrade。
emissary-ingress:
service:
type: LoadBalancer
annotations:
# Enable http/2 Ports
# service.beta.kubernetes.io/do-loadbalancer-http2-ports: "443,80"
# Enable proxy protocol
service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
# Specify whether the DigitalOcean Load Balancer should pass encrypted data to backend droplets
service.beta.kubernetes.io/do-loadbalancer-tls-passthrough: "true"
# You can keep your existing LB when migrating to a new DOKS cluster, or when reinstalling AES
kubernetes.digitalocean.com/load-balancer-id: "f9ab559d-9999-9999-9999-9999999999999"
service.kubernetes.io/do-loadbalancer-disown: false
helm upgrade edge-stack datawire/edge-stack -n ambassador -f ambassador-values.yamlAES Listener設定変更
---
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
name: http-listener
spec:
port: 8080
# protocol: HTTPS
protocolStack:
- PROXY
- HTTP
- TCP
securityModel: XFP
statsPrefix: http-listener
hostBinding:
namespace:
from: ALL
---
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
name: https-listener
spec:
port: 8443
# protocol: HTTPS
protocolStack:
- PROXY
- TLS
- HTTP
- TCP
securityModel: XFP
statsPrefix: https-listener
hostBinding:
namespace:
from: ALL
kubectl apply -f ambassador_listener.yaml