kubernetes metallb 및 네트워크 설정 질문드립니다.

Question

상황 : 쿠버네티스 metallb pod에서 오류 발생 스크린샷 : 추론 결론 : METALLB 설정에 문제가 있는것으로 추론(각각의 파드 로그 까서 확인함)세부 디테일(로그, 디스크라이브)local-volume-provisioner-vpwlzlogs[root@konvoy-master ~]# k -n kube-system logs local-volume-provisioner-vpwlz I1014 07:04:15.204299 1 common.go:344] StorageClass "localvolumeprovisioner" configured with MountDir "/mnt/disks", HostDir "/mnt/disks", VolumeMode "Filesystem", FsType "ext4", BlockCleanerCommand ["/scripts/shred.sh" "2"], NamePattern "*" I1014 07:04:15.204345 1 main.go:63] Loaded configuration: {StorageClassConfig:map[localvolumeprovisioner:{HostDir:/mnt/disks MountDir:/mnt/disks BlockCleanerCommand:[/scripts/shred.sh 2] VolumeMode:Filesystem FsType:ext4 NamePattern:*}] NodeLabelsForPV:[] UseAlphaAPI:false UseJobForCleaning:false MinResyncPeriod:{Duration:5m0s} UseNodeNameOnly:false LabelsForPV:map[] SetPVOwnerRef:false} I1014 07:04:15.204362 1 main.go:64] Ready to run... W1014 07:04:15.204369 1 main.go:73] MY_NAMESPACE environment variable not set, will be set to default. W1014 07:04:15.204375 1 main.go:79] JOB_CONTAINER_IMAGE environment variable not set. I1014 07:04:15.204508 1 common.go:407] Creating client using in-cluster config I1014 07:04:45.206600 1 main.go:128] Could not get node information (remaining retries: 2): Get dial tcp 10.96.0.1:443: i/o timeout I1014 07:05:16.209247 1 main.go:128] Could not get node information (remaining retries: 1): Get dial tcp 10.96.0.1:443: i/o timeout I1014 07:05:47.220246 1 main.go:128] Could not get node information (remaining retries: 0): Get dial tcp 10.96.0.1:443: i/o timeout F1014 07:05:47.220276 1 main.go:131] Could not get node information: Get dial tcp 10.96.0.1:443: i/o timeout 네트워크 이슈로 판단됨describe command k -n kube-system describe pod local-volume-provisioner-vpwlz Controlled By: DaemonSet/local-volume-provisioner Containers: provisioner: Container ID: containerd://664224acf913a25244cec6f62ef47878468bc3ae08acd852fd85ea56b3dd7680 Image: quay.io/external_storage/local-volume-provisioner:v2.4.0 Image ID: quay.io/external_storage/local-volume-provisioner@sha256:cb68797f8c31add7374f23e089c1ade9dfef5f32ff04311e8980e6a5af9d6721 Port: Host Port: State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 255 Started: Fri, 14 Oct 2022 15:57:42 +0900 Finished: Fri, 14 Oct 2022 15:59:14 +0900 Ready: False Restart Count: 9 Environment: MY_NODE_NAME: (v1:spec.nodeName) Mounts: /etc/provisioner/config from provisioner-config (ro) /mnt/disks from disks (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gj4bs (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: provisioner-config: Type: ConfigMap (a volume populated by a ConfigMap) Name: local-provisioner-config Optional: false disks: Type: HostPath (bare host directory volume) Path: /mnt/disks HostPathType: kube-api-access-gj4bs: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: BestEffort Node-Selectors: Tolerations: node-role.kubernetes.io/master:NoSchedule op=Exists node.kubernetes.io/disk-pressure:NoSchedule op=Exists node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists node.kubernetes.io/pid-pressure:NoSchedule op=Exists node.kubernetes.io/unreachable:NoExecute op=Exists node.kubernetes.io/unschedulable:NoSchedule op=Exists Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 41m default-scheduler Successfully assigned kube-system/local-volume-provisioner-vpwlz to worker Normal Pulled 33m (x5 over 41m) kubelet Container image "quay.io/external_storage/local-volume-provisioner:v2.4.0" already present on machine Normal Created 33m (x5 over 41m) kubelet Created container provisioner Normal Started 33m (x5 over 41m) kubelet Started container provisioner Warning BackOff 62s (x115 over 38m) kubelet Back-off restarting failed container controller-5b94ff4f66-t8hgclogs[root@konvoy-master ~]# k logs controller-5b94ff4f66-t8hgc -n metallb-system {"branch":"HEAD","caller":"level.go:63","commit":"v0.12.1","goversion":"gc / go1.16.14 / amd64","level":"info","msg":"MetalLB controller starting version 0.12.1 (commit v0.12.1, branch HEAD)","ts":"2022-10-14T07:39:11.2299578Z","version":"0.12.1"} {"caller":"level.go:63","error":"Get \"": dial tcp 10.96.0.1:443: i/o timeout","level":"error","msg":"failed to create memberlist secret","op":"startup","ts":"2022-10-14T07:39:41.238085492Z"} describe[root@konvoy-master ~]# k -n metallb-system describe po controller-5b94ff4f66-t8hgc Name: controller-5b94ff4f66-t8hgc Namespace: metallb-system Priority: 0 Node: worker/192.168.0.243 Start Time: Fri, 14 Oct 2022 15:22:47 +0900 Labels: app=metallb component=controller konvoy.d2iq.io/cluster-name=preprovisioned-example-qv00m pod-template-hash=5b94ff4f66 Annotations: cni.projectcalico.org/containerID: db892753cc26312c6403ea86d8c4e46b64f19596a0d82d490b87d568b0f27599 cni.projectcalico.org/podIP: 192.168.171.66/32 cni.projectcalico.org/podIPs: 192.168.171.66/32 prometheus.io/port: 7472 prometheus.io/scrape: true Status: Running IP: 192.168.171.66 IPs: IP: 192.168.171.66 Controlled By: ReplicaSet/controller-5b94ff4f66 Containers: controller: Container ID: containerd://321b54c36558cb913ae21addea426d8b4d01d1937439e5d9816cb979bcd90a07 Image: quay.io/metallb/controller:v0.12.1 Image ID: quay.io/metallb/controller@sha256:4db40890faff645102d33a04fbb736265fad2259425f5a01cbb84df524eae53a Port: 7472/TCP Host Port: 0/TCP Args: --port=7472 --config=config --log-level=info State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 1 Started: Fri, 14 Oct 2022 16:39:11 +0900 Finished: Fri, 14 Oct 2022 16:39:41 +0900 Ready: False Restart Count: 18 Liveness: http-get http://:monitoring/metrics delay=10s timeout=1s period=10s #success=1 #failure=3 Readiness: http-get http://:monitoring/metrics delay=10s timeout=1s period=10s #success=1 #failure=3 Environment: METALLB_ML_SECRET_NAME: memberlist METALLB_DEPLOYMENT: controller Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zsgjv (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: kube-api-access-zsgjv: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: BestEffort Node-Selectors: kubernetes.io/os=linux Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Pulled 14m (x17 over 79m) kubelet Container image "quay.io/metallb/controller:v0.12.1" already present on machine Warning BackOff 3m54s (x332 over 78m) kubelet Back-off restarting failed container speaker-gmtdclogs[root@konvoy-master ~]# k -n metallb-system logs speaker-gmtdc Error from server (BadRequest): container "speaker" in pod "speaker-gmtdc" is waiting to start: CreateContainerConfigError describe[root@konvoy-master ~]# k -n metallb-system describe po speaker-gmtdc Name: speaker-gmtdc Namespace: metallb-system Priority: 0 Node: worker/192.168.0.243 Start Time: Fri, 14 Oct 2022 15:22:47 +0900 Labels: app=metallb component=speaker controller-revision-hash=78ccc84775 konvoy.d2iq.io/cluster-name=preprovisioned-example-qv00m pod-template-generation=1 Annotations: prometheus.io/port: 7472 prometheus.io/scrape: true Status: Pending IP: 192.168.0.243 IPs: IP: 192.168.0.243 Controlled By: DaemonSet/speaker Containers: speaker: Container ID: Image: quay.io/metallb/speaker:v0.12.1 Image ID: Ports: 7472/TCP, 7946/TCP, 7946/UDP Host Ports: 7472/TCP, 7946/TCP, 7946/UDP Args: --port=7472 --config=config --log-level=info State: Waiting Reason: CreateContainerConfigError Ready: False Restart Count: 0 Liveness: http-get http://:monitoring/metrics delay=10s timeout=1s period=10s #success=1 #failure=3 Readiness: http-get http://:monitoring/metrics delay=10s timeout=1s period=10s #success=1 #failure=3 Environment: METALLB_NODE_NAME: (v1:spec.nodeName) METALLB_HOST: (v1:status.hostIP) METALLB_ML_BIND_ADDR: (v1:status.podIP) METALLB_ML_LABELS: app=metallb,component=speaker METALLB_ML_SECRET_KEY: Optional: false Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jm75k (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: kube-api-access-jm75k: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: BestEffort Node-Selectors: kubernetes.io/os=linux Tolerations: node-role.kubernetes.io/master:NoSchedule op=Exists node.kubernetes.io/disk-pressure:NoSchedule op=Exists node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/network-unavailable:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists node.kubernetes.io/pid-pressure:NoSchedule op=Exists node.kubernetes.io/unreachable:NoExecute op=Exists node.kubernetes.io/unschedulable:NoSchedule op=Exists Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Pulled 2m14s (x323 over 72m) kubelet Container image "quay.io/metallb/speaker:v0.12.1" already present on machine speaker-t8jxllogs[root@konvoy-master ~]# k -n metallb-system logs speaker-t8jxl Error from server (BadRequest): container "speaker" in pod "speaker-t8jxl" is waiting to start: CreateContainerConfigError describeMETALLB_ML_SECRET_KEY 키의 값을 가져 오지 못함[root@konvoy-master ~]# k -n metallb-system describe po speaker-t8jxl Name: speaker-t8jxl Namespace: metallb-system Priority: 0 Node: master/192.168.0.242 Start Time: Fri, 14 Oct 2022 15:20:53 +0900 Labels: app=metallb component=speaker controller-revision-hash=78ccc84775 konvoy.d2iq.io/cluster-name=preprovisioned-example-qv00m pod-template-generation=1 Annotations: prometheus.io/port: 7472 prometheus.io/scrape: true Status: Pending IP: 192.168.0.242 IPs: IP: 192.168.0.242 Controlled By: DaemonSet/speaker Containers: speaker: Container ID: Image: quay.io/metallb/speaker:v0.12.1 Image ID: Ports: 7472/TCP, 7946/TCP, 7946/UDP Host Ports: 7472/TCP, 7946/TCP, 7946/UDP Args: --port=7472 --config=config --log-level=info State: Waiting Reason: CreateContainerConfigError Ready: False Restart Count: 0 Liveness: http-get http://:monitoring/metrics delay=10s timeout=1s period=10s #success=1 #failure=3 Readiness: http-get http://:monitoring/metrics delay=10s timeout=1s period=10s #success=1 #failure=3 Environment: METALLB_NODE_NAME: (v1:spec.nodeName) METALLB_HOST: (v1:status.hostIP) METALLB_ML_BIND_ADDR: (v1:status.podIP) METALLB_ML_LABELS: app=metallb,component=speaker METALLB_ML_SECRET_KEY: Optional: false Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-t4zc5 (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: kube-api-access-t4zc5: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: BestEffort Node-Selectors: kubernetes.io/os=linux Tolerations: node-role.kubernetes.io/master:NoSchedule op=Exists node.kubernetes.io/disk-pressure:NoSchedule op=Exists node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/network-unavailable:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists node.kubernetes.io/pid-pressure:NoSchedule op=Exists node.kubernetes.io/unreachable:NoExecute op=Exists node.kubernetes.io/unschedulable:NoSchedule op=Exists Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Pulled 28m (x235 over 78m) kubelet Container image "quay.io/metallb/speaker:v0.12.1" already present on machine Warning Failed 24m (x12 over 26m) kubelet Error: secret "memberlist" not found Normal Pulled 90s (x117 over 26m) kubelet Container image "quay.io/metallb/speaker:v0.12.1" already present on machine

kubernetes metallb 및 네트워크 설정 질문드립니다.

0개의 답변

Q&A 목록