# 쿠버네티스 대쉬보드 란?
쿠버네티스 대시보드는 웹 기반 쿠버네티스 유저 인터페이스이다. 대시보드를 통해 컨테이너화 된 애플리케이션을 쿠버네티스 클러스터에 배포할 수 있고, 컨테이너화 된 애플리케이션을 트러블슈팅할 수 있으며, 클러스터 리소스들을 관리할 수 있다.
1. 설치
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.4/aio/deploy/recommended.yaml- 확인
kubectl get services -n kubernetes-dashboardk8s-master-node@k8smasternode-Virtual-Machine:~/yaml$ kubectl get services -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.110.212.64 <none> 8000/TCP 15m
kubernetes-dashboard ClusterIP 10.111.140.89 <none> 443/TCP 15m- 실행 (새로운 터미널에서 실행)
kubectl proxy- 실행 확인
curl http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/k8s-master-node@k8smasternode-Virtual-Machine:~/yaml$ curl http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
<!--
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
--><!DOCTYPE html><html lang="en"><head>
<meta charset="utf-8">
<title>Kubernetes Dashboard</title>
<link rel="icon" type="image/png" href="assets/images/kubernetes-logo.png">
<meta name="viewport" content="width=device-width">
<style>body,html{height:100%;margin:0;}</style><link rel="stylesheet" href="styles.aa0538c9a91ebbb04705.css" media="print" onload="this.media='all'"><noscript><link rel="stylesheet" href="styles.aa0538c9a91ebbb04705.css"></noscript></head>
<body>
<kd-root></kd-root>
<script src="runtime.1a20bc8321eb559541a1.js" defer=""></script><script src="polyfills.2565916e4afd13edaa84.js" defer=""></script><script src="scripts.f76573725d49abb057d3.js" defer=""></script><script src="en.main.7f7baee1f12d075d7cb9.js" defer=""></script>api를 통해 html내용을 받을 수 있다면 성공적으로 dashboard가 실행되고 있다는 뜻이다!
위의 curl은 kube-proxy를 통해서 apiserver로 https:kubernetes-dashboard: 요청을 보내 html 응답을 반환해준 경우이다.
그러나 kube-proxy 통하지 않고 direct로 kubernetes-dashboard로 접속하는 방법애 대해 알아보자.
2. 대쉬보드 서비스 수정하기 및 접속하기
kubectl edit services kubernetes-dashboard -n kubernetes-dashboard여기서 spec.type이 ClusterIP로 되어있는데 이것을 NodePort로 변경 후 wq! 로 저장한다.
- 확인을 해보면 Nodeport 로 변경된것을 확인할 수 있다.
k8s-master-node@k8smasternode-Virtual-Machine:~/yaml$ kubectl get services -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.110.212.64 <none> 8000/TCP 24m
kubernetes-dashboard NodePort 10.111.140.89 <none> 443:30212/TCP 24m- 접속을 해보자 {hostip}:30212
- hostip 확인은 kubectl get nodes -o wide 로 node의 ip를 확인하면 된다.
- 오류 확인 : 웹페이지에서 실행 시키면 아래와 같은 오류가 발생 할 수 있는데, 이것은 url 입력 시 https:// 를 빼먹고 바로 ip와 포트명만 입력해서 이다. 쿠버네티스 대쉬보드 서비스 자체가 https에서 동작하게끔 되어 있기 때문에 이런 문제가 발생 한거 같다.
client sent an http request to an https server- 정상 접속 확인
3. 대시보드 서비서 접속하기
- 모든것이 정상적으로 설정이 되었다면 login 화면으로 접속이 된다.
- apiserver에 접근하여 클러스터 환경의 자원들을 가져와 대시보드를 꾸미려면 권한이 필요하다.
Role-based access control (RBAC)를 기반으로 apiserver에 인증, 권한을 관리하는데 쿠버네티스 대시보드에 로그인하고 올바른 인증을 거쳐야만 정상적으로 대시보드를 사용할 수 있다.
- 모든 ServiceAccount는 apiserver에 접근하기위하여 secret을 가지고 있고 이 안에는 Bearer Token값이 있다.
http 프로토콜을 사용하여 인증하는 여러가지 방법 중 쿠버네티스 대시보드에 접속하기위해서 Bearer Token을 사용해 보자.
- 토큰값 확인하기 : 토큰값은 kubernetes-dashboard-token-tmg8t 로 확인이 된다.
k8s-master-node@k8smasternode-Virtual-Machine:~/yaml$ kubectl describe serviceaccount kubernetes-dashboard -n kubernetes-dashboard
Name: kubernetes-dashboard
Namespace: kubernetes-dashboard
Labels: k8s-app=kubernetes-dashboard
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: kubernetes-dashboard-token-tmg8t
Tokens: kubernetes-dashboard-token-tmg8t
Events: <none>- secret 을 확인해 보자.
kubectl describe secret kubernetes-dashboard-token-tmg8t -n kubernetes-dashboard- 토큰값 확인
k8s-master-node@k8smasternode-Virtual-Machine:~/yaml$ kubectl describe secret kubernetes-dashboard-token-tmg8t -n kubernetes-dashboard
Name: kubernetes-dashboard-token-tmg8t
Namespace: kubernetes-dashboard
Labels: <none>
Annotations: kubernetes.io/service-account.name: kubernetes-dashboard
kubernetes.io/service-account.uid: dfa50da0-f6eb-4ca5-8e4a-f6149c84ba3f
Type: kubernetes.io/service-account-token
Data
====
token: eyJhbGciOiJSUzI1NiIsImtpZCI6InRGdUstd3ZwcVFlWE1aZlVzdGJJYU0yNU4wOW5VOUNINFJwTnBldkJNWDAifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi10bWc4dCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImRmYTUwZGEwLWY2ZWItNGNhNS04ZTRhLWY2MTQ5Yzg0YmEzZiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.DdqWinN26EQViQ95EII0DjCpXXzO6eAeVC2p3NylaDXCCDaBF27RIaU8WFspJxXtTnaiB6wF_2KDtkPDuUrfnkvq0RRjR7jYqHJgG9UEcNaJYWocHnUEQK88s2HscMpbaZWDNDhxRMay2yYOpbtx-39sEYxYVtyf7BYceLsMeE1s_WVxO7ruy-vEW6N-fCrXvNA57Rr6ru5qpWhVnfuAjo_58BpgCM-iLo9lOxdJAW7EOXwREYzpR0Btt_EVV_Wbd7CE5Ij5K2ghqMUpA_MfkW4Wzy5r7iMwenbGbsVm6tuerLdUrRzvKvV04eLV0gkm8CsVHugqpsZYTahuVDuhVg
ca.crt: 1099 bytes
namespace: 20 bytes- grep과 awk를 사용하여 값만 뽑자.
k8s-master-node@k8smasternode-Virtual-Machine:~/yaml$ kubectl describe secret kubernetes-dashboard-token-tmg8t -n kubernetes-dashboard | grep token: | awk '{print $2}'
eyJhbGciOiJSUzI1NiIsImtpZCI6InRGdUstd3ZwcVFlWE1aZlVzdGJJYU0yNU4wOW5VOUNINFJwTnBldkJNWDAifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi10bWc4dCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImRmYTUwZGEwLWY2ZWItNGNhNS04ZTRhLWY2MTQ5Yzg0YmEzZiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.DdqWinN26EQViQ95EII0DjCpXXzO6eAeVC2p3NylaDXCCDaBF27RIaU8WFspJxXtTnaiB6wF_2KDtkPDuUrfnkvq0RRjR7jYqHJgG9UEcNaJYWocHnUEQK88s2HscMpbaZWDNDhxRMay2yYOpbtx-39sEYxYVtyf7BYceLsMeE1s_WVxO7ruy-vEW6N-fCrXvNA57Rr6ru5qpWhVnfuAjo_58BpgCM-iLo9lOxdJAW7EOXwREYzpR0Btt_EVV_Wbd7CE5Ij5K2ghqMUpA_MfkW4Wzy5r7iMwenbGbsVm6tuerLdUrRzvKvV04eLV0gkm8CsVHugqpsZYTahuVDuhVg- 로그인 : 토큰값을 넣으면 로그인이 된다.
- 로그인이 되었는데... 정보가 아무것도 없다.
- 확인해보니 admin Role 권한이 없어서 아래와 같은 오류가 나왔다.
serviceaccounts is forbidden: User "system:serviceaccount:kubernetes-dashboard:kubernetes-dashboard" cannot list resource "serviceaccounts" in API group "" in the namespace "default"
4. Role 확인 및 권한 변경 및 추가하기
- Role 확인
kubectl describe clusterrole kubernetes-dashboardk8s-master-node@k8smasternode-Virtual-Machine:~/yaml$ kubectl describe clusterrole kubernetes-dashboard
Name: kubernetes-dashboard
Labels: k8s-app=kubernetes-dashboard
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
nodes.metrics.k8s.io [] [] [get list watch]
pods.metrics.k8s.io [] [] [get list watch]- Role List 확인
kubectl get clusterrolek8s-master-node@k8smasternode-Virtual-Machine:~/yaml$ kubectl get clusterrole
NAME CREATED AT
admin 2021-11-04T16:21:31Z
cluster-admin 2021-11-04T16:21:31Z
edit 2021-11-04T16:21:31Z
flannel 2021-11-04T16:37:04Z
kubeadm:get-nodes 2021-11-04T16:21:33Z
kubernetes-dashboard 2021-11-06T07:40:41Z
system:aggregate-to-admin 2021-11-04T16:21:31Z
system:aggregate-to-edit 2021-11-04T16:21:31Z
system:aggregate-to-view 2021-11-04T16:21:31Z
system:auth-delegator 2021-11-04T16:21:31Z
system:basic-user 2021-11-04T16:21:31Z
system:certificates.k8s.io:certificatesigningrequests:nodeclient 2021-11-04T16:21:31Z
system:certificates.k8s.io:certificatesigningrequests:selfnodeclient 2021-11-04T16:21:31Z
system:certificates.k8s.io:kube-apiserver-client-approver 2021-11-04T16:21:31Z
system:certificates.k8s.io:kube-apiserver-client-kubelet-approver 2021-11-04T16:21:31Z
system:certificates.k8s.io:kubelet-serving-approver 2021-11-04T16:21:31Z
system:certificates.k8s.io:legacy-unknown-approver 2021-11-04T16:21:31Z
system:controller:attachdetach-controller 2021-11-04T16:21:31Z
system:controller:certificate-controller 2021-11-04T16:21:31Z
system:controller:clusterrole-aggregation-controller 2021-11-04T16:21:31Z
system:controller:cronjob-controller 2021-11-04T16:21:31Z
system:controller:daemon-set-controller 2021-11-04T16:21:31Z
system:controller:deployment-controller 2021-11-04T16:21:31Z
system:controller:disruption-controller 2021-11-04T16:21:31Z
system:controller:endpoint-controller 2021-11-04T16:21:31Z
system:controller:endpointslice-controller 2021-11-04T16:21:31Z
system:controller:endpointslicemirroring-controller 2021-11-04T16:21:31Z
system:controller:ephemeral-volume-controller 2021-11-04T16:21:31Z
system:controller:expand-controller 2021-11-04T16:21:31Z
system:controller:generic-garbage-collector 2021-11-04T16:21:31Z
system:controller:horizontal-pod-autoscaler 2021-11-04T16:21:31Z
system:controller:job-controller 2021-11-04T16:21:31Z
system:controller:namespace-controller 2021-11-04T16:21:31Z
system:controller:node-controller 2021-11-04T16:21:31Z
system:controller:persistent-volume-binder 2021-11-04T16:21:31Z
system:controller:pod-garbage-collector 2021-11-04T16:21:31Z
system:controller:pv-protection-controller 2021-11-04T16:21:31Z
system:controller:pvc-protection-controller 2021-11-04T16:21:31Z
system:controller:replicaset-controller 2021-11-04T16:21:31Z
system:controller:replication-controller 2021-11-04T16:21:31Z
system:controller:resourcequota-controller 2021-11-04T16:21:31Z
system:controller:root-ca-cert-publisher 2021-11-04T16:21:31Z
system:controller:route-controller 2021-11-04T16:21:31Z
system:controller:service-account-controller 2021-11-04T16:21:31Z
system:controller:service-controller 2021-11-04T16:21:31Z
system:controller:statefulset-controller 2021-11-04T16:21:31Z
system:controller:ttl-after-finished-controller 2021-11-04T16:21:31Z
system:controller:ttl-controller 2021-11-04T16:21:31Z
system:coredns 2021-11-04T16:21:33Z
system:discovery 2021-11-04T16:21:31Z
system:heapster 2021-11-04T16:21:31Z
system:kube-aggregator 2021-11-04T16:21:31Z
system:kube-controller-manager 2021-11-04T16:21:31Z
system:kube-dns 2021-11-04T16:21:31Z
system:kube-scheduler 2021-11-04T16:21:31Z
system:kubelet-api-admin 2021-11-04T16:21:31Z
system:monitoring 2021-11-04T16:21:31Z
system:node 2021-11-04T16:21:31Z
system:node-bootstrapper 2021-11-04T16:21:31Z
system:node-problem-detector 2021-11-04T16:21:31Z
system:node-proxier 2021-11-04T16:21:31Z
system:persistent-volume-provisioner 2021-11-04T16:21:31Z
system:public-info-viewer 2021-11-04T16:21:31Z
system:service-account-issuer-discovery 2021-11-04T16:21:31Z
system:volume-scheduler 2021-11-04T16:21:31Z
view 2021-11-04T16:21:31Z- 관리자 토큰을 생성하여 모든 리소스에 접근하는 Role 만들기
관리자 토큰을 생성하기 위해 아래 yaml 파일을 작성합니다.
대부분의 경우 cluster-admin role을 부여하고 ClusterRoleBinding을 생성합니다.
- yaml 파일을 생성
vi kuebernetes-dashboard-service-account.yaml- 아래의 코드 복사 > 붙여넣기
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kube-system- 생성한 yaml 파일을 실행
kubectl apply -f kuebernetes-dashboard-service-account.yaml5. admin-user 로 부터 토큰 생성 및 대시보드 페이지 접근
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')k8s-master-node@k8smasternode-Virtual-Machine:~/yaml$ kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
Name: admin-user-token-wq74l
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: admin-user
kubernetes.io/service-account.uid: f044b314-2028-45d7-9b02-4d6dea73407f
Type: kubernetes.io/service-account-token
Data
====
token: eyJhbGciOiJSUzI1NiIsImtpZCI6InRGdUstd3ZwcVFlWE1aZlVzdGJJYU0yNU4wOW5VOUNINFJwTnBldkJNWDAifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLXdxNzRsIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJmMDQ0YjMxNC0yMDI4LTQ1ZDctOWIwMi00ZDZkZWE3MzQwN2YiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.BfFDnJKSFHmCpZ2XA2KdheXO_3qKNTQGiXM5_WQ5469NGa_fEiJXZ2krVsK2-vB3f7roiwT7cFNfjNQ9mFqOKqq61fosZxUWHIOOM1wiQ6nTXmJSCZ8VXFBPPOKF7PSJIaSui-2zXDaZEcGGjiUVUKGm_DLHfEH6Vl0PJ6ecHPTYDBWIgJwMxRDMLeVbfPvnEKcSMg84qS8UoTsQ9-j4wPyqNkFF9TtO-AXijZ4LanJ9ydaX2OX7ueZ0jeUL0qXj6W-EaTVtVg_3d40ufbtWVzXexBjnubSOc_GI6RN1xRKeU8iGioIwS4QibI3KwW3UDkaWrsEVkPszp7UVgg9E1g
ca.crt: 1099 bytes
namespace: 11 bytes- 대시보드 페이지 접근 > Nodes > 3개의 노드가 활성화 되어 있는것 확인 및 모니터링이 가능하다.
- 기타
참고로 k8s에서는 namespace마다 token을 부여할 수 있으며, 별도의 토큰을 가지고 있습니다. 또한 특정 namespace마다 viewer, admin 등 권한을 다르게 token을 생성할 수 있습니다.
- 끝 -