![[PukiWiki]](image/pukiwiki.png "[PukiWiki]")
|
k8s GitOps |
| CPU | Core i5 (8Gen) 6Core 3GHz |
| MEM | 32GB |
| SSD | 512GB (/dev/nvme0n1p) |
| SSD | 64GB (/dev/sdb) |
| HDD | 1TB (/dev/sda) |
| GPU | RTX2070 8GB |
| /dev/nvme0n1p1 | /boot/efi | efi | 537MB | 537MBの根拠は http://www.rodsbooks.com/efi-bootloaders/principles.html |
| /dev/nvme0n1p2 | / | ext4 | 512GB | |
| /dev/sda1 | swap | 128GB | hibernateのため、最低必要量(メモリ容量の2倍)の2倍を確保 | |
| /deb/bcache0 | /var | ext4 | 796GB | bcache : Backend /dev/sda1 (796GB), Cache /dev/sdb1 (64GB) |
| netmask | 192.168.10.0/24 | |
| gw | 192.168.10.1 | |
| このマシン | 192.168.10.32 | DHCPで、NICのMACアドレスをもとに固定割付 |
| MetalLB | 192.168.10.100-132 | Kubernetes上で動いているPodに割り付けるIPアドレス |
$ sudo apt-get update $ sudo apt-get install -y curl openssh-server ca-certificates tzdata perl $ sudo apt-get install -y postfix $ curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | sudo bash $ sudo EXTERNAL_URL="http://gpu1.at-sushi.com" apt-get install gitlab-ee
$ sudo snap install microk8s --classic
インストールの確認
$ sudo microk8s status --wait-ready
自分を microk8s の管理ユーザにする
$ sudo usermod -a -G microk8s $USER
alias を設定して kubectl を使えるようにする (~/.bash_aliases は、~/.bashrc から読み込まれる)
$ cat ~/.bash_aliases alias kubectl='microk8s kubectl'
Terminal再起動か、$ source ~/.bash_aliases コマンドを実行して、構築確認を行う
$ kubectl get nodes NAME STATUS ROLES AGE VERSION gpu1 Ready <none> 38h v1.20.4-34+1ae8c29bbb48f7
--%%config ${SNAP_DATA}/args/containerd.toml%%
--%%root ${SNAP_COMMON}/var/lib/containerd%%
--%%state ${SNAP_COMMON}/run/containerd%%
--%%address ${SNAP_COMMON}/run/containerd.sock%%--%%config ${SNAP_DATA}/args/containerd.toml%%
--%%root /data/var/lib/containerd%%
--%%state /data/run/containerd%%
--%%address ${SNAP_COMMON}/run/containerd.sock%%$ sudo cp -R /var/snap/microk8s/common/run /data $ sudo cp -R /var/snap/microk8s/common/var/lib /data/var
$ sudo microk8s stop $ sudo microk8s start
$ sudo microk8s.enable rbac dns storage gpu $ sudo microk8s.enable metallb Enter each IP address range delimited by comma : 192.168.10.100-192.168.10.132
$ microk8s.status
microk8s is running
high-availability: no
datastore master nodes: 127.0.0.1:19001
datastore standby nodes: none
addons:
enabled:
dns # CoreDNS
gpu # Automatic enablement of Nvidia CUDA
ha-cluster # Configure high availability on the current node
metallb # Loadbalancer for your Kubernetes cluster
rbac # Role-Based Access Control for authorisation
storage # Storage class; allocates storage from host directory
Ingress や Prometheus, Fluentd なんかは、Gitlab からインストールするので、microk8s でインストールする addon は必要最低限のものにとどめておく
$ microk8s.config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CR ....
server: https://gpu1.at-sushi.com:16443
name: microk8s-cluster
contexts:
- context:
cluster: microk8s-cluster
user: admin
name: microk8s
current-context: microk8s
kind: Config
preferences: {}
users:
- name: admin
user:
token: UGhyTXh ....$ kubectl get secrets
NAME TYPE DATA AGE
default-token-x8sf2 kubernetes.io/service-account-token 3 12d
$ kubectl get secret default-token-x8sf2 -o jsonpath="{['data']['ca\.crt']}" | base64 --decode
-----BEGIN CERTIFICATE-----
MIIDATCCAemgAwIBAgIJAMQnSVF96tY0MA0GCSqGSIb3DQEBCwUAMBcxFTATBgNV
...
-----END CERTIFICATE-----apiVersion: v1
kind: ServiceAccount
metadata:
name: gitlab
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: gitlab-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: gitlab
namespace: kube-system
$ kubectl apply -f gitlab-admin-service-account.yaml
$ kubectl -n kube-system get secret | grep gitlab
gitlab-token-lsk8v kubernetes.io/service-account-token 3 5m35s
$ kubectl -n kube-system describe secret gitlab-token-lsk8v
Name: gitlab-token-lsk8v
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: gitlab
kubernetes.io/service-account.uid: b482c0e6-420e-4192-9f62-7dd960be317c
Type: kubernetes.io/service-account-token
Data
====
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIs ...
ca.crt: 1103 bytes
Something went wrong while installing Prometheus Kubernetes error: 500
$ openssl req -newkey rsa:4096 -nodes -sha256 -keyout registry.key -x509 -days 3650 -out registry.crt
/etc/gitlab# diff -u gitlab.rb.20210412 gitlab.rb
--- gitlab.rb.20210412 2021-04-12 20:40:50.877587005 +0900
+++ gitlab.rb 2021-04-12 21:05:02.792357247 +0900
@@ -732,6 +732,7 @@
################################################################################
# registry_external_url 'https://registry.example.com'
+registry_external_url 'https://192.168.10.32.xip.io:5050'
### Settings used by GitLab application
# gitlab_rails['registry_enabled'] = true
@@ -1776,6 +1777,9 @@
# it listens on this port
# registry_nginx['listen_port'] = 5050
+registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/registry.crt"
+registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/registry.key"
+
$ sudo gitlab-ctl reconfigure
$ openssl s_client -showcerts -servername 192.168.10.32.xip.io -connect 192.168.10.32:5050
$ sudo addgroup --system docker $ sudo adduser $USER docker $ newgrp docker $ sudo snap install docker
basedomain.png 1368件
[詳細]
prometheus.png 1352件
[詳細]
error01.png 1342件
[詳細]
error02.png 1383件
[詳細]
gitlab01.png 1475件
[詳細]
gitlab02.png 1363件
[詳細]
gitlab03.png 1360件
[詳細]
gitlab04.png 1389件
[詳細]
ingress.png 1309件
[詳細]
