1 - minikube
Minikube is a tool that makes it easy to run Kubernetes locally.
Minikube runs a single-node Kubernetes cluster inside a Virtual Machine (VM) on your laptop for users looking to try out Kubernetes or develop with it day-to-day.
Minikube 是一个易于在本地运行 Kubernetes 的工具，
可在笔记本电脑上的虚拟机内轻松创建单机版 Kubernetes 集群，便于尝试 Kubernetes 或使用 Kubernetes 日常开发。
HomePage：
https://minikube.sigs.k8s.io/
Documentation：
https://minikube.sigs.k8s.io/docs/
Getting Started：
https://minikube.sigs.k8s.io/docs/start/
Installation：
https://minikube.sigs.k8s.io/docs/start/linux/
GitHub：
https://github.com/kubernetes/minikube
Releases：
https://github.com/kubernetes/minikube/releases
kubernetes.io
Installing Minikube：
https://kubernetes.io/docs/tasks/tools/install-minikube/
Installing Kubernetes with Minikube：
https://kubernetes.io/docs/setup/learning-environment/minikube/
Hello Minikube
Hello Minikube：
https://kubernetes.io/docs/tutorials/hello-minikube/
This tutorial shows you how to run a simple Hello World Node.js app on Kubernetes using Minikube and Katacoda.
Minikube是由Kubernetes开源社区开发，专为本地部署而设计的一个版本，但Minikube只能局限于包含一个节点的本地集群，因此对于生产级别的多节点Kubernetes集群来说，很难在本地进行模拟测试。。
从1.5.0版本开始，Minikube缺省使用本地最好的驱动来创建Kubernetes本地环境，因此需先确保安装了类似VirtualBox、Hyper-V、KVM、Docker等。
2 - 环境准备
虚拟机中已安装docker
[anliven@anliven ~]$ uname -a
Linux anliven 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[anliven@anliven ~]$
[anliven@anliven ~]$ cat /etc/system-release
CentOS Linux release 7.8.2003 (Core)
[anliven@anliven ~]$
[anliven@anliven ~]$ docker version
Client: Docker Engine - Community
Version: 19.03.11
API version: 1.40
Go version: go1.13.10
Git commit: 42e35e61f3
Built: Mon Jun 1 09:13:48 2020
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.11
API version: 1.40 (minimum version 1.12)
Go version: go1.13.10
Git commit: 42e35e61f3
Built: Mon Jun 1 09:12:26 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.13
GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683
[anliven@anliven ~]$
3 - 用户权限设置
[root@anliven ~]# visudo
[root@anliven ~]#
[root@anliven ~]# cat /etc/sudoers |grep anliven
anliven ALL=(ALL) NOPASSWD: ALL
[root@anliven ~]#
[root@anliven ~]# exit
[anliven@anliven ~]# sudo usermod -aG docker $USER && newgrp docker
4 - 安装kubectl
使用 kubectl 与集群进行交互。
kubectl是Kubernetes命令行工具，可以部署和管理应用， 查看各种资源， 创建、 删除和更新各种组件。
Overview of kubectl：https://kubernetes.io/docs/reference/kubectl/overview/
Install and Set Up kubectl：https://kubernetes.io/docs/tasks/tools/install-kubectl/
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
chmod +x kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
kubectl version --client
[anliven@anliven K8s]$ pwd
/home/anliven/K8s
[anliven@anliven K8s]$
[anliven@anliven K8s]$ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 41.0M 100 41.0M 0 0 2819k 0 0:00:14 0:00:14 --:--:-- 2832k
[anliven@anliven K8s]$
[anliven@anliven K8s]$ chmod +x kubectl
[anliven@anliven K8s]$
[anliven@anliven K8s]$ sudo mv ./kubectl /usr/local/bin/kubectl
[anliven@anliven K8s]$
[anliven@anliven K8s]$ ls -l /usr/local/bin/kubectl
-rwxrwxr-x 1 anliven anliven 43003904 Oct 14 22:43 /usr/local/bin/kubectl
[anliven@anliven K8s]$
[anliven@anliven K8s]$ kubectl version --client
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
[anliven@anliven K8s]$
kubectl命令帮助
kubectl -h # 查看子命令列表
kubectl options # 查看全局选项
Use "kubectl <command> --help" for more information about a given command.
kubectl <command> --help # 查看子命令的帮助
Use "kubectl options" for a list of global command-line options (applies to all commands).
kubectl [command] [PARAMS] -o=<format> # 设置输出格式（如 json、yaml、jsonpath 等）
kubectl explain RESOURCE [options] # 查看资源的定义
kubectl explain -h
kubectl get [flags] [options] # 显示一个或多个资源
kubectl get -h
5 - 安装minikube
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
minikube version
[anliven@anliven K8s]$ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 56.0M 100 56.0M 0 0 2510k 0 0:00:22 0:00:22 --:--:-- 2234k
[anliven@anliven K8s]$
[anliven@anliven K8s]$ sudo install minikube-linux-amd64 /usr/local/bin/minikube
[anliven@anliven K8s]$
[anliven@anliven K8s]$ ls -l /usr/local/bin/minikube
-rwxr-xr-x 1 root root 58733392 Oct 14 22:46 /usr/local/bin/minikube
[anliven@anliven K8s]$
[anliven@anliven K8s]$ minikube version
minikube version: v1.14.0
commit: b09ee50ec047410326a85435f4d99026f9c4f5c4
6 - 从github下载并安装kubectl和minikube
如果无法访问google站点，也可以从github下载并安装kubectl和minikube。
kubectl
访问 https://github.com/kubernetes/kubernetes/releases
点击类似CHANGELOG-*.*.md的链接，在“Client Binaries”区域找到匹配的二进制包，
下载并解压，然后放入/usr/local/bin目录
minikube
找到匹配的二进制包并下载： https://github.com/kubernetes/minikube/releases/
curl -Lo minikube https://github.com/kubernetes/minikube/releases/download/v1.14.0/minikube-linux-arm64
chmod +x minikube
sudo mv minikube /usr/local/bin
也可以使用阿里云的minikube地址： https://github.com/AliyunContainerService/minikube
curl -Lo minikube https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.13.0/minikube-linux-amd64
chmod +x minikube
sudo mv minikube /usr/local/bin/
7 - 启动与运行minikube
[anliven@anliven ~]$ minikube start
* minikube v1.14.0 on Centos 7.8.2003 (vbox/amd64)
* Automatically selected the docker driver # 本例中自动选择了docker驱动来创建Kubernetes本地环境
* Starting control plane node minikube in cluster minikube
* Pulling base image ... # 第一次会拉取基础镜像，用时较久
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.19.2 on Docker 19.03.8 ...
> kubelet.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubeadm.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubectl.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubeadm: 37.30 MiB / 37.30 MiB [-------------] 100.00% 822.08 KiB p/s 47s
> kubectl: 41.01 MiB / 41.01 MiB [-------------] 100.00% 881.98 KiB p/s 48s
> kubelet: 104.88 MiB / 104.88 MiB [------------] 100.00% 1.58 MiB p/s 1m6s
* Verifying Kubernetes components...
* Enabled addons: storage-provisioner, default-storageclass
* Done! kubectl is now configured to use "minikube" by default
[anliven@anliven ~]$
[anliven@anliven ~]$ docker images |grep k8s
gcr.io/k8s-minikube/kicbase v0.0.13 90f1294ff9ac 2 weeks ago 800MB
[anliven@anliven ~]$
[anliven@anliven ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5d520392622e gcr.io/k8s-minikube/kicbase:v0.0.13 "/usr/local/bin/entr…" 4 hours ago Up 4 hours 127.0.0.1:32775->22/tcp, 127.0.0.1:32774->2376/tcp, 127.0.0.1:32773->5000/tcp, 127.0.0.1:32772->8443/tcp minikube
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube ip # 获取minikube的ip地址
192.168.49.2
[anliven@anliven ~]$
可以根据需要配置启动参数，例如minikube start --cpus=2 --memory=2048 -- --kubernetes-version=v1.19.3
--driver 从1.5.0版本开始，Minikube缺省使用本地最好的驱动来创建Kubernetes本地环境
--image-mirror-country cn 将缺省利用 registry.cn-hangzhou.aliyuncs.com/google_containers 作为安装Kubernetes的容器镜像仓库
--iso-url 指定镜像地址下载相应的 .iso 文件
--registry-mirror 为了拉取Docker Hub镜像，需要为 Docker daemon 配置镜像加速
--cpus=2 为minikube虚拟机分配CPU核数
--memory=2048mb 为minikube虚拟机分配内存数
--kubernetes-version minikube 虚拟机将使用的 kubernetes 版本
此时，使用kubectl get nodes检查集群所包含节点的命令可以返回minikube的主机名和状态。
[anliven@anliven ~]$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready master 130d v1.19.2
通过minikube start命令可以重启现有的minikube
[anliven@anliven ~]$ minikube start
* minikube v1.14.0 on Centos 7.8.2003 (vbox/amd64)
* Using the docker driver based on existing profile
* Starting control plane node minikube in cluster minikube
* Restarting existing docker container for "minikube" ... # 重启现有的minikube
* Preparing Kubernetes v1.19.2 on Docker 19.03.8 ...
* Verifying Kubernetes components...
* Enabled addons: default-storageclass, storage-provisioner
* Done! kubectl is now configured to use "minikube" by default
[anliven@anliven ~]$
如果minikube突然出现不稳定的状况，可以使用minikube stop和minikube delete命令删除当前的minikube，然后通过 minikube start命令重新安装。
一个部署示例：
[anliven@anliven ~]$ kubectl get all -A # 查看所有的资源
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-f9fd979d6-qjj7q 1/1 Running 0 4m
kube-system pod/etcd-minikube 1/1 Running 0 4m2s
kube-system pod/kube-apiserver-minikube 1/1 Running 0 4m2s
kube-system pod/kube-controller-manager-minikube 1/1 Running 0 4m1s
kube-system pod/kube-proxy-ldbqb 1/1 Running 0 4m
kube-system pod/kube-scheduler-minikube 1/1 Running 0 4m1s
kube-system pod/storage-provisioner 1/1 Running 0 3m58s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4m8s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 4m5s
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 4m5s
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system deployment.apps/coredns 1/1 1 1 4m5s
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system replicaset.apps/coredns-f9fd979d6 1 1 1 4m
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4m12s
[anliven@anliven ~]$
[anliven@anliven ~]$ docker images|grep nginx # 查看镜像
nginx latest 7e4d58f0e5f3 5 months ago 133MB
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl create deployment nginx --image=nginx:latest # 部署应用
deployment.apps/nginx created
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-55649fd747-lg4d6 0/1 ContainerCreating 0 17s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4m51s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 0/1 1 0 17s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-55649fd747 1 1 0 17s
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl expose deployment nginx --type=NodePort --port=8888 # 暴露8080端口，类型为NodePort（在每个Node上分配一个端口作为外部访问入口）
service/nginx exposed
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-55649fd747-lg4d6 1/1 Running 0 9m26s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 10m
service/nginx NodePort 10.102.183.1 <none> 8888:31765/TCP 8m33s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 9m26s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-55649fd747 1 1 1 9m26s
[anliven@anliven ~]$
8 - Minikube 命令
Use "minikube <command> --help" for more information about a given command.
Use "minikube options" for a list of global command-line options (applies to all commands).
[anliven@anliven ~]$ minikube options
The following options can be passed to any command:
--alsologtostderr=false: log to standard error as well as files
-b, --bootstrapper='kubeadm': The name of the cluster bootstrapper that will set up the Kubernetes cluster.
--log_backtrace_at=:0: when logging hits line file:N, emit a stack trace
--log_dir='': If non-empty, write log files in this directory
--logtostderr=false: log to standard error instead of files
-p, --profile='minikube': The name of the minikube VM being used. This can be set to allow having multiple instances of minikube independently.
--stderrthreshold=2: logs at or above this threshold go to stderr
-v, --v=0: log level for V logs
--vmodule=: comma-separated list of pattern=N settings for file-filtered logging
[anliven@anliven ~]$
Minikube 集群
minikube version # 查看版本
minikube status # 查看状态
minikube profile list # 查看属性
minikube addons list # 查看当前支持的插件
minikube service list # 查看服务列表
minikube node list # 参看添加的node
minikube ssh # 登录
minikube ip # 获取IP地址
minikube start # 启动
minikube stop # 停止
minikube delete # 删除
minikube dashboard # 在默认浏览器中启动仪表盘
rm-rf ~/.minikube # 重置(清理所有缓存的镜像，重头开始)
[anliven@anliven ~]$ minikube status # 查看集群状态
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube profile list # 查看集群属性
|----------|-----------|---------|--------------|------|---------|---------|
| Profile | VM Driver | Runtime | IP | Port | Version | Status |
|----------|-----------|---------|--------------|------|---------|---------|
| minikube | docker | docker | 192.168.49.2 | 8443 | v1.19.2 | Running |
|----------|-----------|---------|--------------|------|---------|---------|
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube addons list # 查看集群当前支持的插件
|-----------------------------|----------|--------------|
| ADDON NAME | PROFILE | STATUS |
|-----------------------------|----------|--------------|
| ambassador | minikube | disabled |
| csi-hostpath-driver | minikube | disabled |
| dashboard | minikube | enabled ✅ |
| default-storageclass | minikube | enabled ✅ |
| efk | minikube | disabled |
| freshpod | minikube | disabled |
| gcp-auth | minikube | disabled |
| gvisor | minikube | disabled |
| helm-tiller | minikube | disabled |
| ingress | minikube | disabled |
| ingress-dns | minikube | disabled |
| istio | minikube | disabled |
| istio-provisioner | minikube | disabled |
| kubevirt | minikube | disabled |
| logviewer | minikube | disabled |
| metallb | minikube | disabled |
| metrics-server | minikube | disabled |
| nvidia-driver-installer | minikube | disabled |
| nvidia-gpu-device-plugin | minikube | disabled |
| olm | minikube | disabled |
| pod-security-policy | minikube | disabled |
| registry | minikube | disabled |
| registry-aliases | minikube | disabled |
| registry-creds | minikube | disabled |
| storage-provisioner | minikube | enabled ✅ |
| storage-provisioner-gluster | minikube | disabled |
| volumesnapshots | minikube | disabled |
|-----------------------------|----------|--------------|
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube service list # 查看集群服务列表
|----------------------|---------------------------|--------------|-----|
| NAMESPACE | NAME | TARGET PORT | URL |
|----------------------|---------------------------|--------------|-----|
| default | kubernetes | No node port |
| kube-system | kube-dns | No node port |
| kubernetes-dashboard | dashboard-metrics-scraper | No node port |
| kubernetes-dashboard | kubernetes-dashboard | No node port |
|----------------------|---------------------------|--------------|-----|
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube node list # node列表
minikube 192.168.49.2
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube ip # ip地址
192.168.49.2
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube ssh # 登录
docker@minikube:~$ whoami
docker
docker@minikube:~$ exit
logout
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube stop # 停止
* Stopping node "minikube" ...
* Powering off "minikube" via SSH ...
* 1 nodes stopped.
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube status # 查看状态
minikube
type: Control Plane
host: Stopped
kubelet: Stopped
apiserver: Stopped
kubeconfig: Stopped
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube start # 重新启动，不会“Pulling base image ...”
* minikube v1.14.0 on Centos 7.8.2003 (vbox/amd64)
* Using the docker driver based on existing profile
* Starting control plane node minikube in cluster minikube
* Restarting existing docker container for "minikube" ...
* Preparing Kubernetes v1.19.2 on Docker 19.03.8 ...
* Verifying Kubernetes components...
* Enabled addons: default-storageclass, storage-provisioner, dashboard
* Done! kubectl is now configured to use "minikube" by default
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube status # 查看状态
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
[anliven@anliven ~]$
启动Dashboard
minikube dashboard # 在默认浏览器中启动仪表盘
9 - 问题处理
1 - minikube启动失败
[root@anliven K8s]# minikube start
* minikube v1.14.0 on Centos 7.8.2003 (vbox/amd64)
* Automatically selected the docker driver
* The "docker" driver should not be used with root privileges.
* If you are running minikube within a VM, consider using --driver=none:
* https://minikube.sigs.k8s.io/docs/reference/drivers/none/
X Exiting due to DRV_AS_ROOT: The "docker" driver should not be used with root privileges.
处理方法： 使用非root用户，并将此用户添加到docker组： sudo usermod -aG docker $USER && newgrp docker
2 - dashboard启动失败
[anliven@anliven ~]$ minikube dashboard
* Enabling dashboard ...
* Verifying dashboard health ...
* Launching proxy ...
* Verifying proxy health ...
* Opening http://127.0.0.1:37006/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ in your default browser...
START /usr/bin/firefox "http://127.0.0.1:37006/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/"
Failed to open connection to "session" message bus: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
Running without a11y support!
Error: no DISPLAY environment variable specified
xdg-open: no method available for opening 'http://127.0.0.1:37006/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/'
X Exiting due to HOST_BROWSER: failed to open browser: exit status 3
[anliven@anliven ~]$
处理方法： 需要图形界面下的命令行中执行minikube dashboard，以便启动浏览器
10 - 参考信息
https://kubernetes.io/docs/tasks/tools/install-kubectl/
https://github.com/AliyunContainerService/minikube/wiki
https://github.com/AliyunContainerService/minikube/releases
