Unable to read the configuration of the container created by dockershim.sock #991
Comments
yangkaa
added
kind/bug
|
The container inspect will just pass on whatever the runtime is providing, and docker doesn't have that info. Any new fields needs to be added to the CRI implementation for the runtime, in this case it is Similar to this modification for the But for ancient versions like 1.19, I don't think there will be any modifications. If you are feeling adventurous you can can try using docker-containerd to get it ?
You will need to use the full sha256 Id. |
type ContainerStatusRequest struct {
// ID of the container for which to retrieve status.
ContainerId string
// Verbose indicates whether to return extra information about the container.
Verbose bool
}
type ContainerStatusResponse struct {
// Status of the container.
Status *ContainerStatus
// Info is extra information of the Container. The key could be arbitrary string, and
// value should be in json format. The information could include anything useful for
// debug, e.g. pid for linux container based container runtime.
// It should only be returned non-empty when Verbose is true.
Info map[string]string
}Infocontainerd (v1.6.8) struct {
SandboxID string `json:"sandboxID"`
Pid uint32 `json:"pid"`
Removing bool `json:"removing"`
SnapshotKey string `json:"snapshotKey"`
Snapshotter string `json:"snapshotter"`
RuntimeType string `json:"runtimeType"`
RuntimeOptions interface{} `json:"runtimeOptions"`
Config *runtime.ContainerConfig `json:"config"`
RuntimeSpec *runtimespec.Spec `json:"runtimeSpec"`
}
...
return &runtime.ContainerStatusResponse{
Status: status,
Info: info,
}, nilcri-o (v1.25.0) struct {
SandboxID string `json:"sandboxID"`
Pid int `json:"pid"`
RuntimeSpec spec.Spec `json:"runtimeSpec"`
Privileged bool `json:"privileged"`
}
...
resp := &types.ContainerStatusResponse{
Status: &types.ContainerStatus{
...
resp.Info = info
...
return resp, nilcri-dockerd (v0.2.5) // Currently missing, but could export Pid and Config as Info in the Verbose response.
// Not sure if SandboxID and RuntimeSpec are exposed, in the ContainerJSON type ?
...
return &v1.ContainerStatusResponse{Status: status}, nilhttps://pkg.go.dev/github.com/docker/docker@v20.10.18+incompatible/api/types#ContainerJSON The Config is a different type though, so it would have to be converted/created to a CRI Config |
|
The SandboxID was available in the docker labels, as "io.kubernetes.sandbox.id" struct {
SandboxID string `json:"sandboxID"`
Pid int `json:"pid"`
} |
|
Thank you very much for your answer. It seems that when I run using containerd as a container, the container it creates should be able to get |
They should all be able to get some kind of "info", but the contents varies between the three runtimes... https://pkg.go.dev/k8s.io/cri-api@v0.25.2/pkg/apis/runtime/v1#ContainerStatusResponse "The information could include anything useful for debug, e.g. pid for linux container based container runtime." With the patch above (cri-dockerd v0.2.6), then at least the docker@minikube:~$ sudo crictl ps | grep etcd
a69a1178d6e9b a8a176a5d5d69 24 hours ago Running etcd 2 10315d75fb4af
docker@minikube:~$ sudo crictl inspect a69a1178d6e9b | tail
"selinuxRelabel": false
}
],
"logPath": "/var/log/pods/kube-system_etcd-minikube_bd495b7643dfc9d3194bd002e968bc3d/etcd/2.log"
},
"info": {
"sandboxID": "10315d75fb4afb8f30e4ca05c59b565dbbd8949aab9090bd91814eefe67a5139",
"pid": 41594
}
}
docker@minikube:~$ docker ps | grep etcd
a69a1178d6e9 a8a176a5d5d6 "etcd --advertise-cl…" 24 hours ago Up 24 hours k8s_etcd_etcd-minikube_kube-system_bd495b7643dfc9d3194bd002e968bc3d_2
10315d75fb4a k8s.gcr.io/pause:3.6 "/pause" 24 hours ago Up 24 hours k8s_POD_etcd-minikube_kube-system_bd495b7643dfc9d3194bd002e968bc3d_2
docker@minikube:~$ docker images | grep a8a176a5d5d6
registry.k8s.io/etcd 3.5.4-0 a8a176a5d5d6 3 months ago 300MB |
The normal response when talking with crictl to the containerd endpoint of docker, is just a: This is because the cri plugin is disabled by default, when running as a subsystem of docker... disabled_plugins = ["cri"]So it would need to use the Some versions also use When Kubernetes uses containerd directly, the CRI plugin is enabled (in config). Eventually Docker might have a CRI plugin, but for now Compare the evolution of the CRI support for containerd, from https://kubernetes.io/blog/2018/05/24/kubernetes-containerd-integration-goes-ga/ The main difference with k8s 1.24 was that internal "dockershim" moved to external "cri-dockerd". In the process, it also made |
|
@yangkaa what do you expect to be available, in the "info" field ? i.e. when comparing The RuntimeSpec seems to be suggested, not sure if From what I can tell, it is just created and passed on - but not retained. |
|
I just want to get the environment variables, and I need to do some log collection through the environment variables. Because the target to send the log is defined in the environment variable. After my attempt, in v1.24.4+k3s1, when using containerd as a container, So I couldn't get the environment variable before, as you said, this information is not in docker. |
|
@afbjorklund thanks |
|
The environment is readily available in the docker Config too, so maybe this should just be dumped as-is ? But I'm confused, since you say both 1.19 docker and 1.24 containerd at once (two different runtimes) |
|
They are two different clusters. At first, I wanted to collect the container runtime logs in a uniform way. The environment variable is a required parameter. In version 1.19 of the cluster, I used the API provided by docker to get the environment variables of the container. Later, I want to use CRI to get environment variables directly, and it is also compatible with existing lower-version clusters (1.19). But it doesn't seem to work in version 1.19. With your help, I changed my mind. The old version still uses docker. The newly deployed environment fully uses CRI, which works fine. |
The CRI for docker (or cri-o) doesn't supply it (Env), but it could be added... So it only works for containerd, and might as well have used EDIT: I think the RuntimeSpec does include the "env", so that would probably have been the most straight-forward implementation, if available. |
What happened:
I have a version 1.19 K8s cluster whose default running endpoint is dockershim.sock. When I use the crictl command to view container details, I can't get container args, environment variables, and so on. I did the following:
{ "status": { "id": "6e899a3980f8bcf58aa4585bb8098196522f93e15b3d259a06b5319219e3b3f2", "metadata": { "attempt": 1, "name": "kube-flannel" }, "state": "CONTAINER_RUNNING", "createdAt": "2022-09-22T22:15:43.524942178+08:00", "startedAt": "2022-09-22T22:15:43.705500106+08:00", "finishedAt": "0001-01-01T00:00:00Z", "exitCode": 0, "image": { "annotations": {}, "image": "registry.cn-hangzhou.aliyuncs.com/demo/coreos-flannel:v0.13.0-rke" }, "imageRef": "docker-pullable://registry.cn-hangzhou.aliyuncs.com/demo/coreos-flannel@sha256:eccf2e521bc631c77805ef1ad27c24de5524f3c3f5756c52ca1c2dd5baf5ec09", "reason": "", "message": "", "labels": { "io.kubernetes.container.name": "kube-flannel" }, "annotations": { "io.kubernetes.container.hash": "d19c9579", "io.kubernetes.container.restartCount": "1" }, "mounts": [], "logPath": "/var/log/pods/kube-system_kube-flannel-ptrjt_80ea7083-2440-4f6f-bee7-a96f0e7b8db6/kube-flannel/1.log" } }What you expected to happen:
I want to be able to show the details of the container, as shown below. This is the container information I created manually through crictl, using containerd.sock , I want to get the information in the
infofield.{ "status": { "id": "9bcccae456330f9182de50a0a4769889a97d1a44d1b7ad6827331074cb26c62b", "metadata": { "attempt": 0, "name": "busybox" }, "state": "CONTAINER_RUNNING", "createdAt": "2022-09-22T23:13:35.987487555+08:00", "startedAt": "2022-09-22T23:13:36.060890611+08:00", "finishedAt": "0001-01-01T00:00:00Z", "exitCode": 0, "image": { "annotations": {}, "image": "docker.io/library/nginx:latest" }, "imageRef": "docker.io/library/nginx@sha256:0b970013351304af46f322da1263516b188318682b2ab1091862497591189ff1", "logPath": "nginx.log" }, "info": { "sandboxID": "635ea5f719992f0dda8665bcf71b5e102fde074f68748b698ac31d5da97ac169", "pid": 9342, "removing": false, "snapshotKey": "9bcccae456330f9182de50a0a4769889a97d1a44d1b7ad6827331074cb26c62b", "snapshotter": "overlayfs", "runtimeType": "io.containerd.runtime.v1.linux", "runtimeOptions": {}, "config": { "metadata": { "name": "busybox" }, "image": { "image": "nginx:latest" }, "envs": [ { "key": "WORDPRESS_DB_HOST", "value": "127.0.0.1" } ], "log_path": "nginx.log" }, "runtimeSpec": { "ociVersion": "1.0.1-dev", "mounts": [], "annotations": { "io.kubernetes.cri.container-type": "container", "io.kubernetes.cri.sandbox-id": "635ea5f719992f0dda8665bcf71b5e102fde074f68748b698ac31d5da97ac169" }, "linux": { "cgroupsPath": "/k8s.io/9bcccae456330f9182de50a0a4769889a97d1a44d1b7ad6827331074cb26c62b", "namespaces": [] } } } }How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
cat /etc/os-release): Centos 7uname -a): 3.10.0-862.14.4.el7.x86_64The text was updated successfully, but these errors were encountered: