Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

I've setup Prometheus to monitor Kubernetes. However when i watch the Prometheus dashboard I see kubernetes-cadvisor DOWN

I would want to know if we need it to monitor Kubernetes because on Grafana i already get different information as memory usage, disk space ...

Would it be used to monitor containers in order to make precise requests such as the use of memory used by a pod of a specific namespace ?

What's your version of Kubernetes and can you post your prometheus.yml config for kubernetes-cadvisor ? Oliver Jan 14, 2019 at 16:31

The error you have provided means that the cAdvisor's content does not comply with the Prometheus exposition format. [1] But to be honest, it is one of the possibilities and as you did not provide more information we will have to leave it for now (I mean the information asked by Oliver + versions of Prometheus and Grafana and environment in which you are running the cluster).

Answering your question, although you don't need to use cAdvisor for monitoring, it does provide some important metrics and is pretty well integrated with Kubernetes. So until you need container level metrics, then you should use cAdvisor. As specified in this article (you can find configuration tutorial there):

you can’t access cAdvisor directly (through 4194). You can (!) access cAdvisor by duplicating the job_name (called “k8s”) in the prometheus.yml file, calling the copy “cAdvisor” (perhaps) and inserting an additional line to define “metrics_path”. Prometheus assumes exporters are on “/metrics” but, for cAdvisor, our metrics are on “/metrics/cadvisor”.

I think that could be the reason, but if this does not solve your issue I will try to recreate it in my cluster.

Update:

Judging from your yaml file, you did not configure Prometheus to scrape metrics from the cAdvisor. Add this to your yaml file:

scrape_configs:
- job_name: cadvisor
  scrape_interval: 5s
  static_configs:
  - targets:
    - cadvisor:8080

As specified here.

you will find here my prometheus-core.yaml (deployment) gist.github.com/zyriuse75/917734abf77358dcafab8cfa56668aee – morla Jan 15, 2019 at 10:21 kubernetes version : Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:00:59Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"} – morla Jan 15, 2019 at 10:23

To get the metrics of container we need CADVISOR !!

to setup it i just follow the procedure below

https://github.com/google/cadvisor

i installed it on each of my nodes ! i run on each

sudo docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --volume=/dev/disk/:/dev/disk:ro \
  --publish=8080:8080 \
  --detach=true \
  --name=cadvisor \
  google/cadvisor:latest

i hope this will help you guys ;)

Thanks for contributing an answer to Stack Overflow!

  • Please be sure to answer the question. Provide details and share your research!

But avoid

  • Asking for help, clarification, or responding to other answers.
  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.