Aller au contenu principal

Prometheus

1 - Architecture​

2 - Installation​

2.1 Docker​

# docker-compose.yml
version: '3.8'
services:
  prometheus:
    image: prom/prometheus:v2.47.0
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--storage.tsdb.retention.time=15d'
      
volumes:
  prometheus_data:

2.2 Kubernetes (Helm)​

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus prometheus-community/kube-prometheus-stack \
    --namespace monitoring \
    --create-namespace

2.3 Configuration de base​

# prometheus.yml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

alerting:
  alertmanagers:
    - static_configs:
        - targets: ['alertmanager:9093']

rule_files:
  - /etc/prometheus/rules/*.yml

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']
  
  - job_name: 'node-exporter'
    static_configs:
      - targets: ['node-exporter:9100']
  
  - job_name: 'applications'
    static_configs:
      - targets:
          - 'app1:8080'
          - 'app2:8080'

3 - Types de métriques​

3.1 Counter​

# Valeur qui ne fait qu'augmenter
http_requests_total{method="GET", status="200"} 1234

# Rate: requêtes par seconde
rate(http_requests_total[5m])

# Increase: augmentation sur une période
increase(http_requests_total[1h])

3.2 Gauge​

# Valeur qui peut monter/descendre
node_memory_MemAvailable_bytes 4294967296

# Valeur actuelle
node_cpu_seconds_total

# Moyenne
avg_over_time(node_memory_MemAvailable_bytes[1h])

3.3 Histogram​

# Distribution des valeurs
http_request_duration_seconds_bucket{le="0.1"} 1000
http_request_duration_seconds_bucket{le="0.5"} 1500
http_request_duration_seconds_bucket{le="1.0"} 1600
http_request_duration_seconds_bucket{le="+Inf"} 1650
http_request_duration_seconds_sum 850
http_request_duration_seconds_count 1650

# Percentile 99
histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m]))

3.4 Summary​

# Percentiles pré-calculés
http_request_duration_seconds{quantile="0.5"} 0.05
http_request_duration_seconds{quantile="0.9"} 0.1
http_request_duration_seconds{quantile="0.99"} 0.2

4 - PromQL​

4.1 Sélecteurs​

# Métrique simple
http_requests_total

# Avec labels
http_requests_total{method="GET"}
http_requests_total{status=~"2.."}  # Regex
http_requests_total{status!="500"}  # Différent

# Combinaisons
http_requests_total{method="POST", status=~"5.."}

4.2 Fonctions de taux​

# Rate: taux moyen par seconde
rate(http_requests_total[5m])

# Irate: taux instantané (2 derniers points)
irate(http_requests_total[5m])

# Increase: augmentation totale
increase(http_requests_total[1h])

4.3 Agrégations​

# Sum par label
sum(rate(http_requests_total[5m])) by (service)

# Average
avg(rate(http_requests_total[5m])) by (instance)

# Count
count(up == 1)

# Top K
topk(5, rate(http_requests_total[5m]))

4.4 Opérations arithmétiques​

# Taux d'erreurs
sum(rate(http_requests_total{status=~"5.."}[5m])) 
/ 
sum(rate(http_requests_total[5m])) * 100

# Mémoire utilisée en %
(node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) 
/ node_memory_MemTotal_bytes * 100

# CPU usage
100 - (avg by(instance) (irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

4.5 Fonctions utiles​

# Percentile
histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m]))

# Absence de données
absent(up{job="myapp"})

# Clamp (borner les valeurs)
clamp_max(rate(http_requests_total[5m]), 100)

# Delta (différence)
delta(temperature_celsius[1h])

# Prédiction
predict_linear(node_filesystem_free_bytes[1h], 3600*24)

5 - Service Discovery​

5.1 Kubernetes​

scrape_configs:
  - job_name: 'kubernetes-pods'
    kubernetes_sd_configs:
      - role: pod
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)
      - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
        action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        target_label: __address__

5.2 Consul​

scrape_configs:
  - job_name: 'consul-services'
    consul_sd_configs:
      - server: 'consul:8500'
        services: []
    relabel_configs:
      - source_labels: [__meta_consul_service]
        target_label: service

5.3 AWS EC2​

scrape_configs:
  - job_name: 'ec2-instances'
    ec2_sd_configs:
      - region: eu-west-1
        port: 9100
    relabel_configs:
      - source_labels: [__meta_ec2_tag_Name]
        target_label: instance_name

6 - Exporters​

6.1 Node Exporter​

# Installation
docker run -d \
    --name node-exporter \
    --net host \
    --pid host \
    -v /:/host:ro \
    prom/node-exporter:latest \
    --path.rootfs=/host
# Métriques clés
node_cpu_seconds_total
node_memory_MemTotal_bytes
node_filesystem_size_bytes
node_network_receive_bytes_total

6.2 Blackbox Exporter​

# Probe HTTP
modules:
  http_2xx:
    prober: http
    timeout: 5s
    http:
      valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
      valid_status_codes: [200]
      method: GET

scrape_configs:
  - job_name: 'blackbox'
    metrics_path: /probe
    params:
      module: [http_2xx]
    static_configs:
      - targets:
          - https://example.com
          - https://api.example.com
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - target_label: __address__
        replacement: blackbox-exporter:9115

7 - Recording Rules​

# rules/recording.yml
groups:
  - name: http_rules
    interval: 30s
    rules:
      - record: job:http_requests_total:rate5m
        expr: sum(rate(http_requests_total[5m])) by (job)
      
      - record: job:http_request_duration_seconds:p99
        expr: histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket[5m])) by (job, le))
      
      - record: job:http_errors_ratio:rate5m
        expr: |
          sum(rate(http_requests_total{status=~"5.."}[5m])) by (job)
          /
          sum(rate(http_requests_total[5m])) by (job)

8 - High Availability​

8.1 Thanos​

8.2 Configuration​

# Thanos sidecar
args:
  - sidecar
  - --prometheus.url=http://localhost:9090
  - --objstore.config-file=/etc/thanos/bucket.yaml

Résumé​

Dans ce chapitre, nous avons appris :

  • L'architecture de Prometheus
  • Les types de métriques
  • PromQL en détail
  • Le Service Discovery
  • Les Exporters
  • Les Recording Rules
  • La High Availability avec Thanos

Prochaine étape​

Dans le prochain chapitre, nous verrons Grafana.

→ Chapitre suivant : Grafana

← Retour à la table des matières