Monitoring et Logging

---

1 - CloudWatch Logs

1.1 Configuration awslogs

{
  "containerDefinitions": [
    {
      "name": "app",
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/mon-app",
          "awslogs-region": "eu-west-1",
          "awslogs-stream-prefix": "app",
          "awslogs-create-group": "true"
        }
      }
    }
  ]
}

1.2 Log Group CloudFormation

LogGroup:
  Type: AWS::Logs::LogGroup
  Properties:
    LogGroupName: /ecs/mon-app
    RetentionInDays: 30
    Tags:
      - Key: Application
        Value: MonApp

1.3 Mode non-blocking

{
  "logConfiguration": {
    "logDriver": "awslogs",
    "options": {
      "awslogs-group": "/ecs/app",
      "awslogs-region": "eu-west-1",
      "awslogs-stream-prefix": "app",
      "mode": "non-blocking",
      "max-buffer-size": "25m"
    }
  }
}

---

2 - Container Insights

2.1 Activer Container Insights

ECSCluster:
  Type: AWS::ECS::Cluster
  Properties:
    ClusterName: prod-cluster
    ClusterSettings:
      - Name: containerInsights
        Value: enabled

# Via CLI
aws ecs update-cluster-settings \
    --cluster prod-cluster \
    --settings name=containerInsights,value=enabled

2.2 Métriques disponibles

Métrique	Description
CpuUtilized	CPU utilisé
CpuReserved	CPU réservé
MemoryUtilized	Mémoire utilisée
MemoryReserved	Mémoire réservée
NetworkRxBytes	Bytes reçus
NetworkTxBytes	Bytes transmis
StorageReadBytes	Lecture disque
StorageWriteBytes	Écriture disque

2.3 Dashboard automatique

Container Insights crée automatiquement un dashboard avec :

• Vue cluster
• Vue service
• Vue task
• Métriques de performance

---

3 - CloudWatch Alarms

3.1 Alarme CPU

HighCPUAlarm:
  Type: AWS::CloudWatch::Alarm
  Properties:
    AlarmName: ECS-HighCPU
    AlarmDescription: CPU above 80% for 5 minutes
    MetricName: CPUUtilization
    Namespace: AWS/ECS
    Statistic: Average
    Period: 60
    EvaluationPeriods: 5
    Threshold: 80
    ComparisonOperator: GreaterThanThreshold
    Dimensions:
      - Name: ClusterName
        Value: !Ref ECSCluster
      - Name: ServiceName
        Value: !GetAtt ECSService.Name
    AlarmActions:
      - !Ref SNSTopic

3.2 Alarme mémoire

HighMemoryAlarm:
  Type: AWS::CloudWatch::Alarm
  Properties:
    AlarmName: ECS-HighMemory
    MetricName: MemoryUtilization
    Namespace: AWS/ECS
    Statistic: Average
    Period: 60
    EvaluationPeriods: 5
    Threshold: 85
    ComparisonOperator: GreaterThanThreshold
    Dimensions:
      - Name: ClusterName
        Value: !Ref ECSCluster
      - Name: ServiceName
        Value: !GetAtt ECSService.Name
    AlarmActions:
      - !Ref SNSTopic

3.3 Alarme tasks unhealthy

UnhealthyTasksAlarm:
  Type: AWS::CloudWatch::Alarm
  Properties:
    AlarmName: ECS-UnhealthyTasks
    MetricName: RunningTaskCount
    Namespace: ECS/ContainerInsights
    Statistic: Average
    Period: 60
    EvaluationPeriods: 3
    Threshold: !Ref MinHealthyTasks
    ComparisonOperator: LessThanThreshold
    Dimensions:
      - Name: ClusterName
        Value: !Ref ECSCluster
      - Name: ServiceName
        Value: !GetAtt ECSService.Name
    AlarmActions:
      - !Ref SNSTopic

---

4 - Logs Insights

4.1 Requêtes utiles

-- Erreurs par heure
fields @timestamp, @message
| filter @message like /ERROR/
| stats count() by bin(1h)

-- Latence des requêtes
fields @timestamp, @message
| parse @message /latency=(?<latency>\d+)/
| stats avg(latency), max(latency), percentile(latency, 99) by bin(5m)

-- Top erreurs
fields @timestamp, @message
| filter @message like /ERROR/
| parse @message /ERROR: (?<error>.+)/
| stats count() by error
| sort count desc
| limit 10

-- Containers restartés
fields @timestamp, @message
| filter @message like /Task stopped/
| stats count() by bin(1h)

4.2 Saved queries

# Via CLI
aws logs put-query-definition \
    --name "ECS-Errors" \
    --query-string 'fields @timestamp, @message | filter @message like /ERROR/ | sort @timestamp desc | limit 100' \
    --log-group-names "/ecs/mon-app"

---

5 - X-Ray Tracing

5.1 Sidecar X-Ray

{
  "containerDefinitions": [
    {
      "name": "app",
      "image": "my-app:latest",
      "environment": [
        {"name": "AWS_XRAY_DAEMON_ADDRESS", "value": "localhost:2000"}
      ]
    },
    {
      "name": "xray-daemon",
      "image": "amazon/aws-xray-daemon:latest",
      "essential": false,
      "cpu": 32,
      "memory": 256,
      "portMappings": [
        {"containerPort": 2000, "protocol": "udp"}
      ]
    }
  ]
}

5.2 Task Role pour X-Ray

{
  "Effect": "Allow",
  "Action": [
    "xray:PutTraceSegments",
    "xray:PutTelemetryRecords",
    "xray:GetSamplingRules",
    "xray:GetSamplingTargets",
    "xray:GetSamplingStatisticSummaries"
  ],
  "Resource": "*"
}

---

6 - FireLens

6.1 Configuration Fluent Bit

{
  "containerDefinitions": [
    {
      "name": "log_router",
      "image": "public.ecr.aws/aws-observability/aws-for-fluent-bit:stable",
      "essential": true,
      "firelensConfiguration": {
        "type": "fluentbit",
        "options": {
          "enable-ecs-log-metadata": "true"
        }
      },
      "memoryReservation": 50
    },
    {
      "name": "app",
      "image": "my-app:latest",
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch_logs",
          "region": "eu-west-1",
          "log_group_name": "/ecs/app",
          "auto_create_group": "true",
          "log_stream_prefix": "app-"
        }
      }
    }
  ]
}

6.2 Multi-destination

{
  "logConfiguration": {
    "logDriver": "awsfirelens",
    "options": {
      "Name": "cloudwatch_logs",
      "region": "eu-west-1",
      "log_group_name": "/ecs/app"
    },
    "secretOptions": []
  }
}

---

7 - Métriques personnalisées

7.1 Publier depuis le conteneur

import boto3

cloudwatch = boto3.client('cloudwatch')

def publish_metric(name, value, unit='Count'):
    cloudwatch.put_metric_data(
        Namespace='MyApp',
        MetricData=[
            {
                'MetricName': name,
                'Value': value,
                'Unit': unit,
                'Dimensions': [
                    {'Name': 'Environment', 'Value': 'Production'},
                    {'Name': 'Service', 'Value': 'API'}
                ]
            }
        ]
    )

# Exemple
publish_metric('OrdersProcessed', 150)
publish_metric('ProcessingTime', 250, 'Milliseconds')

7.2 Embedded Metric Format

import json

# EMF log line
print(json.dumps({
    "_aws": {
        "Timestamp": 1234567890000,
        "CloudWatchMetrics": [{
            "Namespace": "MyApp",
            "Dimensions": [["Service"]],
            "Metrics": [
                {"Name": "RequestCount", "Unit": "Count"},
                {"Name": "Latency", "Unit": "Milliseconds"}
            ]
        }]
    },
    "Service": "API",
    "RequestCount": 150,
    "Latency": 45
}))

---

8 - Dashboard complet

ECSMonitoringDashboard:
  Type: AWS::CloudWatch::Dashboard
  Properties:
    DashboardName: ECS-Monitoring
    DashboardBody: !Sub |
      {
        "widgets": [
          {
            "type": "metric",
            "x": 0, "y": 0, "width": 12, "height": 6,
            "properties": {
              "title": "CPU & Memory",
              "metrics": [
                ["AWS/ECS", "CPUUtilization", "ServiceName", "${ECSService.Name}", "ClusterName", "${ECSCluster}"],
                [".", "MemoryUtilization", ".", ".", ".", "."]
              ],
              "period": 60,
              "stat": "Average"
            }
          },
          {
            "type": "metric",
            "x": 12, "y": 0, "width": 12, "height": 6,
            "properties": {
              "title": "Task Count",
              "metrics": [
                ["ECS/ContainerInsights", "DesiredTaskCount", "ServiceName", "${ECSService.Name}", "ClusterName", "${ECSCluster}"],
                [".", "RunningTaskCount", ".", ".", ".", "."],
                [".", "PendingTaskCount", ".", ".", ".", "."]
              ]
            }
          },
          {
            "type": "log",
            "x": 0, "y": 6, "width": 24, "height": 6,
            "properties": {
              "title": "Application Logs",
              "query": "SOURCE '/ecs/${ECSService.Name}' | fields @timestamp, @message | sort @timestamp desc | limit 100"
            }
          }
        ]
      }

---

Résumé

Dans ce chapitre, nous avons appris :

• La configuration de CloudWatch Logs
• L'activation de Container Insights
• La création d'alarmes CloudWatch
• Les requêtes Logs Insights
• Le tracing avec X-Ray
• FireLens pour les logs avancés
• Les métriques personnalisées

---

Prochaine étape

Dans le prochain chapitre, nous verrons les Bonnes pratiques.

→ Chapitre suivant : Bonnes pratiques

---

← Retour à la table des matières