Gestion des données Multi-Cloud

---

1 - Data Gravity

1.1 Le concept

Data Gravity : Les données massives attirent les workloads. Il est plus économique et performant de déplacer le compute vers les données que l'inverse.

Facteur	Impact
Egress costs	$0.08-0.12/GB entre clouds
Latence	10-100ms cross-cloud
Compliance	GDPR exige localisation
Bande passante	Limitée entre clouds

1.2 Stratégies

Stratégie	Description	Use Case
Colocation	Apps près des données	Analytics
Caching	Copie locale des données	Read-heavy
Réplication	Sync bidirectionnelle	Active-Active
Event streaming	Sync asynchrone	Real-time

---

2 - Réplication de bases de données

2.1 PostgreSQL Cross-Cloud

-- Sur le primary (AWS RDS)
-- Créer une publication
CREATE PUBLICATION my_publication FOR ALL TABLES;

-- Sur le replica (Azure)
-- Créer une souscription
CREATE SUBSCRIPTION my_subscription
CONNECTION 'host=rds-primary.aws.com dbname=mydb user=repl password=xxx'
PUBLICATION my_publication;

2.2 Configuration Terraform

# AWS RDS Primary
resource "aws_db_instance" "primary" {
  identifier        = "primary-db"
  engine            = "postgres"
  engine_version    = "15"
  instance_class    = "db.r6g.large"
  allocated_storage = 100
  
  # Activer la réplication logique
  parameter_group_name = aws_db_parameter_group.logical_replication.name
  
  publicly_accessible = true  # Pour cross-cloud (avec VPN recommandé)
}

resource "aws_db_parameter_group" "logical_replication" {
  family = "postgres15"
  name   = "logical-replication"
  
  parameter {
    name  = "rds.logical_replication"
    value = "1"
  }
}

2.3 MySQL avec Debezium

# Debezium CDC pour MySQL
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: mysql-source
spec:
  class: io.debezium.connector.mysql.MySqlConnector
  tasksMax: 1
  config:
    database.hostname: mysql.aws.example.com
    database.port: 3306
    database.user: debezium
    database.password: ${MYSQL_PASSWORD}
    database.server.id: 184054
    database.server.name: aws-mysql
    database.include.list: mydb
    schema.history.internal.kafka.bootstrap.servers: kafka:9092
    schema.history.internal.kafka.topic: schema-changes.mydb

---

3 - Object Storage Cross-Cloud

3.1 Architecture

3.2 Réplication avec rclone

# Configuration rclone
# ~/.config/rclone/rclone.conf
[aws]
type = s3
provider = AWS
access_key_id = xxx
secret_access_key = xxx
region = eu-west-1

[azure]
type = azureblob
account = myaccount
key = xxx

[gcp]
type = google cloud storage
project_number = 123456789
service_account_file = /path/to/sa.json

# Sync bidirectionnel
rclone sync aws:my-bucket azure:my-container --progress

# Sync en continu
rclone sync aws:my-bucket gcp:my-bucket --checksum --progress

3.3 MinIO Gateway Multi-Cloud

# docker-compose.yml
version: '3'
services:
  minio:
    image: minio/minio
    command: gateway s3 https://s3.amazonaws.com
    environment:
      MINIO_ROOT_USER: admin
      MINIO_ROOT_PASSWORD: password
    ports:
      - "9000:9000"

---

4 - Event Streaming

4.1 Kafka Cross-Cloud

4.2 Kafka MirrorMaker 2

# mm2.properties
clusters = source, target

source.bootstrap.servers = msk.aws.example.com:9092
target.bootstrap.servers = kafka.azure.example.com:9092

source->target.enabled = true
source->target.topics = .*

replication.factor = 3
checkpoints.topic.replication.factor = 3
heartbeats.topic.replication.factor = 3

offset-syncs.topic.replication.factor = 3

sync.topic.acls.enabled = false

4.3 Confluent Cluster Linking

# Créer un lien entre clusters
confluent kafka link create azure-link \
    --cluster source-cluster-id \
    --destination-cluster destination-cluster-id \
    --destination-bootstrap-server kafka.azure.example.com:9092

# Créer un mirror topic
confluent kafka mirror create my-topic \
    --link azure-link \
    --cluster destination-cluster-id

---

5 - Backup Cross-Cloud

5.1 Stratégie 3-2-1

3 copies - 2 types de stockage - 1 offsite (autre cloud)

5.2 Velero pour Kubernetes

# AWS Backup Location
velero backup-location create aws-backups \
    --provider aws \
    --bucket velero-backups-aws \
    --config region=eu-west-1

# Azure Backup Location
velero backup-location create azure-backups \
    --provider azure \
    --bucket velero-backups-azure \
    --config storageAccount=myaccount

# Backup vers les deux
velero backup create daily-backup \
    --storage-location aws-backups \
    --snapshot-volumes

# Copier vers Azure
velero backup copy daily-backup \
    --destination-storage-location azure-backups

5.3 Database Backup Script

#!/bin/bash
# backup-db.sh

DATE=$(date +%Y%m%d)
DB_NAME="production"

# Dump PostgreSQL
pg_dump -h rds.aws.example.com -U admin $DB_NAME | gzip > /tmp/$DB_NAME-$DATE.sql.gz

# Upload vers AWS
aws s3 cp /tmp/$DB_NAME-$DATE.sql.gz s3://backups-aws/db/

# Upload vers Azure
az storage blob upload \
    --account-name backups \
    --container db \
    --file /tmp/$DB_NAME-$DATE.sql.gz \
    --name $DB_NAME-$DATE.sql.gz

# Upload vers GCP
gsutil cp /tmp/$DB_NAME-$DATE.sql.gz gs://backups-gcp/db/

# Cleanup
rm /tmp/$DB_NAME-$DATE.sql.gz

---

6 - Data Compliance

6.1 GDPR et localisation

Exigence	Solution
Données en EU	Régions EU uniquement
Portabilité	Export standard formats
Effacement	Procédures multi-cloud
Audit trail	Logs centralisés

6.2 Encryption at rest

# AWS S3
resource "aws_s3_bucket_server_side_encryption_configuration" "main" {
  bucket = aws_s3_bucket.main.id
  
  rule {
    apply_server_side_encryption_by_default {
      kms_master_key_id = aws_kms_key.main.arn
      sse_algorithm     = "aws:kms"
    }
  }
}

# Azure Blob
resource "azurerm_storage_account" "main" {
  # ...
  
  blob_properties {
    versioning_enabled = true
  }
  
  identity {
    type = "SystemAssigned"
  }
}

# GCP Storage
resource "google_storage_bucket" "main" {
  name     = "my-bucket"
  location = "EU"
  
  encryption {
    default_kms_key_name = google_kms_crypto_key.main.id
  }
}

---

Résumé

Dans ce chapitre, nous avons appris :

• Le concept de Data Gravity
• La réplication de bases de données
• Le stockage objet cross-cloud
• L'event streaming multi-cloud
• Les stratégies de backup
• La compliance des données

---

Prochaine étape

Dans le prochain chapitre, nous verrons le Monitoring Multi-Cloud.

→ Chapitre suivant : Monitoring Multi-Cloud

---

← Retour à la table des matières