Multi-Cluster Setup

Meshery excels at managing infrastructure across multiple Kubernetes clusters and cloud providers. This guide covers setting up multi-cluster management, cross-cluster deployments, and centralized observability.

Why Multi-Cluster?

Multi-cluster architectures provide:

High Availability - Distribute workloads across clusters
Geographic Distribution - Deploy closer to users
Environment Isolation - Separate dev, staging, and production
Cloud Provider Diversity - Avoid vendor lock-in
Compliance - Meet data residency requirements

Prerequisites

Before setting up multi-cluster management:

Meshery Server running and accessible
kubectl access to each cluster
Proper RBAC permissions on each cluster
Network connectivity between Meshery and cluster API servers

Connecting Multiple Clusters

Auto-Discovery from Kubeconfig

Configure Kubeconfig

Ensure your kubeconfig contains all cluster contexts:

# View available contexts
kubectl config get-contexts

# Expected output:
# CURRENT   NAME              CLUSTER           AUTHINFO
# *         prod-cluster-1    prod-cluster-1    prod-admin
#           prod-cluster-2    prod-cluster-2    prod-admin  
#           staging-cluster   staging-cluster   staging-admin
#           dev-cluster       dev-cluster       dev-admin

Create Connections

Create Meshery connections for each context:

# Production clusters
mesheryctl system context create prod-cluster-1
mesheryctl system context create prod-cluster-2

# Staging cluster  
mesheryctl system context create staging-cluster

# Development cluster
mesheryctl system context create dev-cluster

Each command:

Reads credentials from kubeconfig
Registers connection with Meshery
Deploys Meshery Operator to cluster
Starts MeshSync for discovery

Verify Connections

List all registered connections:

mesheryctl connections list

Example output:

ID                                    NAME              KIND        STATUS      
a1b2c3d4-e5f6-7890-abcd-ef1234567890  prod-cluster-1    kubernetes  CONNECTED
b2c3d4e5-f6a7-8901-bcde-fa2345678901  prod-cluster-2    kubernetes  CONNECTED
c3d4e5f6-a7b8-9012-cdef-ab3456789012  staging-cluster   kubernetes  CONNECTED
d4e5f6a7-b8c9-0123-defa-bc4567890123  dev-cluster       kubernetes  REGISTERED

Manual Connection Creation

For clusters not in your kubeconfig:

Gather Cluster Information

You’ll need:

Cluster API server URL
CA certificate
Service account token or client certificate

# Get cluster info
kubectl cluster-info

# Create service account for Meshery
kubectl create serviceaccount meshery-server -n meshery

# Create cluster role binding
kubectl create clusterrolebinding meshery-binding \
  --clusterrole=cluster-admin \
  --serviceaccount=meshery:meshery-server

# Get token (Kubernetes 1.24+)
kubectl create token meshery-server -n meshery --duration=87600h

Create Connection via UI

Navigate to Settings → Connections
Click Create Connection
Select Kubernetes as connection type
Fill in connection details:
- Name: Descriptive name (e.g., “GKE Production”)
- Server URL: https://your-cluster.example.com:6443
- Certificate Authority: Cluster CA cert
- Token: Service account token
Click Save and Connect

Test Connection

Verify the connection is working:

mesheryctl connections view <connection-id>

Connection status should transition: REGISTERED → CONNECTED

Meshery requires cluster-admin or equivalent permissions to deploy the operator and discover resources. For production, consider creating a custom role with minimum required permissions.

Organizing with Environments

Environments group clusters logically and enable environment-specific deployments:

Create Environments

Define environments matching your deployment strategy:Via UI:

Navigate to Environments
Click Create Environment
Configure:
- Name: production, staging, development
- Description: Purpose and scope
- Organization: Your org ID

Assign Connections

Associate connections with environments:

Open environment settings
Navigate to Connections tab
Add connections:
- Production: prod-cluster-1, prod-cluster-2
- Staging: staging-cluster
- Development: dev-cluster

This enables environment-scoped deployments.

Link to Workspaces

Connect environments to team workspaces:

Open Workspace settings
Go to Environments tab
Associate relevant environments

Team members in the workspace can now deploy to assigned environments.

Cross-Cluster Deployments

Deploying to Multiple Clusters

Create Multi-Cluster Design

Design your application in Meshery canvas:

# example: web-app-design.yaml
name: multi-cluster-web-app
services:
  nginx-deployment:
    type: Deployment
    model: kubernetes
    settings:
      spec:
        replicas: 3
        selector:
          matchLabels:
            app: nginx
        template:
          metadata:
            labels:
              app: nginx
          spec:
            containers:
            - name: nginx
              image: nginx:1.21
              ports:
              - containerPort: 80
  
  nginx-service:
    type: Service
    model: kubernetes
    settings:
      spec:
        type: LoadBalancer
        selector:
          app: nginx
        ports:
        - port: 80
          targetPort: 80

Deploy to Production Clusters

Deploy the same design to multiple clusters:

# Deploy to first production cluster
mesheryctl system context switch prod-cluster-1
mesheryctl design apply -f web-app-design.yaml

# Deploy to second production cluster
mesheryctl system context switch prod-cluster-2  
mesheryctl design apply -f web-app-design.yaml

Or deploy via UI:

Open design
Click Deploy
Select multiple environments/connections
Confirm bulk deployment

Verify Deployment

Check deployment status across clusters:

# Cluster 1
kubectl --context=prod-cluster-1 get deployments

# Cluster 2
kubectl --context=prod-cluster-2 get deployments

Environment-Specific Configurations

Customize deployments per environment:

# base-app.yaml with parameters
name: parameterized-app
services:
  app:
    type: Deployment
    settings:
      spec:
        replicas: {{ .Values.replicas }}
        template:
          spec:
            containers:
            - name: app
              image: myapp:{{ .Values.version }}
              env:
              - name: ENVIRONMENT
                value: {{ .Values.environment }}
              - name: LOG_LEVEL
                value: {{ .Values.logLevel }}

Deploy with environment-specific values:

# Production (high replicas, stable version)
mesheryctl design apply -f base-app.yaml \
  --environment production \
  --set replicas=10 \
  --set version=v1.5.0 \
  --set environment=production \
  --set logLevel=warn

# Staging (moderate replicas, latest version)
mesheryctl design apply -f base-app.yaml \
  --environment staging \
  --set replicas=3 \
  --set version=latest \
  --set environment=staging \
  --set logLevel=info

# Development (minimal replicas, debug logging)
mesheryctl design apply -f base-app.yaml \
  --environment development \
  --set replicas=1 \
  --set version=dev \
  --set environment=development \
  --set logLevel=debug

Multi-Cloud Management

Meshery supports clusters across cloud providers:

AWS EKS

# Update kubeconfig for EKS
aws eks update-kubeconfig --region us-west-2 --name prod-cluster

# Create Meshery connection
mesheryctl system context create eks-prod-cluster

Azure AKS

# Get AKS credentials
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster

# Create Meshery connection
mesheryctl system context create aks-prod-cluster

Google GKE

# Get GKE credentials
gcloud container clusters get-credentials prod-cluster --zone us-central1-a

# Create Meshery connection
mesheryctl system context create gke-prod-cluster

On-Premises

# Ensure kubeconfig is configured
kubectl config use-context on-prem-cluster

# Create Meshery connection
mesheryctl system context create on-prem-cluster

AWS

EKS, EC2, ECS integrations

Azure

AKS, Container Instances

Google Cloud

GKE, Cloud Run, GCE

On-Premises

Self-managed Kubernetes

Centralized Observability

MeshSync Discovery

MeshSync runs in each cluster to discover resources:

# Check MeshSync status in cluster
kubectl get pods -n meshery -l app=meshsync

# View discovered resources
kubectl get meshsync -n meshery

MeshSync publishes discovery events to Meshery Server via NATS, enabling:

Real-time resource inventory
Cross-cluster visibility
Configuration drift detection

Unified Dashboard

Meshery UI provides centralized view across clusters:

Visualizer: See resources from all clusters
Filters: Toggle cluster visibility
Search: Find resources across infrastructure
Health: Monitor component status

Monitoring Integration

Connect monitoring systems for each cluster:

Prometheus Connections

Create connection for each cluster’s Prometheus:

# Production cluster 1 Prometheus
mesheryctl connections create \
  --name "prod-1-prometheus" \
  --type prometheus \
  --url http://prometheus.prod-1.svc:9090

# Production cluster 2 Prometheus  
mesheryctl connections create \
  --name "prod-2-prometheus" \
  --type prometheus \
  --url http://prometheus.prod-2.svc:9090

Grafana Dashboards

Connect Grafana instances:

mesheryctl connections create \
  --name "prod-grafana" \
  --type grafana \
  --url https://grafana.example.com

Aggregate Metrics

View aggregated metrics in Meshery Performance dashboard:

Query across clusters
Compare performance
Generate unified reports

Connection Management

Connection Lifecycle

Connections have the following statuses:

DISCOVERED - Found but not registered
REGISTERED - Registered, ready to connect
CONNECTED - Active, syncing data
DISCONNECTED - Previously connected, now offline
MAINTENANCE - Temporarily disabled
IGNORED - Discovered but deliberately ignored
DELETED - Marked for deletion

Updating Connections

# View connection details
mesheryctl connections view <connection-name>

# Update connection (e.g., rotate credentials)
# Via UI: Settings → Connections → Edit

Removing Connections

Deleting a connection removes Meshery components from the cluster and stops synchronization.

# Delete connection
mesheryctl connections delete <connection-name>

# This will:
# 1. Stop MeshSync
# 2. Remove Meshery Operator (optional)
# 3. Delete connection from Meshery

Security Considerations

Credential Management

Service Accounts: Use dedicated service accounts per cluster
Token Rotation: Regularly rotate authentication tokens
Least Privilege: Grant minimum required permissions
Secret Storage: Meshery encrypts stored credentials

Network Security

API Server Access: Ensure Meshery can reach cluster API servers
Firewalls: Whitelist Meshery Server IP addresses
VPN/Private Networks: Use secure networks for communication
mTLS: Enable mutual TLS where supported

RBAC Configuration

Minimum RBAC for Meshery Operator:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: meshery-operator
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments", "daemonsets", "statefulsets"]
  verbs: ["create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["services", "configmaps", "secrets"]
  verbs: ["create", "update", "patch", "delete"]

Troubleshooting

Connection Failures

Symptom: Connection status stuck at REGISTERED Solutions:

# Verify cluster accessibility
kubectl --context=<cluster> cluster-info

# Check Meshery operator logs
kubectl --context=<cluster> logs -n meshery -l app=meshery-operator

# Verify network connectivity
telnet <cluster-api-server> 6443

MeshSync Not Running

Symptom: Resources not appearing in Meshery Solutions:

# Check MeshSync deployment
kubectl get deployment meshsync -n meshery

# View MeshSync logs
kubectl logs -n meshery -l app=meshsync --tail=100

# Restart MeshSync
kubectl rollout restart deployment/meshsync -n meshery

Cross-Cluster Communication

Symptom: Unable to deploy across clusters Solutions:

Verify active connection: mesheryctl connections list
Switch context: mesheryctl system context switch <cluster>
Test kubectl access: kubectl get nodes
Check environment associations

Best Practices

Naming Conventions

Descriptive Names: aws-eks-prod-us-west-2
Environment Prefix: prod-, staging-, dev-
Provider Labels: Include cloud provider
Region Tags: Add geographic region

Organization

Environment Isolation: Separate prod/staging/dev
Workspace Alignment: Match teams to workspaces
Connection Grouping: Group by cloud provider or region

Monitoring

Health Checks: Regularly verify connection status
Resource Limits: Monitor cluster capacity
Sync Status: Ensure MeshSync is running
Event Monitoring: Watch Notification Center for issues

Next Steps

Implement Policy Engine across clusters
Set up Team Collaboration with workspaces
Configure Infrastructure Management
Explore Cloud Provider Integrations

​Why Multi-Cluster?

​Prerequisites

​Connecting Multiple Clusters

​Auto-Discovery from Kubeconfig

​Manual Connection Creation

​Organizing with Environments

​Cross-Cluster Deployments

​Deploying to Multiple Clusters

​Environment-Specific Configurations

​Multi-Cloud Management

​AWS EKS

​Azure AKS

​Google GKE

​On-Premises

AWS

Azure

Google Cloud

On-Premises

​Centralized Observability

​MeshSync Discovery

​Unified Dashboard

​Monitoring Integration

​Connection Management

​Connection Lifecycle

​Updating Connections

​Removing Connections

​Security Considerations

​Credential Management

​Network Security

​RBAC Configuration

​Troubleshooting

​Connection Failures

​MeshSync Not Running

​Cross-Cluster Communication

​Best Practices

​Naming Conventions

​Organization

​Monitoring

​Next Steps

Why Multi-Cluster?

Prerequisites

Connecting Multiple Clusters

Auto-Discovery from Kubeconfig

Manual Connection Creation

Organizing with Environments

Cross-Cluster Deployments

Deploying to Multiple Clusters

Environment-Specific Configurations

Multi-Cloud Management

AWS EKS

Azure AKS

Google GKE

On-Premises

Centralized Observability

MeshSync Discovery

Unified Dashboard

Monitoring Integration

Connection Management

Connection Lifecycle

Updating Connections

Removing Connections

Security Considerations

Credential Management

Network Security

RBAC Configuration

Troubleshooting

Connection Failures

MeshSync Not Running

Cross-Cluster Communication

Best Practices

Naming Conventions

Organization

Monitoring

Next Steps