Step-by-Step Proxmox and Ceph High Availability Setup Guide | Free High Availability Storage

by SinaOnline System Training, Virtualization Training

Step 1: Prepare Proxmox Nodes

  1. Update and Upgrade Proxmox VE on all nodes:

apt update && apt full-upgrade -y

2. Ensure that all nodes have the same version of Proxmox VE:

pveversion

Step 2: Set Up the Proxmox Cluster

  1. Create a new cluster on the first node:
    • pvecm create my-cluster
  2. Add the other nodes to the cluster:
    • pvecm add <IP_of_first_node>
  3. Verify the cluster status:
    • pvecm status

Step 3: Install Ceph on Proxmox Nodes

  1. Install Ceph packages on all nodes:

install ceph ceph-mgr -y

Step 4: Create the Ceph Cluster

  1. Initialize the Ceph cluster on the first node:
    • pveceph init --network <cluster_network>
  2. Create the manager daemon on the first node:
    • pveceph createmgr

Step 5: Add OSDs (Object Storage Daemons)

  1. Prepare disks on each node for Ceph OSDs:
    • pveceph createosd /dev/sdX
  2. Repeat the process for each node and disk.

Step 6: Create Ceph Pools

  1. Create a Ceph pool for VM storage:
    • pveceph pool create mypool 128

Step 7: Configure Proxmox to Use Ceph Storage

  1. Add the Ceph storage to Proxmox:
    • Navigate to Datacenter > Storage > Add > RBD.
    • Enter the required details like ID, Pool, and Monitor hosts.
    • Save the configuration.

Step 8: Enable HA (High Availability)

  1. Configure HA on Proxmox:
    • Navigate to Datacenter > HA.
    • Add resources (VMs or containers) to the HA manager.
    • Configure the HA policy and set desired node priorities.

Step 9: Testing High Availability

  1. Simulate node failure: Power off one of the nodes and observe how the VMs or containers are automatically migrated to other nodes.

Step 10: Monitoring and Maintenance

  1. Use the Proxmox and Ceph dashboards to monitor the health of your cluster.
  2. Regularly update all nodes to ensure stability and security.

Optional: Additional Ceph Configuration

  1. Add Ceph Monitors for redundancy:bashKodu kopyalapveceph createmon
  2. Add more Ceph MDS (Metadata Servers) if using CephFS:bashKodu kopyalapveceph createmds
  3. Tune Ceph settings for performance and reliability based on your specific needs.

By following these steps, you will have a robust Proxmox VE and Ceph high availability setup, ensuring that your VMs and containers remain highly available even in the event of hardware failures.