More actions
Ceph Installation for Proxmox
Ceph is a very powerful enterprise level tool that has founds its way into may free an open source based projects. It provides a robust multi-island style file system that can used to provide fast, redundant, and reliable storage solutions. Proxmox comes with Ceph as an option bake into its its graphical user interface and this allows us to use it effectively and reliably with the platform.
In this guide I will be listing the steps required, to allow a Proxmox cluster to run HA (High Availability) VM's without the need for a traditional Network based storage such as a SAN or NAS.
The Hardware
The hardware for this setup is quite simple, but robust. it will be featuring 4 Proxmox Hosts, all in a singular cluster, with both a 1TB SATA SSD as the boot and local storage for the Proxmox installation, as well as a secondary 2TB NVME SSD in each, which will be the drives Ceph will be handling. Each of these devices also feature a 1Gbe connection, and a 10Gbe SFP+ Connection for the backbone and Ceph network.
The Layout
They Layout of this hardware is simple. We have a 10Gbe backbone network which Ceph will be using for data mirroring, as well as internal traffic of the cluster, but this network will be separated from all other traffic. On the 1Gbe interfaces will be standard VM traffic, in and out, as well as quorum for Proxmox, as we want that too have good latency, while not being bogged down by Ceph, Backup, our migration tasks running on the 10Gbe Network.
The Goal is to essentially mirror all of the 4 drives on the cluster with each other, in intervals, essentially making the 2TB in each machine, act as though it was a single 2TB disk shared between all systems. This gives us native NVME speeds not the disk, with much higher IOps then what a network storage solution can provide, while also allowing HA VM migrations, as the data for each VM on each host is available to all of them, allowing a drive, or a host to die completely and the Vm's be re-spun on a new compute resource with limited rollback.