Skip to main content

Intro

This section of the website provides you step by step process of

  • Setting up Ceph cluster in Raspberry Pi machine
  • Setting up Ente Photos and integrating with Ceph RGW S3 endpoint to manage your photos from mobile and desktop
  • Setting up Nextcloud and integrating with Ceph RGW S3 endpoint to manage you traditional files and data

Linux, if you think about it, is one of the most complex engineering software system in the world. And we have thousands of Linux admins to manage linux environments. However, we have billions of people using Android phone which uses linux, every minute of their life without knowing what happens in the background. In regards to the installation and management of CephBox, our philosophy is the same.

Ceph is a complex distributed storage system. But the end user doesn't need to know that!

Our vision for CephBox is for it to operate like a storage appliance: you purchase the hardware, plug it in, power it up, do the very minimal configuration and start using it immediately. While we’re still working towards that vision, that's a long way to go... In the meantime, if anyone wants to give a try, be it may on a single Raspberry Pi or on actual servers, follow this doc. (We are in the process of automating the installation of different components as much as possible here.)

You can setup everything on a single raspberry pi or server. But... that defeats the purpose of a highly available and resiliant distributed storage system such as Ceph.

As a general rule of thumb:

  • Single node with 2 disks --> You are casually testing the setup and don't intend to store any actual data.

  • Two node with 2 disks each --> You get the real feel of a distributed storage system. If you know to hack around Ceph, this would almost do in most cases.

  • Three node with 2 disks each --> You have decided to take the leap of faith and store your precious data in-house. Mean time to data loss (MTTDL) due to disk failures would be in decades..

    Sample Image

My Setup​

I am currently using a 2 node setup with 2 disks each.

  • Hardware:

    • Raspberry Pi 5 - 2 Nos
    • 512 GB NVMe card - 4 Nos
    • 64 GB Micro SD card - 2 Nos for Operating System
    • PCIe To 2-ch M.2 Adapter HAT - 2 Nos

  • Architecture

    • Ceph Cluster on 2 nodes
    • Ente photo app server on 1 node
    • Nextcloud server on the other node

  • Approximate cost of the setup

My setup is a bit overdone of course. (The universally accepted moto of any DIY project is : if it's worth doing, it's worth overdoing.) You can reduce it considerably, for example by replacing NVMe cards with SSDs.

  • Raspberry Pi 5 - 7900 * 2 = 15800
  • NVMe card - 2200 * 4 = 8800
  • 64 GB Micro SD card - 540 * 2 = 1080
  • PCI-e to 2 CH HAT - 1600 * 2 = 3200

The steps in the following sections are slightly aligned to the 2 node setup that I am having. We have tried to generalize it as much as we can, but keep this in your mind to avoid confusions.

Okay, let's start.. Click on the Next page button below ⬇️