How to Configure GlusterFS Distributed Storage on RHEL 7
GlusterFS is a scalable, open-source distributed filesystem that aggregates storage from multiple servers into a single, unified namespace. It is commonly used for high-availability shared storage in environments without expensive SAN hardware. RHEL 7 can consume GlusterFS via the CentOS Storage SIG repository or the upstream community packages. This tutorial covers installing GlusterFS server on multiple nodes, forming a trusted storage pool, creating and starting a volume, and mounting it on clients using the native FUSE client.
Prerequisites
- Two or more RHEL 7 servers with unique hostnames and static IP addresses
- A dedicated, unformatted brick partition or directory on each node (e.g.,
/dev/sdb) - Root access on all nodes
- All nodes resolvable by hostname (via DNS or
/etc/hosts) - Firewall ports 24007-24008/tcp, 49152+/tcp open between storage nodes and clients
- NTP configured so clocks are synchronized across all nodes
Step 1: Configure /etc/hosts Resolution
On every node (storage and client), ensure all storage hostnames resolve correctly. Edit /etc/hosts:
192.168.10.11 gluster1.example.com gluster1
192.168.10.12 gluster2.example.com gluster2
192.168.10.13 gluster3.example.com gluster3
GlusterFS internally uses hostnames during peer discovery; inconsistent resolution causes peer probe failures.
Step 2: Add the GlusterFS Repository
The CentOS Storage SIG provides GlusterFS packages compatible with RHEL 7. On all storage nodes:
yum install -y centos-release-gluster
If centos-release-gluster is not available directly, create the repo file manually:
cat > /etc/yum.repos.d/glusterfs.repo <<'EOF'
[glusterfs-rhel7]
name=GlusterFS for RHEL 7
baseurl=https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-9/
enabled=1
gpgcheck=0
EOF
Step 3: Install GlusterFS Server on All Nodes
Run the following on every storage node:
yum install -y glusterfs-server glusterfs glusterfs-fuse
systemctl enable glusterd
systemctl start glusterd
systemctl status glusterd
Open the required firewall ports on each node:
firewall-cmd --permanent --add-service=glusterfs
firewall-cmd --permanent --add-port=49152-49251/tcp
firewall-cmd --reload
Step 4: Prepare Brick Directories
GlusterFS recommends placing bricks on an XFS-formatted partition. On each storage node:
# Create and format the brick partition
mkfs.xfs -i size=512 /dev/sdb
mkdir -p /data/glusterfs/myvol/brick1
echo '/dev/sdb /data/glusterfs/myvol/brick1 xfs defaults 1 2' >> /etc/fstab
mount -a
Step 5: Form the Trusted Storage Pool
Run peer probe commands from a single node (e.g., gluster1). You only need to probe once per pair:
# On gluster1:
gluster peer probe gluster2
gluster peer probe gluster3
Verify the pool membership:
gluster peer status
Expected output shows each peer with state Peer in Cluster (Connected).
Step 6: Create a GlusterFS Volume
GlusterFS supports several volume types. Choose the one that matches your redundancy and performance requirements.
Distributed volume — files are spread across bricks without redundancy (maximum capacity, no HA):
gluster volume create myvol
gluster1:/data/glusterfs/myvol/brick1
gluster2:/data/glusterfs/myvol/brick1
gluster3:/data/glusterfs/myvol/brick1
Replicated volume — each file is duplicated on all bricks (full HA, capacity = 1 brick):
gluster volume create myvol replica 3
gluster1:/data/glusterfs/myvol/brick1
gluster2:/data/glusterfs/myvol/brick1
gluster3:/data/glusterfs/myvol/brick1
Distributed-replicated volume — recommended for large clusters (combines scale-out with redundancy):
# 6 bricks, replica count 3 → 2 distributed sets of 3 replicas
gluster volume create myvol replica 3
gluster1:/data/glusterfs/myvol/brick1
gluster2:/data/glusterfs/myvol/brick1
gluster3:/data/glusterfs/myvol/brick1
gluster1:/data/glusterfs/myvol/brick2
gluster2:/data/glusterfs/myvol/brick2
gluster3:/data/glusterfs/myvol/brick2
Step 7: Start the Volume
gluster volume start myvol
Confirm the volume is running:
gluster volume info myvol
gluster volume status myvol
Step 8: Mount the Volume on a Client
Install the FUSE client on the client machine (this can be any RHEL 7 host, including a storage node):
yum install -y glusterfs glusterfs-fuse
Mount using the native FUSE client. Specify any one storage node as the contact point — GlusterFS will discover the full topology automatically:
mkdir -p /mnt/gluster_data
mount -t glusterfs gluster1:/myvol /mnt/gluster_data
For a persistent mount, add to /etc/fstab:
gluster1:/myvol /mnt/gluster_data glusterfs defaults,_netdev 0 0
Verify the mount and write a test file:
df -h /mnt/gluster_data
echo "GlusterFS test" > /mnt/gluster_data/test.txt
cat /mnt/gluster_data/test.txt
Step 9: Mount via NFS or SMB (Optional)
GlusterFS can also serve volumes over NFS (using the built-in Gluster NFS server or NFS-Ganesha) and SMB via Samba. For basic NFS access without additional packages:
# On the client, mount via NFS (port 38465)
mount -t nfs -o vers=3 gluster1:/myvol /mnt/gluster_nfs
Ensure the gluster NFS option is enabled:
gluster volume set myvol nfs.disable off
Step 10: Verify Replication and Volume Health
gluster volume heal myvol info
gluster volume bitrot myvol enable
gluster volume quota myvol enable
gluster volume quota myvol limit-usage / 100GB
Check the self-heal status when a replica brick was temporarily offline:
gluster volume heal myvol info healed
gluster volume heal myvol info heal-failed
Conclusion
You have set up a fully functional GlusterFS distributed storage cluster on RHEL 7, created a volume of your chosen type, and mounted it on a client using the FUSE driver. GlusterFS’s flexible volume types allow you to balance capacity, throughput, and redundancy to match workload needs. For production use, consider enabling quota management to prevent single users from filling bricks, scheduling periodic self-heal checks, and monitoring cluster health with gluster volume status in a cron job or monitoring system such as Nagios or Prometheus with the GlusterFS exporter.
