Set up MongoDB 8.0 sharding cluster for horizontal scaling with replica sets and automated balancing

Advanced 45 min Apr 06, 2026 38 views
Ubuntu 24.04 Debian 12 AlmaLinux 9 Rocky Linux 9

Configure a production-grade MongoDB 8.0 sharded cluster with replica sets, config servers, and mongos routers for horizontal scaling. This tutorial covers automated chunk balancing, high availability, and enterprise-ready deployment patterns.

Prerequisites

  • At least 9 servers (3 config, 6 shard members)
  • Minimum 4GB RAM per server
  • Network connectivity between all nodes
  • Root or sudo access on all servers

What this solves

MongoDB sharding distributes data across multiple servers to handle large datasets and high-throughput operations that exceed single-server capacity. A properly configured sharded cluster provides horizontal scaling, automatic failover through replica sets, and transparent data distribution.

Step-by-step configuration

Update system packages and install MongoDB 8.0

Start by updating your package manager and installing MongoDB 8.0 on all cluster nodes.

sudo apt update && sudo apt upgrade -y
wget -qO - https://www.mongodb.org/static/pgp/server-8.0.asc | sudo apt-key add -
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/8.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-8.0.list
sudo apt update
sudo apt install -y mongodb-org
sudo dnf update -y
sudo tee /etc/yum.repos.d/mongodb-org-8.0.repo << 'EOF'
[mongodb-org-8.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/9/mongodb-org/8.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-8.0.asc
EOF
sudo dnf install -y mongodb-org

Configure config server replica set

Config servers store cluster metadata and shard configuration. Set up three config servers for high availability on separate nodes.

# Config Server Configuration
storage:
  dbPath: /var/lib/mongodb-config
  journal:
    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod-config.log

net:
  port: 27019
  bindIp: 0.0.0.0

processManagement:
  fork: true
  pidFilePath: /var/run/mongod-config.pid
  timeZoneInfo: /usr/share/zoneinfo

sharding:
  clusterRole: configsvr

replication:
  replSetName: configReplSet

security:
  authorization: enabled
  keyFile: /etc/mongodb-keyfile

Create shared authentication keyfile

Generate a keyfile for internal cluster authentication and distribute it to all cluster nodes with proper permissions.

sudo mkdir -p /etc/mongodb
sudo openssl rand -base64 756 | sudo tee /etc/mongodb-keyfile
sudo chown mongodb:mongodb /etc/mongodb-keyfile
sudo chmod 400 /etc/mongodb-keyfile
Never use chmod 777. Keyfiles must be readable only by the MongoDB user. Use chmod 400 for maximum security and proper ownership with chown.

Create data directories and start config servers

Create dedicated data directories for config servers and start the service on all three config server nodes.

sudo mkdir -p /var/lib/mongodb-config
sudo chown mongodb:mongodb /var/lib/mongodb-config
sudo chmod 755 /var/lib/mongodb-config
sudo mongod --config /etc/mongod-config.conf

Initialize config server replica set

Connect to one config server and initialize the replica set with all three config server members.

mongosh --port 27019
rs.initiate({
  _id: "configReplSet",
  configsvr: true,
  members: [
    { _id: 0, host: "203.0.113.10:27019" },
    { _id: 1, host: "203.0.113.11:27019" },
    { _id: 2, host: "203.0.113.12:27019" }
  ]
})
rs.status()

Create admin user on config servers

Create an administrative user with cluster management privileges on the config server replica set.

use admin
db.createUser({
  user: "clusterAdmin",
  pwd: "SecureClusterPass123!",
  roles: [
    { role: "clusterAdmin", db: "admin" },
    { role: "userAdminAnyDatabase", db: "admin" },
    { role: "readWriteAnyDatabase", db: "admin" }
  ]
})

Configure shard replica sets

Set up replica sets for each shard. Create configuration files for the first shard replica set members.

# Shard 1 Configuration
storage:
  dbPath: /var/lib/mongodb-shard1
  journal:
    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod-shard1.log

net:
  port: 27018
  bindIp: 0.0.0.0

processManagement:
  fork: true
  pidFilePath: /var/run/mongod-shard1.pid
  timeZoneInfo: /usr/share/zoneinfo

sharding:
  clusterRole: shardsvr

replication:
  replSetName: shard1ReplSet

security:
  authorization: enabled
  keyFile: /etc/mongodb-keyfile

Start shard replica set members

Create data directories and start MongoDB instances for the first shard replica set on three separate nodes.

sudo mkdir -p /var/lib/mongodb-shard1
sudo chown mongodb:mongodb /var/lib/mongodb-shard1
sudo chmod 755 /var/lib/mongodb-shard1
sudo mongod --config /etc/mongod-shard1.conf

Initialize shard replica sets

Connect to the primary shard node and initialize the shard replica set with all members.

mongosh --port 27018
rs.initiate({
  _id: "shard1ReplSet",
  members: [
    { _id: 0, host: "203.0.113.20:27018" },
    { _id: 1, host: "203.0.113.21:27018" },
    { _id: 2, host: "203.0.113.22:27018" }
  ]
})
rs.status()

Configure mongos routers

Set up mongos instances that route queries to appropriate shards. Mongos instances are stateless query routers.

# Mongos Router Configuration
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongos.log

net:
  port: 27017
  bindIp: 0.0.0.0

processManagement:
  fork: true
  pidFilePath: /var/run/mongos.pid
  timeZoneInfo: /usr/share/zoneinfo

sharding:
  configDB: configReplSet/203.0.113.10:27019,203.0.113.11:27019,203.0.113.12:27019

security:
  keyFile: /etc/mongodb-keyfile

Start mongos routers

Start mongos instances on multiple nodes for high availability and load distribution.

sudo mongos --config /etc/mongos.conf

Add shards to cluster

Connect to mongos and add the configured shard replica sets to the cluster.

mongosh --port 27017 -u clusterAdmin -p SecureClusterPass123! --authenticationDatabase admin
sh.addShard("shard1ReplSet/203.0.113.20:27018,203.0.113.21:27018,203.0.113.22:27018")
sh.addShard("shard2ReplSet/203.0.113.30:27018,203.0.113.31:27018,203.0.113.32:27018")
sh.status()

Enable sharding for databases and collections

Enable sharding on target databases and configure shard keys for optimal data distribution.

sh.enableSharding("myapp")
sh.shardCollection("myapp.users", { "user_id": 1 })
sh.shardCollection("myapp.orders", { "order_date": 1, "customer_id": 1 })

Configure chunk balancing settings

Customize the balancer settings for optimal performance and configure balancing windows if needed.

use config
db.settings.updateOne(
   { _id: "balancer" },
   { $set: { activeWindow : { start : "01:00", stop : "05:00" } } },
   { upsert: true }
)
sh.setBalancerState(true)
sh.getBalancerState()

Create systemd service files

Create systemd service files for automatic startup and management of MongoDB components.

[Unit]
Description=MongoDB Config Server
After=network.target

[Service]
Type=forking
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongod --config /etc/mongod-config.conf
PIDFile=/var/run/mongod-config.pid
Restart=always
RestartSec=10
StandardOutput=syslog
StandardError=syslog

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl enable mongod-config
sudo systemctl enable mongod-shard1
sudo systemctl enable mongos

Verify your setup

Check cluster status and verify proper shard distribution.

mongosh --port 27017 -u clusterAdmin -p SecureClusterPass123! --authenticationDatabase admin
sh.status()
use myapp
db.users.getShardDistribution()
db.stats()

For more advanced database configuration, check out our MongoDB 8.0 authentication and SSL encryption tutorial or learn about connection pooling strategies.

Common issues

SymptomCauseFix
Config server won't startIncorrect keyfile permissionssudo chown mongodb:mongodb /etc/mongodb-keyfile && sudo chmod 400 /etc/mongodb-keyfile
Shard addition failsReplica set not initializedConnect to shard and run rs.status() to verify replica set health
Balancer not runningBalancer disabled or window restrictionsh.setBalancerState(true) and check db.settings.find({_id:"balancer"})
Authentication failuresKeyfile mismatch between nodesEnsure identical keyfile content on all cluster members
High chunk migrationPoor shard key choiceAnalyze query patterns and consider resharding with compound shard key

Next steps

Automated install script

Run this to automate the entire setup

Need help?

Don't want to manage this yourself?

We handle high availability infrastructure for businesses that depend on uptime. From initial setup to ongoing operations.