MongoDB Setup

This guide covers how to set up and configure MongoDB for use with Gigantics, including installation, starting/stopping the service, and performance tuning, particularly when running MongoDB on the same machine as Gigantics.

Installation

Prerequisites

Gigantics requires MongoDB version 4.0 or higher. Before installing MongoDB, ensure your system meets the minimum requirements:

Linux/MacOS: MongoDB 4.0+ with WiredTiger storage engine
Windows: MongoDB 4.0+ with WiredTiger storage engine

For detailed system requirements, see the MongoDB Production Notes.

Recommended Hardware

For optimal performance with Gigantics:

Separate MongoDB Server: 8GB RAM minimum
Combined Gigantics + MongoDB Server: 16GB RAM minimum

When running both Gigantics and MongoDB on the same server, we recommend at least 16GB of total system RAM to ensure adequate resources for both applications.

Installing MongoDB

MacOS

Using Homebrew:

# Install MongoDB
brew tap mongodb/brew
brew install mongodb-community@6.0

# Start MongoDB service
brew services start mongodb-community@6.0

# Stop MongoDB service
brew services stop mongodb-community@6.0

Linux (Ubuntu/Debian)

# Import MongoDB public GPG key
wget -qO - https://www.mongodb.org/static/pgp/server-6.0.asc | sudo apt-key add -

# Create list file for MongoDB
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list

# Reload local package database
sudo apt-get update

# Install MongoDB packages
sudo apt-get install -y mongodb-org

# Start MongoDB service
sudo systemctl start mongod

# Enable MongoDB to start on boot
sudo systemctl enable mongod

# Stop MongoDB service
sudo systemctl stop mongod

Linux (CentOS/RHEL)

# Create MongoDB repository file
sudo vi /etc/yum.repos.d/mongodb-org-6.0.repo

# Add the following content to the file:
[mongodb-org-6.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/6.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-6.0.asc

# Install MongoDB
sudo yum install -y mongodb-org

# Start MongoDB service
sudo systemctl start mongod

# Enable MongoDB to start on boot
sudo systemctl enable mongod

# Stop MongoDB service
sudo systemctl stop mongod

Windows

Download the MongoDB Community Server from MongoDB Download Center
Run the installation wizard with default settings
MongoDB service will start automatically after installation

To manually start/stop MongoDB service on Windows:

# Start MongoDB service
net start MongoDB

# Stop MongoDB service
net stop MongoDB

Starting and Stopping MongoDB

Using System Services

MacOS (Homebrew):

# Start MongoDB
brew services start mongodb-community@6.0

# Stop MongoDB
brew services stop mongodb-community@6.0

# Check status
brew services list | grep mongodb

Linux (systemd):

# Start MongoDB
sudo systemctl start mongod

# Stop MongoDB
sudo systemctl stop mongod

# Restart MongoDB
sudo systemctl restart mongod

# Check status
sudo systemctl status mongod

# View MongoDB logs
sudo journalctl -u mongod

Manual Startup

To start MongoDB manually with specific configuration:

# Start MongoDB with default configuration
mongod

# Start MongoDB with specific configuration file
mongod --config /etc/mongod.conf

# Start MongoDB on a specific port and bind IP
mongod --port 27017 --bind_ip localhost

# Start MongoDB with custom database path
mongod --dbpath /var/lib/mongodb

MongoDB Configuration

Configuration File Location

MacOS: /usr/local/etc/mongod.conf
Linux: /etc/mongod.conf
Windows: C:\Program Files\MongoDB\Server\[version]\bin\mongod.cfg

Gigantics Configuration

Gigantics connects to MongoDB using the configuration in your config/default.yaml file:

mongodb:
  host: localhost
  port: 27017
  dbname: gigantics
  username: ''
  password: ''

For detailed information about configuring the MongoDB connection in Gigantics, see MongoDB Configuration.

Memory Management for Shared Servers

When running MongoDB on the same machine as Gigantics, you need to limit MongoDB's memory usage to prevent it from consuming all available RAM and leaving insufficient resources for Gigantics.

WiredTiger Cache Size Configuration

MongoDB's WiredTiger storage engine uses a cache to store frequently accessed data. By default, WiredTiger allocates up to 50% of available system RAM minus 1GB (or 256 MB, whichever is larger) for its cache, which can be excessive on servers with limited RAM.

To limit MongoDB memory usage, set the cacheSizeGB parameter in your MongoDB configuration file.

Configuration Example

Edit your MongoDB configuration file (/etc/mongod.conf on Linux or /usr/local/etc/mongod.conf on MacOS):

# mongod.conf

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
  wiredTiger:
    engineConfig:
      cacheSizeGB: 4

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1

Recommended Cache Sizes

When sharing a server with Gigantics, we recommend the following cache sizes based on total system RAM:

Total System RAM	Recommended WiredTiger Cache
16GB	4GB
32GB	8GB
64GB	16GB

To calculate the appropriate cache size for your system:

Determine total system RAM
Reserve at least 8GB for Gigantics runtime
Set cacheSizeGB to approximately 25-30% of remaining RAM

For example, on a 16GB server:

Total RAM: 16GB
Reserved for Gigantics: 8GB
Remaining: 8GB
Recommended cache size: 4GB (25% of remaining RAM)

Applying Memory Limit Configuration

After modifying the MongoDB configuration file, restart the MongoDB service:

MacOS (Homebrew):

brew services restart mongodb-community@6.0

Linux:

sudo systemctl restart mongod

Windows:

net stop MongoDB
net start MongoDB

Best Practices for Shared Environment

1. Limit Cache Size

Always configure cacheSizeGB when running MongoDB with Gigantics on the same server:

storage:
  wiredTiger:
    engineConfig:
      cacheSizeGB: [appropriate value based on your RAM]

2. Monitor Resource Usage

Use system monitoring tools to ensure both applications have adequate resources:

# View memory usage of MongoDB process
ps aux | grep mongod

# Monitor system resources continuously
top
# or
htop

3. Optimize Gigantics Workers

When running on a shared server, consider reducing the number of Gigantics workers:

# Start with fewer workers to reduce memory usage
./gig start -w=2 -j=2

4. Use Separate Drives (If Possible)

If your system has multiple drives, consider placing MongoDB data files on a separate drive from your Gigantics application:

storage:
  dbPath: /separate-drive/mongodb-data

Database Authentication

To secure your MongoDB installation, enable authentication:

Start MongoDB without authentication
Connect using the mongo shell:
```
mongosh
```

Create an admin user:

use admin
db.createUser({
  user: "admin",
  pwd: "your-secure-password",
  roles: ["userAdminAnyDatabase", "dbAdminAnyDatabase", "readWriteAnyDatabase"]
})

Edit MongoDB configuration file to enable authentication:
```
security:
  authorization: enabled
```
Restart MongoDB service

Update your Gigantics configuration to include credentials:

mongodb:
  host: localhost
  port: 27017
  dbname: gigantics
  username: 'admin'
  password: 'your-secure-password'

Troubleshooting

Connection Issues

If Gigantics cannot connect to MongoDB:

Check that MongoDB is running:

# Linux
sudo systemctl status mongod

# MacOS
brew services list | grep mongodb

Verify MongoDB is listening on the correct port:
```
mongosh --host localhost --port 27017
```
Check Gigantics logs for connection errors:
```
tail -f logs/out.txt
```

Memory Issues

If you experience memory issues with MongoDB:

Check current cache size configuration:

mongosh
db.serverStatus().wiredTiger.cache

Verify that cacheSizeGB is properly configured in your MongoDB configuration file
Consider further reducing the cache size if needed

On this page