Multi-Agent Data Replication Simulator
An agent-based simulation in Python where software agents place, replicate and monitor data across three MongoDB nodes, with a Flask dashboard to run it and watch the distribution.
Overview
I built this simulator to explore how a multi-agent system can manage data in a distributed database. Using the Mesa agent-based modelling framework, three kinds of agents work on three simulated MongoDB nodes: data managers write records, replication agents copy them to the other nodes, and monitoring agents record how many documents each node holds. A Flask web interface starts the simulation and shows the results as charts.
The problem
Spreading data over several database nodes raises practical questions: where to write each record, how to keep copies on other nodes for redundancy, and how to observe the state of the cluster while it changes. These are hard to see without a model you can run and watch.
What I built
I modelled the system with Mesa: a DataReplicationModel schedules six agents in random order at each step. DataManagerAgents choose a node and insert records, ReplicationAgents copy every document from one node to the others, and MonitoringAgents count the documents per node and store these metrics in a dedicated MongoDB collection. After each step the model updates its distribution figures. A Flask app runs the simulation, displays the document count per node and exposes JSON endpoints, including live CPU, memory and network metrics from psutil, which a Chart.js dashboard plots.
Results
- Runnable simulation with three agent types on three MongoDB nodes
- Metrics for every step stored in a separate MongoDB collection
- Flask interface with a distribution page and a Chart.js dashboard
- Unit tests for the agents, the model and the database connection
Highlights
- Agent-based modelling with Mesa and a random activation scheduler
- Data placement, replication and monitoring split into separate agents
- Three MongoDB databases used as simulated cluster nodes
- JSON endpoints for data distribution and real-time system metrics
- Command-line runner and web interface for the same model
