High-Performance Data Architecture with MongoDB
MongoDB is the database of choice for modern scalable applications due to its flexible JSON-native document model, horizontal sharding capabilities, and rich aggregation framework.
However, poor schema design and missing indexes often lead to slow table-scan queries (COLLSCAN), high CPU utilization, and application timeouts.
In this guide, we break down the engineering principles required to maintain sub-10ms query execution in production.
1. Document Modeling: Embedding vs Referencing
A common mistake in NoSQL is designing schemas either as completely normalized relational tables (too many references) or storing infinitely growing arrays inside a single document (violating the 16MB document limit).
Apply these rules of thumb:
- •Embed when:
- • - Data is 1:1 or 1:Few (e.g. an order with up to 10 line items, or a user with 2 address records).
- • - Data is consistently queried and displayed together.
- • - The embedded array does not grow unboundedly.
- •Reference when:
- • - Data is 1:Many (e.g. a course with thousands of enrolled student logs) or Many:Many.
- • - Documents frequently mutate independently of the parent record.
- • - You need to avoid the 16MB BSON size ceiling.
2. The ESR Indexing Rule (Equality, Sort, Range)
Compound indexes are crucial for optimizing multi-criteria queries. When structuring the order of keys in a compound index, always follow the ESR Rule:
- 1.E — Equality: Fields queried using exact matches (
status: 'PUBLISHED',categoryId: 'cat_web') should come FIRST. - 2.S — Sort: Fields used for ordering results (
createdAt: -1) should come SECOND. - 3.R — Range: Fields queried using range filters (
score: { $gte: 75 },date: { $lt: now }) should come LAST.
// Query:
// db.submissions.find({ status: 'APPROVED', score: { $gte: 80 } }).sort({ submittedAt: -1 })
// OPTIMAL INDEX applying the ESR rule:
// 1. Equality: status
// 2. Sort: submittedAt
// 3. Range: score
db.submissions.createIndex({ status: 1, submittedAt: -1, score: 1 });3. Profiling Queries with `explain("executionStats")`
Never guess why a database query is slow. Use MongoDB's explain() method to inspect the query execution plan:
const stats = await db.collection('blog_posts')
.find({ status: 'PUBLISHED', categoryName: 'Web Development' })
.sort({ publishedAt: -1 })
.explain('executionStats');
console.log('Execution Stage:', stats.executionStats.executionStages.stage);
console.log('Total Docs Examined:', stats.executionStats.totalDocsExamined);
console.log('Total Keys Examined:', stats.executionStats.totalKeysExamined);
console.log('Execution Time (ms):', stats.executionStats.executionTimeMillis);- •Red Flag: If
totalDocsExaminedis much higher than the number of returned documents, or if the stage isCOLLSCAN, your query is scanning entire collections sequentially. - •Target: Aim for
IXSCAN(Index Scan) withtotalDocsExaminedequal to the return count.
4. Advanced Aggregation Pipelines: `$facet` for Search & Faceting
The aggregation pipeline processes data through sequential stages. Using $facet, you can calculate total match counts, paginated documents, and category distributions in a single round-trip query:
const results = await db.collection('courses').aggregate([
{ $match: { published: true } },
{
$facet: {
paginatedResults: [
{ $sort: { enrolledCount: -1 } },
{ $skip: 0 },
{ $limit: 10 },
{ $project: { title: 1, slug: 1, category: 1, enrolledCount: 1 } }
],
totalCount: [
{ $count: 'total' }
],
categories: [
{ $group: { _id: '$category', count: { $sum: 1 } } }
]
}
}
]).toArray();5. Serverless Connection Pooling Best Practices
In serverless environments like Next.js API routes or AWS Lambda, creating a new MongoClient on every incoming request quickly exhausts database connection pools. Always cache the client promise globally:
// lib/db/mongodb.ts
import { MongoClient } from 'mongodb';
const uri = process.env.MONGODB_URI!;
let cachedClient: MongoClient | null = null;
export async function getMongoDb() {
if (!cachedClient) {
cachedClient = new MongoClient(uri, {
maxPoolSize: 10,
serverSelectionTimeoutMS: 5000,
});
await cachedClient.connect();
}
return cachedClient.db('prayxis_production');
}Master database engineering and backend design in the PRAYXIS Full Stack Web Development Track.