NPM npm

GitHub branch checks state Sonar Quality Gate

Sonar Tests Sonar Coverage

MongoDB Pipeline Builder

A type-safe, fluent API for building MongoDB aggregation pipelines

🚀 Overview

mongodb-pipeline-builder is a powerful TypeScript library that simplifies the creation of MongoDB aggregation pipelines. It provides a fluent, type-safe API that makes your aggregation pipelines more readable, maintainable, and less error-prone.

Key Features

✨ Type-Safe - Full TypeScript support with generics for typed responses
📖 Readable - Fluent API that makes complex pipelines easy to understand
🔧 Maintainable - Modular design with reusable helpers
🎯 Complete - Supports all MongoDB aggregation stages and operators
⚡ Efficient - Built-in pagination support with optimized queries
🛡️ Validated - Automatic validation of pipeline stages

Supported Platforms

✅ MongoDB Native Driver - db.collection.aggregate()
✅ MongoDB Database Commands - db.aggregate()
✅ Mongoose - Model.aggregate()

📦 Installation

npm install mongodb-pipeline-builder

yarn add mongodb-pipeline-builder

pnpm add mongodb-pipeline-builder

🎯 Quick Start

Basic Example

import { PipelineBuilder } from 'mongodb-pipeline-builder';
import { ProjectOnlyHelper } from 'mongodb-pipeline-builder/helpers';
import { $Expression, $Equal } from 'mongodb-pipeline-builder/operators';

const pipeline = new PipelineBuilder('users-query')
  .Match($Expression($Equal('$status', 'active')))
  .Project(ProjectOnlyHelper('name', 'email', 'createdAt'))
  .Sort({ createdAt: -1 })
  .Limit(10)
  .build();

// Use with MongoDB
const results = await db.collection('users').aggregate(pipeline).toArray();

// Use with Mongoose
const results = await User.aggregate(pipeline);

With Pagination

import { PipelineBuilder, GetPagingResult } from 'mongodb-pipeline-builder';
import { $Expression, $GreaterThan } from 'mongodb-pipeline-builder/operators';

const pipeline = new PipelineBuilder('paginated-users')
  .Match($Expression($GreaterThan('$age', 18)))
  .Sort({ createdAt: -1 })
  .Paging(20, 1) // 20 items per page, page 1
  .build();

const result = await GetPagingResult(User, pipeline);

console.log(result.GetDocs());           // Document array
console.log(result.GetCount());          // Total count
console.log(result.GetTotalPageNumber()); // Total pages

With Lookups (Joins)

import { PipelineBuilder } from 'mongodb-pipeline-builder';
import { LookupEqualityHelper, Field } from 'mongodb-pipeline-builder/helpers';
import { $ArrayElementAt } from 'mongodb-pipeline-builder/operators';

const pipeline = new PipelineBuilder('users-with-profiles')
  .Lookup(LookupEqualityHelper('profiles', 'profile', 'profileId', '_id'))
  .AddFields(
    Field('profileData', $ArrayElementAt('$profile', 0))
  )
  .Unset('profile')
  .build();

📚 Documentation

Core Concepts

Getting Started Guide - Detailed introduction and setup
Migration Guide v3 → v4 - Upgrade from version 3

API Reference

Pipeline Stages - All MongoDB aggregation stages
Operators - Aggregation operators ($match, $group, etc.)
Helpers - Utility functions and stage helpers (Field, ProjectHelper, LookupHelper, etc.)

Examples & Tutorials

Pagination Examples - Implementing pagination patterns
Lookup & Join Examples - Working with multiple collections

🔥 What's New in v4

Breaking Changes

PipelineBuilder

✨ build() replaces getPipeline()
🆕 New stages: ChangeStream, ChangeStreamSplitLargeEvent, Densify, Documents, Fill, ListLocalSessions, ListSampledQueries, ListSearchIndexes, SearchMeta, SetWindowFields, ShardedDataDistribution
🆕 Insert() stage for adding custom stages without validation
✅ Automatic detection of non-duplicable stages

Helpers

🔄 Payload suffix → Helper suffix
🏷️ Prefixed with stage name (e.g., LookupEqualityHelper)

Operators

🏷️ All operators prefixed with $ (e.g., $Add, $Match)
🔄 MapOperator → $Map
🆕 New Aggregation Operators (MongoDB 7.0+): $Median, $Percentile, $Top, $TopN
🆕 New Object Operators (MongoDB 5.0+): $GetField, $SetField

Result Methods

🎯 GetResult<T>() - For non-paginated queries
- ✨ GetElement(index | 'last') - New method to get specific document
- 🚀 Generic type support for typed responses
🎯 GetPagingResult<T>() - Exclusively for paginated queries
- 🚀 Generic type support for typed responses

💡 Real-World Example

import { PipelineBuilder, GetResult } from 'mongodb-pipeline-builder';
import { 
  LookupEqualityHelper, 
  ProjectOnlyHelper, 
  Field 
} from 'mongodb-pipeline-builder/helpers';
import { 
  $Expression, 
  $And,
  $Equal, 
  $GreaterThanEqual,
  $ArrayElementAt 
} from 'mongodb-pipeline-builder/operators';

// Complex query: Active users with their orders
const pipeline = new PipelineBuilder('active-users-with-orders', { debug: true })
  // Filter active users over 18
  .Match($Expression(
    $And(
      $Equal('$status', 'active'),
      $GreaterThanEqual('$age', 18)
    )
  ))
  
  // Join with orders collection
  .Lookup(LookupEqualityHelper('orders', 'orders', '_id', 'userId'))
  
  // Project specific fields
  .Project(ProjectOnlyHelper('name', 'email', 'age'))
  
  // Add computed fields
  .AddFields(
    Field('totalOrders', { $size: '$orders' }),
    Field('lastOrder', $ArrayElementAt('$orders', -1))
  )
  
  // Sort by total orders descending
  .Sort(Field('totalOrders', -1))
  
  // Limit results
  .Limit(50)
  
  .build();

// Execute query with typed response
interface UserWithOrders {
  name: string;
  email: string;
  age: number;
  totalOrders: number;
  lastOrder?: any;
}

const result = await GetResult<UserWithOrders>(User, pipeline);

// Access results
const users = result.GetDocs();        // UserWithOrders[]
const count = result.GetCount();       // number
const firstUser = result.GetElement(0); // UserWithOrders | undefined
const lastUser = result.GetElement('last'); // UserWithOrders | undefined

🧪 Try It Online

→ Try the library on NPM RunKit

📊 Comparison: Before & After

Before (Raw MongoDB)

const pipeline = [
  { 
    $match: { 
      $expr: { 
        $and: [
          { $eq: ["$status", "active"] },
          { $gte: ["$age", 18] }
        ]
      } 
    } 
  },
  {
    $lookup: {
      from: "orders",
      localField: "_id",
      foreignField: "userId",
      as: "orders"
    }
  },
  { 
    $project: { 
      _id: 0, 
      name: 1, 
      email: 1, 
      age: 1,
      totalOrders: { $size: "$orders" },
      lastOrder: { $arrayElemAt: ["$orders", -1] }
    } 
  },
  { $sort: { totalOrders: -1 } },
  { $limit: 50 }
];

After (With Pipeline Builder)

const pipeline = new PipelineBuilder('active-users')
  .Match($Expression($And(
    $Equal('$status', 'active'),
    $GreaterThanEqual('$age', 18)
  )))
  .Lookup(LookupEqualityHelper('orders', 'orders', '_id', 'userId'))
  .Project(
    ProjectOnlyHelper('name', 'email', 'age'),
    Field('totalOrders', { $size: '$orders' }),
    Field('lastOrder', $ArrayElementAt('$orders', -1))
  )
  .Sort(Field('totalOrders', -1))
  .Limit(50)
  .build();

🔍 Working with Results

GetResult - For Non-Paginated Queries

Use when your pipeline does not include the Paging stage:

import { GetResult } from 'mongodb-pipeline-builder';

interface User {
  name: string;
  email: string;
  age: number;
}

const pipeline = new PipelineBuilder('users')
  .Match($Expression($Equal('$status', 'active')))
  .build();

const result = await GetResult<User>(User, pipeline);

const users = result.GetDocs();           // User[] - all documents  
const count = result.GetCount();          // number - total count
const firstUser = result.GetElement(0);   // User | undefined
const lastUser = result.GetElement('last'); // User | undefined

GetPagingResult - For Paginated Queries

Use exclusively when your pipeline includes the Paging stage:

import { GetPagingResult } from 'mongodb-pipeline-builder';

const pipeline = new PipelineBuilder('users')
  .Match($Expression($Equal('$status', 'active')))
  .Paging(20, 1) // Required for GetPagingResult
  .build();

const result = await GetPagingResult<User>(User, pipeline);

const users = result.GetDocs();           // User[] - current page
const totalCount = result.GetCount();     // number - total documents
const totalPages = result.GetTotalPageNumber(); // number - total pages

📖 Learn More: See Getting Started Guide for detailed examples

📚 API Reference Quick Links

Pipeline Stages

All MongoDB aggregation stages are supported. See complete reference.

Common Stages:

Match() - Filter documents
Project() - Select/transform fields
Group() - Group and aggregate
Sort() - Sort documents
Limit() - Limit results
Lookup() - Join collections
Unwind() - Deconstruct arrays
AddFields() - Add computed fields
Paging() - Add pagination

Operators

156+ MongoDB operators supported. See complete reference.

Common Operators:

Comparison: $Equal, $GreaterThan, $LessThan
Logical: $And, $Or, $Not
Arithmetic: $Add, $Subtract, $Multiply, $Divide
Array: $Size, $Filter, $Map, $ArrayElementAt
String: $Concat, $ToLower, $ToUpper, $Split
Date: $DateAdd, $DateSubtract, $DateDifference
Aggregation: $Sum, $Average, $Min, $Max, $Median, $Percentile, $Top, $TopN
Object: $GetField, $SetField, $MergeObjects

Helpers

20+ helper functions for common patterns. See complete list in stages.md.

Common Helpers:

Field(name, value) - Universal field helper
ProjectOnlyHelper(...fields) - Include specific fields
LookupEqualityHelper(...) - Simple joins
BucketHelper(...) - Categorize documents
SearchHelper(...) - Full-text search

🎓 Quick Examples

Example 1: Simple Query

const pipeline = new PipelineBuilder('active-users')
  .Match($Expression($Equal('$status', 'active')))
  .Project(ProjectOnlyHelper('name', 'email'))
  .Sort({ createdAt: -1 })
  .Limit(10)
  .build();

const result = await GetResult<User>(User, pipeline);

Example 2: With Pagination

const pipeline = new PipelineBuilder('paginated')
  .Match($Expression($GreaterThan('$price', 100)))
  .Sort({ price: -1 })
  .Paging(20, 1)
  .build();

const result = await GetPagingResult<Product>(Product, pipeline);
console.log(`Page 1 of ${result.GetTotalPageNumber()}`);

Example 3: With Joins

const pipeline = new PipelineBuilder('users-with-orders')
  .Lookup(LookupEqualityHelper('orders', 'orders', '_id', 'userId'))
  .AddFields(
    Field('orderCount', $Size('$orders')),
    Field('lastOrder', $ArrayElementAt('$orders', -1))
  )
  .Match($Expression($GreaterThan('$orderCount', 0)))
  .build();

Example 4: Complex Aggregation

const pipeline = new PipelineBuilder('sales-by-category')
  .Match($Expression($Equal('$year', 2024)))
  .Group({
    _id: '$category',
    totalSales: $Sum('$amount'),
    averagePrice: $Average('$price'),
    count: $Sum(1)
  })
  .Sort({ totalSales: -1 })
  .Limit(10)
  .build();

Example 5: Statistical Analysis (MongoDB 7.0+)

import { $Median, $Percentile, $Top, $TopN } from 'mongodb-pipeline-builder/operators';

const pipeline = new PipelineBuilder('price-stats')
  .Match($Expression($Equal('$year', 2024)))
  .Group({
    _id: '$category',
    medianPrice: $Median('$price'),
    p95Price: $Percentile('$price', [0.95]),
    topProduct: $Top({ sales: -1 }, '$productName'),
    top3Products: $TopN(3, { sales: -1 }, '$productName', '$price')
  })
  .Sort({ _id: 1 })
  .build();

❓ FAQ

When should I use GetResult vs GetPagingResult?

Use GetResult for queries without the Paging stage
Use GetPagingResult exclusively when using the Paging stage

The Paging stage wraps your pipeline in $facet for efficient pagination.

Can I use raw MongoDB syntax?

Yes! Use the Insert stage:

builder.Insert({ $myCustomStage: { field: 'value' } })

Useful for new MongoDB stages or custom implementations.

Does it work with Mongoose?

Yes! Works with:

MongoDB Native Driver: db.collection.aggregate()
Mongoose: Model.aggregate()
MongoDB Database Commands: db.aggregate()

Are all MongoDB stages supported?

Yes! All stages through MongoDB 7.0+ including $densify, $fill, $setWindowFields, $searchMeta, and more.

What new operators were added in recent versions?

MongoDB 7.0+ introduced several statistical operators:

$Median - Calculates the median of values (MongoDB 7.2+)
$Percentile - Calculates percentiles (MongoDB 7.2+)
$Top - Returns the top element in a group (MongoDB 7.0+)
$TopN - Returns the top N elements in a group (MongoDB 7.0+)
$GetField - Retrieves a field value dynamically (MongoDB 5.0+)
$SetField - Sets a field value dynamically (MongoDB 5.0+)

See Example 5 for usage.

How do I handle TypeScript types?

Use generics:

interface User { name: string; email: string; }
const result = await GetResult<User>(User, pipeline);
const users: User[] = result.GetDocs(); // Fully typed!

Can I chain multiple lookups?

Yes! Chain as many as needed:

builder
  .Lookup(LookupEqualityHelper('profiles', 'profile', '_id', 'userId'))
  .Lookup(LookupEqualityHelper('orders', 'orders', '_id', 'userId'))
  .Lookup(LookupEqualityHelper('subscriptions', 'sub', '_id', 'userId'))

See Lookup Examples.

📈 Performance Tips

Index your fields - Create indexes on $match and $sort fields
Match early - Apply $match as early as possible
Project early - Remove unnecessary fields to reduce memory
Limit lookups - Use pipeline syntax to limit joined documents
Use covered queries - Query only indexed fields when possible

📖 Learn More: Pagination Examples

🆘 Support

Getting Help

Reporting Issues

Include:

Package version
Node.js and MongoDB versions
Code snippet
Expected vs actual behavior

📜 Changelog

See CHANGELOG.md for version history.

Made with ❤️ by Mickaël NODANCHE

MongoDB Pipeline Builder

🚀 Overview

Key Features

Supported Platforms

📦 Installation

🎯 Quick Start

Basic Example

With Pagination

With Lookups (Joins)

📚 Documentation

Core Concepts

API Reference

Examples & Tutorials

🔥 What's New in v4

Breaking Changes

PipelineBuilder

Helpers

Operators

Result Methods

💡 Real-World Example

🧪 Try It Online

📊 Comparison: Before & After

Before (Raw MongoDB)

After (With Pipeline Builder)

🔍 Working with Results

GetResult - For Non-Paginated Queries

GetPagingResult - For Paginated Queries

📚 API Reference Quick Links

Pipeline Stages

Operators

Helpers

🎓 Quick Examples

Example 1: Simple Query

Example 2: With Pagination

Example 3: With Joins

Example 4: Complex Aggregation

Example 5: Statistical Analysis (MongoDB 7.0+)

❓ FAQ

📈 Performance Tips

🆘 Support

Getting Help

Reporting Issues

📜 Changelog

results matching ""

No results matching ""