Documentation

Quick Start

Deploy Burrow to AWS using our interactive CLI that automates Terraform and simplifies setup.

Prerequisites

To get started with Burrow, ensure the following are set up:

• An active AWS account
• AWS CLI installed and configured Run aws configure and provide:
  • AWS Access Key ID
  • AWS Secret Access Key
  • Default region name
• An existing VPC with subnets:
  • 2 public subnets
  • 2 private subnets
• Node.js and npm installed on your system

Installation

Clone the burrow-cli repository:

git clone https://github.com/burrow-io/burrow-cli.git

Navigate to the cloned directory:

cd burrow-cli

Install the required dependencies:

npm install

Deploying Burrow

To deploy Burrow infrastructure using the CLI, execute the following command:

npm run deploy

This command will guide you through the deployment process with interactive prompts:

Configuration Prompts

You will be asked to provide the following information, one prompt at a time:

• AWS Region - The region where you want to deploy Burrow (e.g., us-east-1)
• VPC ID - Your existing VPC identifier (e.g., vpc-0abcd1234efgh5678)
• First Public Subnet ID - Your existing first public subnet
• Second Public Subnet ID - Your existing second public subnet
• First Private Subnet ID - Your existing first private subnet
• Second Private Subnet ID - Your existing second private subnet

Deployment Outputs

Once deployment completes, you'll receive the following information:

• CloudFront DNS Name: The URL to access your Burrow frontend (e.g., https://d1234abcd5678.cloudfront.net)
• RAG API Key: Authentication key for making requests to the RAG API
• Admin Password: Credentials for logging into the frontend dashboard

Save these outputs—you'll need them to access and use Burrow.

Accessing the User Interface

Once deployment is complete, navigate to the CloudFront DNS URL provided in the deployment outputs.

Logging In

Use the admin credentials from the deployment output to log into the dashboard:

• Username: admin
• Password: [provided during deployment]

Using the Dashboard

From the UI, you can:

• Upload documents for processing through a simple drag-and-drop interface
• View document metadata including name, size, upload date, and processing status
• Track processing progress as documents move through different stages
• Monitor status updates delivered via server-sent events from the Management API

The event-driven architecture means you simply upload documents and Burrow handles everything else behind the scenes—no manual intervention required.

Querying Documents

Once documents are processed, you can query them using the RAG API.

Making API Requests

Use the Query API key provided during deployment to authenticate your requests. All requests go through your CloudFront distribution:

curl -X POST https://your-cloudfront-dns/query-service \
  -H 'x-api-token: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "What are the main topics discussed?",
    "top_k": 5
  }'

Advanced RAG Features

The Query API supports multiple retrieval techniques:

• Keyword Search: Traditional term-based matching
• Semantic Search: Vector similarity using embeddings
• Hybrid Search: Combines keyword and semantic approaches
• Reranking: Re-orders results for improved relevance
• Metadata Filtering: Filter by document properties (date, type, etc.)

These features enable you to retrieve the most relevant document chunks for feeding into your LLM-powered RAG applications.

Destroying Burrow Infrastructure

To tear down the Burrow infrastructure, run the following command:

npm run destroy

This command will perform the following steps:

Resource Destruction

Deletes all Burrow-related resources in your AWS account, including:

• CloudFront distribution
• S3 buckets (frontend assets and document storage)
• Application Load Balancer
• ECS clusters and services
• DynamoDB tables
• RDS Aurora database
• EventBridge rules and targets
• Security groups and IAM roles

Preserved Resources

The following resources will not be deleted:

• Your VPC
• Public and private subnets you provided

This allows you to safely destroy Burrow without affecting other infrastructure in your AWS account.

Terraform will display progress as resources are destroyed.

Confirmation

Once complete, you will receive a confirmation message that all Burrow resources have been removed.

⚠️ Important: Destroying the infrastructure will permanently delete all associated resources and data, including uploaded documents and vector embeddings. This action is irreversible, so proceed with caution.

Overview

Burrow provides two main APIs for document management and retrieval. The Pipeline API handles document ingestion and processing, while the Query API provides retrieval capabilities for your RAG application.

Burrow Pipeline API

The Pipeline API handles document ingestion, processing, and management operations. It provides endpoints for uploading documents, tracking their processing status, and managing the document lifecycle.

https://your-burrow-instance.com

{
  "type": "object",
  "properties": {
    "jwt": {
      "type": "string",
      "description": "JWT authentication token"
    }
  }
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "type": "object",
  "properties": {
    "items": {
      "type": "array",
      "items": {
        "$ref": "#/components/schemas/DocumentData"
      }
    },
    "lastEvaluatedKey": {
      "type": "string",
      "description": "Key for pagination to get next page"
    }
  }
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/DocumentData"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/DocumentData"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "type": "object",
  "properties": {
    "message": {
      "type": "string",
      "example": "Delete initiated"
    },
    "documentId": {
      "type": "string"
    },
    "status": {
      "type": "string",
      "enum": [
        "deleting"
      ]
    }
  }
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "$ref": "#/components/schemas/Error"
}

{
  "type": "object",
  "properties": {
    "documentId": {
      "type": "string"
    },
    "fileName": {
      "type": "string"
    },
    "size": {
      "type": "number"
    },
    "status": {
      "type": "string",
      "enum": [
        "pending",
        "running",
        "finished",
        "failed",
        "deleting",
        "deleted",
        "delete_failed"
      ]
    },
    "mimetype": {
      "type": "string"
    },
    "createdAt": {
      "type": "string"
    }
  }
}

{
  "type": "object",
  "properties": {
    "error": {
      "type": "string"
    }
  }
}

Burrow Query API

Vector database query API for document retrieval The RAG API provides endpoints for querying documents, retrieving relevant content, and performing vector similarity searches across your ingested data.

https://your-burrow-instance.com/query-service

{
  "properties": {
    "query": {
      "type": "string",
      "title": "Query",
      "description": "Query text for similarity search"
    },
    "top_k": {
      "type": "integer",
      "maximum": 100,
      "minimum": 1,
      "title": "Top K",
      "description": "Number of top results to return",
      "default": 5
    },
    "mode": {
      "$ref": "#/components/schemas/SearchMode",
      "description": "Search mode: vector, keyword, or hybrid",
      "default": "vector"
    },
    "rerank": {
      "type": "boolean",
      "title": "Rerank",
      "description": "Whether to apply reranking using Cohere Rerank",
      "default": false
    },
    "rerank_top_n": {
      "anyOf": [
        {
          "type": "integer",
          "maximum": 50,
          "minimum": 1
        },
        {
          "type": "null"
        }
      ],
      "title": "Rerank Top N",
      "description": "Number of results after reranking (defaults to top_k)"
    },
    "filters": {
      "anyOf": [
        {
          "$ref": "#/components/schemas/MetadataFilters"
        },
        {
          "type": "null"
        }
      ],
      "description": "Optional metadata filters"
    }
  },
  "type": "object",
  "required": [
    "query"
  ],
  "title": "RetrieveRequest"
}

{
  "properties": {
    "nodes": {
      "items": {
        "$ref": "#/components/schemas/NodeResponse"
      },
      "type": "array",
      "title": "Nodes",
      "description": "Retrieved nodes"
    },
    "query": {
      "type": "string",
      "title": "Query",
      "description": "Original query"
    },
    "total_results": {
      "type": "integer",
      "title": "Total Results",
      "description": "Number of results returned"
    }
  },
  "type": "object",
  "required": [
    "nodes",
    "query",
    "total_results"
  ],
  "title": "RetrieveResponse"
}

{
  "properties": {
    "detail": {
      "items": {
        "$ref": "#/components/schemas/ValidationError"
      },
      "type": "array",
      "title": "Detail"
    }
  },
  "type": "object",
  "title": "HTTPValidationError"
}

{
  "properties": {
    "query": {
      "type": "string",
      "title": "Query",
      "description": "Query text"
    },
    "top_k": {
      "type": "integer",
      "maximum": 100,
      "minimum": 1,
      "title": "Top K",
      "description": "Number of documents to retrieve",
      "default": 5
    },
    "filters": {
      "anyOf": [
        {
          "$ref": "#/components/schemas/MetadataFilters"
        },
        {
          "type": "null"
        }
      ],
      "description": "Optional metadata filters"
    }
  },
  "type": "object",
  "required": [
    "query"
  ],
  "title": "QueryRequest"
}

{
  "properties": {
    "response": {
      "type": "string",
      "title": "Response",
      "description": "Synthesized response"
    },
    "source_nodes": {
      "items": {
        "$ref": "#/components/schemas/NodeResponse"
      },
      "type": "array",
      "title": "Source Nodes",
      "description": "Source nodes used"
    },
    "query": {
      "type": "string",
      "title": "Query",
      "description": "Original query"
    }
  },
  "type": "object",
  "required": [
    "response",
    "source_nodes",
    "query"
  ],
  "title": "QueryResponse"
}

{
  "properties": {
    "detail": {
      "items": {
        "$ref": "#/components/schemas/ValidationError"
      },
      "type": "array",
      "title": "Detail"
    }
  },
  "type": "object",
  "title": "HTTPValidationError"
}

{
  "properties": {
    "detail": {
      "items": {
        "$ref": "#/components/schemas/ValidationError"
      },
      "type": "array",
      "title": "Detail"
    }
  },
  "type": "object",
  "title": "HTTPValidationError"
}

{
  "properties": {
    "status": {
      "type": "string",
      "title": "Status",
      "description": "Service status"
    },
    "database_connected": {
      "type": "boolean",
      "title": "Database Connected",
      "description": "Database connection status"
    },
    "vector_store_initialized": {
      "type": "boolean",
      "title": "Vector Store Initialized",
      "description": "Vector store initialization status"
    }
  },
  "type": "object",
  "required": [
    "status",
    "database_connected",
    "vector_store_initialized"
  ],
  "title": "HealthResponse"
}

{
  "properties": {
    "key": {
      "type": "string",
      "title": "Key",
      "description": "Metadata key to filter on"
    },
    "value": {
      "title": "Value",
      "description": "Value to filter by"
    },
    "operator": {
      "$ref": "#/components/schemas/MetadataFilterOperator",
      "description": "Comparison operator",
      "default": "=="
    }
  },
  "type": "object",
  "required": [
    "key",
    "value"
  ],
  "title": "MetadataFilter"
}

{
  "type": "string",
  "enum": [
    "==",
    ">",
    "<",
    ">=",
    "<=",
    "!=",
    "in",
    "nin"
  ],
  "title": "MetadataFilterOperator"
}

{
  "properties": {
    "filters": {
      "items": {
        "$ref": "#/components/schemas/MetadataFilter"
      },
      "type": "array",
      "title": "Filters"
    },
    "condition": {
      "type": "string",
      "title": "Condition",
      "description": "Logical condition: 'and' or 'or'",
      "default": "and"
    }
  },
  "type": "object",
  "title": "MetadataFilters"
}

{
  "properties": {
    "node_id": {
      "type": "string",
      "title": "Node Id",
      "description": "Unique node identifier"
    },
    "text": {
      "type": "string",
      "title": "Text",
      "description": "Node text content"
    },
    "score": {
      "type": "number",
      "title": "Score",
      "description": "Similarity score"
    },
    "metadata": {
      "additionalProperties": true,
      "type": "object",
      "title": "Metadata",
      "description": "Node metadata"
    }
  },
  "type": "object",
  "required": [
    "node_id",
    "text",
    "score"
  ],
  "title": "NodeResponse"
}

{
  "properties": {
    "query": {
      "type": "string",
      "title": "Query",
      "description": "Query text"
    },
    "top_k": {
      "type": "integer",
      "maximum": 100,
      "minimum": 1,
      "title": "Top K",
      "description": "Number of documents to retrieve",
      "default": 5
    },
    "filters": {
      "anyOf": [
        {
          "$ref": "#/components/schemas/MetadataFilters"
        },
        {
          "type": "null"
        }
      ],
      "description": "Optional metadata filters"
    }
  },
  "type": "object",
  "required": [
    "query"
  ],
  "title": "QueryRequest"
}

{
  "properties": {
    "response": {
      "type": "string",
      "title": "Response",
      "description": "Synthesized response"
    },
    "source_nodes": {
      "items": {
        "$ref": "#/components/schemas/NodeResponse"
      },
      "type": "array",
      "title": "Source Nodes",
      "description": "Source nodes used"
    },
    "query": {
      "type": "string",
      "title": "Query",
      "description": "Original query"
    }
  },
  "type": "object",
  "required": [
    "response",
    "source_nodes",
    "query"
  ],
  "title": "QueryResponse"
}

{
  "properties": {
    "query": {
      "type": "string",
      "title": "Query",
      "description": "Query text for similarity search"
    },
    "top_k": {
      "type": "integer",
      "maximum": 100,
      "minimum": 1,
      "title": "Top K",
      "description": "Number of top results to return",
      "default": 5
    },
    "mode": {
      "$ref": "#/components/schemas/SearchMode",
      "description": "Search mode: vector, keyword, or hybrid",
      "default": "vector"
    },
    "rerank": {
      "type": "boolean",
      "title": "Rerank",
      "description": "Whether to apply reranking using Cohere Rerank",
      "default": false
    },
    "rerank_top_n": {
      "anyOf": [
        {
          "type": "integer",
          "maximum": 50,
          "minimum": 1
        },
        {
          "type": "null"
        }
      ],
      "title": "Rerank Top N",
      "description": "Number of results after reranking (defaults to top_k)"
    },
    "filters": {
      "anyOf": [
        {
          "$ref": "#/components/schemas/MetadataFilters"
        },
        {
          "type": "null"
        }
      ],
      "description": "Optional metadata filters"
    }
  },
  "type": "object",
  "required": [
    "query"
  ],
  "title": "RetrieveRequest"
}

{
  "properties": {
    "nodes": {
      "items": {
        "$ref": "#/components/schemas/NodeResponse"
      },
      "type": "array",
      "title": "Nodes",
      "description": "Retrieved nodes"
    },
    "query": {
      "type": "string",
      "title": "Query",
      "description": "Original query"
    },
    "total_results": {
      "type": "integer",
      "title": "Total Results",
      "description": "Number of results returned"
    }
  },
  "type": "object",
  "required": [
    "nodes",
    "query",
    "total_results"
  ],
  "title": "RetrieveResponse"
}

{
  "type": "string",
  "enum": [
    "vector",
    "keyword",
    "hybrid"
  ],
  "title": "SearchMode"
}

{
  "properties": {
    "loc": {
      "items": {
        "anyOf": [
          {
            "type": "string"
          },
          {
            "type": "integer"
          }
        ]
      },
      "type": "array",
      "title": "Location"
    },
    "msg": {
      "type": "string",
      "title": "Message"
    },
    "type": {
      "type": "string",
      "title": "Error Type"
    }
  },
  "type": "object",
  "required": [
    "loc",
    "msg",
    "type"
  ],
  "title": "ValidationError"
}