Getting Started

Table of contents

  1. Prerequisites
  2. Installation Options
    1. Option 1: Pre-compiled Binaries (Recommended)
    2. Option 2: Building from Source
  3. Configuration
  4. Running the Server
  5. Verification
  6. Next Steps

Prerequisites

Before installing Batch-GPT, ensure you have:

  • Go 1.23.0 or later
  • Docker and Docker Compose
  • An OpenAI API key
  • MongoDB (included in docker-compose setup)

Installation Options

  1. Download the latest release for your system from the Releases page

  2. Extract the archive: 
    tar xzf batch-gpt_[version]_[os]_[arch].tar.gz
  3. Set up MongoDB: 
    cd local/mongo
docker-compose up -d

Option 2: Building from Source

  1. Clone the repository: 
    git clone https://github.com/tanmay17061/batch-gpt.git
cd batch-gpt
  2. Set up MongoDB: 
    cd local/mongo
docker-compose up -d
cd ../..
  3. Build the server: 
    go build -o batch-gpt server/main.go

Configuration

Set required environment variables:

export OPENAI_API_KEY=your_openai_api_key
export COLLATE_BATCHES_FOR_DURATION_IN_MS=5000
export MONGO_HOST=localhost
export MONGO_PORT=27017
export MONGO_USER=admin
export MONGO_PASSWORD=password
export MONGO_DATABASE=batchgpt

Running the Server

Start the server:

./batch-gpt

The server will start on http://localhost:8080.

Verification

Test the installation using the provided Python client:

cd test-python-client
pip install -r requirements.txt
python client.py --api chat_completions --content "Hello, World!"

Next Steps