Configuration

BucketDB is designed to be configured completely via the instantiation object.

Because BucketDB is meant for distributed environments (like standard containerized tasks or Fly.io edge nodes), the configuration footprint is kept small, predictable, and fully visible in code.

Instantiation Options

The BucketDb(options) factory accepts a single configuration object.

const { BucketDb } = require('bucket-db');
const { S3Driver } = require('bucket-db/lib/storage');

const db = BucketDb({
  // --- REQUIRED ---
  dbPrefix: process.env.DB_PREFIX || 'prod', // e.g. "tenant_1" or "prod"
  nodeId: parseInt(process.env.NODE_ID || '0', 10),

  // --- OPTIONAL / DRIVER OVERRIDES ---
  // Defaults to a built-in S3Driver if BUCKETDB_S3_BUCKET is set in environment.
  driver: S3Driver({
    bucket: 'my-bucket',
    region: 'us-east-1',
    credentials: {
      accessKeyId: process.env.DB_ACCESS_KEY_ID,
      secretAccessKey: process.env.DB_SECRET_ACCESS_KEY
    }
  }),
  
  // --- ADVANCED TUNING ---
  writeForwardInterval: 5000,
  blockSizeBytes: 65536,
  readStalenessMs: 1000,
  logLevel: 'info',
  logger: null,
  maxMemoryBytes: 104857600, // 100MB
  cacheDriver: null,
  cacheBlockSize: 65536,
  
  // --- PREFIXES (Internal Paths) ---
  dataPrefix: 'data',
  committedPrefix: 'committed',
  blobPrefix: 'blobs',
  indexPrefix: 'indexes'
});

Required Fields

• dbPrefix (String): A root folder inside the storage driver. This allows multiple completely separate databases (or tenants) to safely share the same driver or S3 bucket.
• nodeId (Number): An integer representing this specific instance in your cluster (0-255). It is used to generate universally unique MISC values to prevent conflicts in the distributed timesharing mechanism.

Driver Configuration

By default, if the environment variable BUCKETDB_S3_BUCKET is set, BucketDB will dynamically require @aws-sdk/client-s3 and wire up the default S3Driver.

It is recommended to explicitly inject an agnostic StorageDriver compliant object to options.driver (such as S3Driver or MemoryDriver). See Storage Drivers for detailed examples.

Advanced Tuning

These defaults are calibrated for the "boring web app" sweet spot (low throughput, high consistency).

• writeForwardInterval (Number, Default: 5000): The duration in milliseconds between write authority window checks. A smaller window rotates leadership faster but increases S3 API calls as nodes constantly poll Block 0. A larger window is more cost-effective but increases the latency for background commits to merge into immutable blocks.
• blockSizeBytes (Number, Default: 65536 / 64KB): The target size for immutable data blocks. Smaller blocks mean faster reads for highly specific queries but result in many more files (and S3 PUTs) during write-heavy workloads.
• readStalenessMs (Number, Default: 1000): The maximum acceptable staleness for query reads. Determines how often the database fetches the latest write-ahead logs (WALs) from storage. See Consistency & Caching for tuning implications.
• logLevel (String, Default: 'info'): Output verbosity. Accepts 'debug', 'info', 'warn', 'error', or 'silent'. Setting to 'silent' will mute all logs.
• logger (Object, Optional): A custom logger instance (e.g., Pino or Winston) implementing .debug(), .info(), .warn(), and .error() methods.
• maxMemoryBytes (Number, Default: 104857600 / 100MB): The maximum amount of memory the internal SplitPool orchestrator is allowed to use. Balances the mandatory WAL cache against the optional block cache.
• cacheDriver (Object, Optional): A custom cache driver (e.g., Redis or an external memory store) for caching S3 blocks. Must implement get, set, delete, clear, and getUsage. Defaults to an internal in-memory driver.
• cacheBlockSize (Number, Default: 65536): The estimated block size used by the SplitPool to calculate how many blocks can fit into the remaining memory budget.

Internal Path Prefixes

BucketDB partitions different types of data under the dbPrefix. You can rename these if they conflict with an existing bucket structure, but the defaults are recommended.

• dataPrefix (Default: data): Where immutable Row/Heap blocks are stored.
• committedPrefix (Default: committed): The temporary Write-Forward log.
• blobPrefix (Default: blobs): Managed, unstructured object storage (e.g., PDFs, images).
• indexPrefix (Default: indexes): Persistent CoW B+Tree index blocks.