Storage Drivers

BucketDB isolates its core logic from the physical storage layer using the StorageDriver interface. This ensures that the engine can operate against any storage backend that provides basic Key/Value guarantees without modifying the underlying database code.

The StorageDriver Interface

Any valid Driver passed to options.driver must implement four basic methods returning Promises.

const CustomDriver = () => ({
  /**
   * get(key)
   * Resolves: { buffer: Uint8Array, eTag?: string }
   * Rejects: Error with `.name === 'NoSuchKey'` if not found.
   */
  get: async (key) => { ... },

  /**
   * put(key, buffer, options?)
   * options: { ifMatch?: string } (ETag conditional update)
   * Resolves: { eTag?: string }
   * Rejects: Error with `.name === 'PreconditionFailed'` if condition fails.
   */
  put: async (key, buffer, options) => { ... },

  /**
   * delete(key)
   * Resolves: void (idempotent, no error if missing)
   */
  delete: async (key) => { ... },

  /**
   * list(prefix, options?)
   * options: { continuationToken?: string }
   * Resolves: { keys: Array<string>, IsTruncated: boolean, NextContinuationToken?: string }
   */
  list: async (prefix, options) => { ... }
});

Built-In Drivers

BucketDB ships with three reference drivers.

1. S3Driver

The default, production-grade driver. It wraps the AWS SDK v3 @aws-sdk/client-s3.

Because BucketDB is explicitly designed to minimize vendor lock-in, the S3Driver only relies on the most basic S3 APIs (GetObject, PutObject, DeleteObject, ListObjectsV2). This means it works perfectly with:

• Amazon S3
• Cloudflare R2
• Tigris Data
• Backblaze B2
• MinIO (Localstack)

Usage:

// Usually, BucketDB configures this automatically if you provide `endpoint`, 
// `bucket`, `accessKeyId`, and `secretAccessKey`. 
// However, you can instantiate it manually if you need a custom SDK setup:

const { S3Client } = require('@aws-sdk/client-s3');
const { S3Driver } = require('@sullux/bucketdb/drivers');

const customAwsClient = new S3Client({ region: 'eu-west-1' });
const driver = S3Driver({ s3Client: customAwsClient, bucket: 'my-bucket' });

const db = await createDatabase({
  driver,
  dbPrefix: 'prod',
  nodeId: 0,
  ringSize: 1
});

2. MemoryDriver

A pure JavaScript Map-backed store. It holds all keys and buffers in memory.

Use Cases:

• Unit testing your application's database logic without mocking S3 or running Localstack.
• Extreme performance scenarios where data persistence is not strictly required.
• CI/CD pipelines.

Usage:

const { MemoryDriver } = require('@sullux/bucketdb/drivers');

const driver = MemoryDriver();

const db = await createDatabase({
  driver,
  dbPrefix: 'test-db',
  nodeId: 0,
  ringSize: 1
});

3. FileDriver

A filesystem-backed driver. It maps S3 keys directly to directories and files on disk.

Use Cases:

• Local development when you want data to persist across restarts but don't want the overhead of running MinIO.
• Single-node IoT devices or edge servers with an attached block volume.

Usage:

const { FileDriver } = require('@sullux/bucketdb/drivers');
const path = require('node:path');

const driver = FileDriver({ 
  baseDir: path.join(process.cwd(), '.local-db') 
});

const db = await createDatabase({
  driver,
  dbPrefix: 'dev-db',
  nodeId: 0,
  ringSize: 1
});

Example: Building a Custom Redis Driver

If you wanted to back BucketDB with a Redis cluster instead of S3, you could easily implement the StorageDriver interface.

const Redis = require('ioredis');

const RedisDriver = (connectionString) => {
  const redis = new Redis(connectionString);

  return {
    get: async (key) => {
      const data = await redis.getBuffer(key);
      if (!data) {
        const err = new Error('Not Found');
        err.name = 'NoSuchKey';
        throw err;
      }
      return { buffer: data, eTag: 'redis-etag' };
    },

    put: async (key, buffer, options = {}) => {
      // NOTE: For true cluster safety, you would need to implement 
      // WATCH/MULTI/EXEC for the `ifMatch` ETag conditional logic here.
      await redis.set(key, buffer);
      return { eTag: 'redis-etag' };
    },

    delete: async (key) => {
      await redis.del(key);
    },

    list: async (prefix, options = {}) => {
      // Redis SCAN implementation
      let cursor = options.continuationToken || '0';
      const [nextCursor, keys] = await redis.scan(
        cursor, 
        'MATCH', 
        `${prefix}*`, 
        'COUNT', 
        1000
      );
      
      return {
        keys,
        IsTruncated: nextCursor !== '0',
        NextContinuationToken: nextCursor !== '0' ? nextCursor : undefined
      };
    }
  };
};

module.exports = { RedisDriver };