IsleDB / Docs / Get started with IsleDB

Get started with IsleDB

IsleDB keeps a key-value database in your object storage bucket. Run it as HTTP services and use it from any language, or embed it in your Go program.

The first part starts a database on your machine with Docker and uses it over HTTP. The second part does the same from Go. Each takes a few minutes.

Run it as services

A database is served by three services: one writer takes writes, any number of readers serve reads, and one maintenance service keeps the database compact. They share nothing but the bucket.

Start a database on your machine

You need Docker. Download the example Compose file and start it:

curl -O https://raw.githubusercontent.com/ankur-anand/isledb/main/examples/services/compose.yaml
docker compose up -d

It starts four containers:

Write and read

Write a key to the writer. The answer comes once the write is durable in the bucket:

curl -X PUT --data 'hello' localhost:8070/v1/kv/greeting
{"write_id":"1.1","revision":"2"}

Read it from the reader:

curl -i localhost:8080/v1/kv/greeting
HTTP/1.1 200 OK
…
Isledb-Revision: 2

hello

A reader picks up new writes within about a second. Its Isledb-Revision header says how far it has read: a read includes every write whose revision is at or below it. Until then, a new key answers 404.

List keys by prefix with a scan:

curl 'localhost:8080/v1/kv?prefix=gree'
{"items":[{"key":"greeting","value":"hello"}]}

Keys and values can be any bytes. A single read returns the value as it is; in a scan, keys and values that are not valid UTF-8 come back as key_base64 and value_base64.

Follow changes

The example's writer turns on the change feed, so the reader also serves every committed change, in order:

curl 'localhost:8080/v1/changes?from=oldest'
{"changes":[{"sequence":1,"revision":2,"op":"put","key":"greeting","value":"hello"}],
 "next":"cf1_AAAAAAAAAAIAAAAAAAAAAA","caught_up":true}

Save next, and poll with from= that cursor to get only newer changes. Each consumer keeps its own cursor, and any reader accepts it.

When you are done, stop everything and delete its data:

docker compose down -v

Use your bucket

Download isledb for Linux or macOS from the latest release, or use the ghcr.io/ankur-anand/isledb image. Point each service at your bucket and a database prefix, and give it a file of bearer tokens, one per line:

isledb writer \
  --bucket-url 's3://my-bucket?region=us-east-1' \
  --prefix prod/accounts \
  --auth-token-file /etc/isledb/tokens

isledb reader \
  --bucket-url 's3://my-bucket?region=us-east-1' \
  --prefix prod/accounts \
  --cache-dir /var/cache/isledb \
  --auth-token-file /etc/isledb/tokens

isledb maintenance \
  --bucket-url 's3://my-bucket?region=us-east-1' \
  --prefix prod/accounts

Requests then send one of the tokens:

curl -H "Authorization: Bearer $TOKEN" https://isledb-reader.example.com/v1/kv/greeting

Credentials come from the provider's usual environment variables or instance roles. Run one writer and one maintenance service per database, and as many readers as you need.

Embed it in Go

In a Go program, the same database is a library: no services, no network hop. Use Go 1.25 or newer:

mkdir isledb-start && cd isledb-start
go mod init example.com/isledb-start
go get github.com/ankur-anand/isledb

Save this as main.go:

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/ankur-anand/isledb"
)

func main() {
	ctx := context.Background()

	// mem:// keeps the database in memory, for a first run.
	db, err := isledb.Open(ctx, "mem://", isledb.DBOptions{Prefix: "demo"})
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	w, err := db.OpenWriter(ctx, isledb.DefaultWriterOptions())
	if err != nil {
		log.Fatal(err)
	}
	defer w.Close(ctx)

	if _, err := w.Put(ctx, []byte("greeting"), []byte("hello")); err != nil {
		log.Fatal(err)
	}
	if err := w.Flush(ctx); err != nil { // durable once Flush returns
		log.Fatal(err)
	}

	r, err := db.OpenReader(ctx, isledb.DefaultReaderOpenOptions("./reader-cache"))
	if err != nil {
		log.Fatal(err)
	}
	defer r.Close()

	value, found, err := r.Get(ctx, []byte("greeting"))
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(string(value), found)
}

Run it:

go run .
hello true

To keep the data, replace mem:// with your bucket URL, from the table above. The writer, readers and maintenance can then run in separate processes, each opening the same bucket and prefix.

Next steps

• How IsleDB works The writer, readers, maintenance, revisions, and the change feed.
• Deploy IsleDB What to run, how many of each, and a production checklist.
• Writer HTTP API Batches, deletes, TTLs, and waiting for a write.
• Reader HTTP API Reads, scans, freshness, and the change feed.
• Shared read cache Serve many readers in a region from one copy of each read.
• Go API Writers, readers, snapshots, the change feed, and maintenance.
• Examples Complete programs, including separate services on MinIO.