IsleDB / Docs / Go API reference

Go API reference

Embed IsleDB in a Go program: open a database on your bucket, write and read keys, follow the change feed, and run maintenance.

This guide covers the public API most applications use, with the behavior that affects correctness: when writes are durable, when readers see them, how long a read view lasts, how change-feed cursors work, and how maintenance runs in its own process.

import "github.com/ankur-anand/isledb"

Overview

An IsleDB database is one object-store bucket (or container) plus a prefix. A DB value is the local runtime for that prefix, and it hands out four kinds of handle:

Writer and maintenance ownership are also fenced through the object store, so two processes can never act as the same owner at once. Readers and change readers scale freely: open the same prefix from as many processes as you like.

How a write becomes visible:

Put / Delete
    -> buffered in the writer
    -> Flush, background flush, Drain or Close
    -> committed to the manifest
    -> visible to a newly opened or refreshed reader

Quick start

This example turns off timed background flushing, so the explicit Flush is the moment the write becomes durable and visible.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/ankur-anand/isledb"
)

func main() {
    if err := run(context.Background()); err != nil {
        log.Fatal(err)
    }
}

func run(ctx context.Context) error {
    db, err := isledb.Open(
        ctx,
        "s3://my-bucket?region=us-east-1",
        isledb.DBOptions{Prefix: "accounts"},
    )
    if err != nil {
        return err
    }
    defer func() { _ = db.Close() }()

    writerOptions := isledb.DefaultWriterOptions()
    writerOptions.Flush.Interval = 0

    writer, err := db.OpenWriter(ctx, writerOptions)
    if err != nil {
        return err
    }
    if _, err := writer.Put(ctx, []byte("user:1"), []byte("Ankur")); err != nil {
        return err
    }
    if err := writer.Flush(ctx); err != nil {
        return err
    }
    if err := writer.Close(ctx); err != nil {
        return err
    }

    reader, err := db.OpenReader(
        ctx,
        isledb.DefaultReaderOpenOptions("./isledb-cache"),
    )
    if err != nil {
        return err
    }
    defer func() { _ = reader.Close() }()

    value, found, err := reader.Get(ctx, []byte("user:1"))
    if err != nil {
        return err
    }
    if found {
        fmt.Printf("user:1 = %s\n", value)
    }
    return nil
}

Open takes a Go Cloud bucket URL. The production schemes are s3://, gs:// and azblob:// (S3-compatible stores work through s3://). Credentials come from the matching Go Cloud driver and cloud SDK.

file:// and mem:// buckets are for development and tests. They behave correctly within one process, but across processes their conditional writes are not atomic: never point two processes at the same file:// directory, and never use either in production.

Open and close a database

func Open(ctx context.Context, bucketURL string, opts DBOptions) (*DB, error)
func OpenBucket(ctx context.Context, bucket *blob.Bucket, bucketName string, opts DBOptions) (*DB, error)

type DBOptions struct {
    Prefix     string
    ChangeFeed *ChangeFeedOptions
    SSTOutput  SSTOutputOptions
    Policy     StorePolicy
}

type StorePolicy struct {
    MaxPinnedViewAge time.Duration
}

• Open creates the bucket connection and DB.Close closes it. OpenBucket borrows a *blob.Bucket you already have; closing it stays your job.
• Prefix is the database's root inside the bucket. Give each database its own prefix.
• ChangeFeed turns on the change feed.
• SSTOutput sets how new SST files are encoded; see SST output policy.
• MaxPinnedViewAge is the longest a loaded manifest view stays usable. Zero selects DefaultMaxPinnedViewAge, one hour. The first writer stores this policy; a later writer with a different value fails to open with ErrStorePolicyMismatch. Readers and the deletion of retired objects (including change-feed batches) both use it, so an old view never refers to an object that has been deleted.

func (db *DB) OpenWriter(ctx context.Context, opts WriterOptions) (*Writer, error)
func (db *DB) OpenReader(ctx context.Context, opts ReaderOpenOptions) (*Reader, error)
func (db *DB) OpenChangeReader(ctx context.Context, opts ...ChangeReaderOptions) (*ChangeReader, error)
func (db *DB) OpenMaintenance(ctx context.Context, opts MaintenanceOptions) (*Maintenance, error)
func (db *DB) Close() error

DB.Close closes any handles still open on that DB. It is process-level cleanup: close your handles yourself when their errors matter. An open writer is closed with a fixed 30-second deadline and no retry, so if writes may be in flight, run Drain and Writer.Close first; see Shutting down.

Write data

func (w *Writer) Put(ctx context.Context, key, value []byte) (uint64, error)
func (w *Writer) PutWithTTL(ctx context.Context, key, value []byte, ttl time.Duration) (uint64, error)
func (w *Writer) Delete(ctx context.Context, key []byte) (uint64, error)
func (w *Writer) Apply(ctx context.Context, b *Batch) (uint64, error)
func (w *Writer) Flush(ctx context.Context) error
func (w *Writer) WaitCommitted(ctx context.Context, seq uint64) (ViewPosition, error)
func (w *Writer) CommittedSequence() uint64
func (w *Writer) AcceptedSequence() uint64
func (w *Writer) Epoch() uint64
func (w *Writer) CommitStatus(epoch, seq uint64) (CommitStatus, error)
func (w *Writer) State() WriterState
func (w *Writer) StopWrites()
func (w *Writer) Drain(ctx context.Context) error
func (w *Writer) Close(ctx context.Context) error

Every method is safe to call from any goroutine.

Sequences. Each write gets the next sequence number, in the order writes are accepted. Sequences continue across writers from the highest committed one, and a change-feed consumer sees the same number as Change.Sequence. A writer taking over reuses the sequences its predecessor accepted but did not commit, so a sequence names a write only together with the writer's Epoch.

Validation. An empty or oversized key, an oversized value, or a negative TTL returns an error wrapping ErrInvalidMutation, so you know not to retry it. A TTL of zero means no expiration.

TTLs. Readers hide expired values. Expiry does not delete anything immediately; compaction removes expired data later.

Write several keys atomically

Put and Delete each commit on their own: between two of them the writer can start a new memtable, so a reader may see the first without the second. Apply buffers a whole batch under one lock and in one memtable, so readers see all of it or none of it, and the change feed publishes it all at once, in one change batch. A feed consumer that reads with a limit may still receive the batch over several pages; the feed does not mark where a batch ends.

b := isledb.NewBatch() // or: var b isledb.Batch
b.Put([]byte("order/42"), order)
b.Put([]byte("stock/7"), stock)
b.Delete([]byte("cart/42"))

last, err := writer.Apply(ctx, b)
if err != nil {
    return err // nothing was applied; see below
}
position, err := writer.WaitCommitted(ctx, last) // durable, all of it

• It does not flush. Like Put, the batch commits with the next background flush, Flush, Drain or Close; wait with WaitCommitted on the last sequence. Mutation i has sequence last - b.Len() + 1 + i.
• Whole or nothing. On any error nothing was applied and the same batch can be retried: ErrInvalidMutation names the first bad mutation (or an empty batch), ErrBackpressure means wait for commits, and ErrBatchTooLarge means it does not fit one memtable: split it.
• Not a transaction. There are no reads or conditions, and a later write can overwrite any key in it. A key written twice in one batch keeps its last write.
• Size. A batch must fit in Memtable.TargetBytes, or in the largest single write if that is bigger, so a one-entry batch takes anything Put does. Each entry's memtable overhead counts, so many tiny entries reach the limit sooner than their bytes suggest.
• The Batch. Adding copies keys and values; Grow(n) makes room for n bytes of them first, when you know the total. Apply does not change the batch or keep its memory, so drop it or Reset it for reuse. A Batch is for one goroutine.

Is my write durable?

A write is durable once it is committed to object storage. To acknowledge a client only then:

seq, err := writer.Put(ctx, key, value)
if err != nil {
    return err
}
position, err := writer.WaitCommitted(ctx, seq)
if err != nil {
    return err // see the table below
}
// The write is in object storage, visible to any reader at or past position.

WaitCommitted(ctx, seq) waits until that write, and every one before it, is committed. It does not flush by itself: writes commit with the next background flush, Flush, Drain or Close, so many waiters share one commit, and with healthy storage the wait is at most one flush interval. Without a flush interval, call Flush.

Treat an unknown write as unknown, not lost, and make retries idempotent (for example, Put the same key and value again).

Read your own writes. The position WaitCommitted returns is a manifest log position: a Reader whose ViewPosition() is at or past it sees the write. Hand it to the client with the write's acknowledgement; a read that carries it is served once the reader's view has caught up, after a Refresh or the read cache's next ring. Unlike sequences, positions are never reused by a later writer.

Did it commit after a failover?

A client whose wait ended without an answer can ask the writer that holds the database now. Keep the epoch with each sequence handed out:

seq, err := writer.Put(ctx, key, value)
token := WriteToken{Epoch: writer.Epoch(), Seq: seq} // your type

// Later, on whichever writer is current:
status, err := current.CommitStatus(token.Epoch, token.Seq)

Each writer's fence records where the writer before it stopped, and where the one before that did, so CommitStatus answers exactly for this writer and the two before it:

CommitStatus does no I/O, so an HTTP endpoint on the writer can answer from memory. A fenced writer learns where its commits ended from the fence that replaced it, a commit whose response was lost included, and WaitCommitted returns nil for those writes rather than ErrFenced. Where every write's outcome must stay knowable across any number of takeovers, store a request ID in the value written, and read it back after Refresh.

Writer lifecycle

State().Status reports where the writer is:

Status only moves forward. WriterStopped is one-way: there is no reopening. Storage errors never end a writer; only Close and losing the fence do.

State returns everything in one consistent snapshot:

type WriterState struct {
    Epoch             uint64       // the writer's fence epoch, as Epoch returns
    Status            WriterStatus // WriterOpen, WriterStopped, WriterClosed, WriterFenced
    Accepted          uint64       // highest sequence handed out
    Committed         uint64       // highest sequence in object storage
    PendingMemtables  int          // memtables not yet committed, the active one included
    PendingBytes      int64        // their approximate in-memory size, before compression
    OldestUncommitted time.Time    // when the oldest uncommitted write arrived; zero if none
}

Use it for readiness probes (stop routing writes once the status is not WriterOpen), to show drain progress, and to slow producers before PendingMemtables reaches MaxPendingMemtables. It can be out of date as soon as it returns.

Failures and retries

One committer. A single goroutine inside the writer does every commit: background flushes, Flush, Drain, Close, and applying maintenance commands. Flush asks it for a commit and waits. Your context bounds only the wait, not the commit: a Flush that times out or is cancelled returns at once, and the commit carries on.

A failed commit is retried, never dropped. The writes stay queued. Background flushes retry with a delay that doubles from the flush interval up to 30 seconds. A failed Flush returns its error; calling it again retries at once. Every attempt first checks whether the previous one landed before its response was lost, so a commit never lands twice.

Every attempt has a deadline of 30 seconds plus one second per MiB to upload, so a hung storage request costs one attempt, not the writer. Each attempt in a row that times out doubles the next one's deadline, up to 10 minutes, so a slow link still gets a commit through; a successful commit resets it. A timed-out attempt is reported, and returned by a Flush waiting on it, as ErrCommitTimeout. That error does not wrap context.DeadlineExceeded: it is the writer's deadline, not yours.

Backpressure. While commits keep failing, writes pile up in memory. Once MaxPendingMemtables frozen memtables are waiting, a write that needs another one returns ErrBackpressure without being accepted. Retry after a delay, or call Flush.

Reporting. OnFlushError receives every commit and maintenance failure, such as an expired credential, an unavailable bucket or a timed-out attempt: the first failure of a run, then at most once a minute while failures continue. Commit failures and maintenance failures are separate runs, and each ends, with a log line, at its next success. Without OnFlushError, the writer logs a warning instead. Your own cancellation or deadline in Flush is returned to you, not reported.

OnFlushError runs on its own goroutine. It may call Close, and it may run after Close returns, so it must not use anything you tear down on close.

Some failures need a person. Missing permissions, a deleted bucket, a manifest entry the storage rejects, or ErrCommitIndeterminate will not fix themselves. The writer still never gives up: it retries every 30 seconds, and Put keeps succeeding until ErrBackpressure. Alert on OnFlushError, or on isledb_writer_oldest_uncommitted_timestamp_seconds (see metrics), fix the cause, or close the writer.

Maintenance commands are applied by the same committer, in order with data commits. A failure to apply one is reported and retried; it never holds up data commits. See Run maintenance.

Shutting down

Drain stops new writes and commits everything accepted, retrying while there is time; Close then finishes the writer:

func shutdownWriter(ctx context.Context, w *isledb.Writer) error {
    // 1. Take no more writes, and commit every accepted one. Put and Delete
    //    now return ErrWritesStopped; failed attempts are retried until ctx.
    drainErr := w.Drain(ctx)

    // 2. Finish the writer: one attempt, then everything it runs stops.
    closeCtx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
    defer cancel()
    return errors.Join(drainErr, w.Close(closeCtx))
}

• Spend most of the shutdown window in Drain. It returns nil once every write the writer accepted is in object storage; ErrFenced at once if another writer took over; or the context's error, with the last commit failure, when time runs out. It works without a flush interval.
• Rolling your own: Drain is StopWrites plus a retry loop. Call StopWrites first, then wait for WaitCommitted(ctx, w.AcceptedSequence()), calling Flush if there is no flush interval. Without StopWrites, other goroutines can keep calling Put while you flush, and Flush only covers writes accepted before it was called.
• Give Close its own deadline, long enough for one commit attempt (30 seconds plus one second per MiB pending). After a successful Drain it has nothing left to commit.
• Close always finishes the writer, whether or not its attempt succeeds; after it returns, nothing of the writer is running. It returns nil when everything is committed. Otherwise its error names the sequence range not known to be committed: those writes are unknown, not confirmed lost, and the range tells you what to check or replay.
• Do not pass Close a context that has already ended: it then finishes the writer without attempting a commit.

DB.Close is cleanup, not the durability step. It closes a writer still open with a fixed 30-second deadline and no retry, and returns the writer's error. That is enough when nothing is pending; otherwise call Drain and Writer.Close yourself first.

To keep a crash cheap, set Flush.Interval so at most one interval of writes is unflushed.

Writer options

type WriterOptions struct {
    OwnerID      string
    Memtable     WriterMemtableOptions
    Flush        WriterFlushOptions
    Maintenance  WriterMaintenanceOptions
    Values       ValueOptions
    OnFlushError func(error)
    Metrics      *WriterMetrics
}

type WriterMemtableOptions struct {
    TargetBytes         int64
    MaxPendingMemtables int
}

type WriterFlushOptions struct {
    Interval time.Duration
}

type WriterMaintenanceOptions struct {
    PollInterval time.Duration
}

type ValueOptions struct {
    MaxKeyBytes   int
    MaxValueBytes int64
}

func DefaultWriterOptions() WriterOptions

Flush.Interval is the one option where zero does not mean "default": it disables background flushing. Start from DefaultWriterOptions when you want every default.

Read data

type ReaderOpenOptions struct {
    CacheDir       string
    DiskCacheSize  int64
    BlockCacheSize int64
    BloomCacheSize int64
    Views          ReaderViewPolicy
    ReadCache      ReadCacheOptions
    Metrics        *ReaderMetrics
}

type ReaderViewPolicy struct {
    RefreshAfter time.Duration
    Manual       bool
    MaxLag       time.Duration
}

type ReadCacheOptions struct {
    URL           string
    Token         string
    HTTPClient    *http.Client
    Timeout       time.Duration
    MaxCurrentAge time.Duration
}

func DefaultReaderOpenOptions(cacheDir string) ReaderOpenOptions

CacheDir is required, and only one live reader process may use it. Zero sizes select the defaults; negative sizes are rejected.

With a read cache, every read of an SST, a manifest page or a manifest snapshot is sent to it first: a read the cache cannot answer, for any reason (down, slow, refused, missing object), is read from storage instead, and everything read is verified as it is from storage, so the cache can make reads slower but never wrong. A scan's reads past the first 64 KiB of each SST ask the cache not to keep what it reads. A cache that stops answering costs a few reads one timeout each, then is bypassed: after 3 timeouts or connection failures in a row, reads go straight to storage for 5 seconds, then one read tries the cache again, and the first answer from it ends the bypass. Refusals and size mismatches do not cause a bypass; the cache is answering, and the next read would fail the same way.

Damaged bytes from the cache do not fail reads. A manifest page or snapshot, a small SST read whole, or a Bloom filter that fails its checksum is read again from storage. Any other part of an SST is checked later, by its block checksums; when that finds damage, the reader reads that SST from storage for the next ten minutes, and its retry succeeds.

ReaderMetrics.ReadCacheReads counts reads the cache answered and ReadCacheFallbacks those that went to storage, by reason: timeout, unavailable, not_found, size_mismatch, refused, invalid, bypassed while the cache is skipped, checksum_mismatch for bytes from the cache that failed their checksum, or after_damage for reads of an SST found damaged recently. A steady count of the last two points at damaged chunks on the cache's disk. ReadCacheCurrent counts refreshes that asked the cache for CURRENT, by outcome: relayed, or too_old, behind or failed, which read storage instead; ReadCacheCurrentAge is the age of the last copy the cache gave; ReadCacheDoorbells counts long-poll answers: rang, quiet or error. A refused read or a size mismatch points at a misconfiguration and is also logged, at most once a minute; the start and end of a bypass are logged too. A bad URL fails OpenReader with ErrInvalidReaderOptions.

Refreshes through the cache. A refresh reads CURRENT from the cache's copy, which the cache reads from storage twice a second. The copy comes with its age, and the view's lifetime is counted from when the cache read it from storage, never from when the reader got it. A copy older than MaxCurrentAge, from a cache that cannot reach storage, or behind the view the reader already has, is not taken: the refresh reads storage instead. Refresh always reads storage, so it sees every commit made before it was called. Opening a reader reads storage too. Because the age is subtracted, a reader that refreshes through the cache shows a view loaded a little earlier, by the copy's age, than one that reads storage: view_loaded_timestamp sits behind on cached readers by the copy's age, normally under a second, by design.

The reader also holds a long poll on the cache, which answers as soon as the cache sees a new commit; the reader then refreshes as RequestRefresh does. New commits reach readers within about half a second, with RefreshAfter left as a safety net that can be set long. If the long poll fails, the reader retries it after a random wait of up to a few seconds, growing to 30 seconds while it keeps failing; refreshes on RefreshAfter go on regardless.

func (r *Reader) Get(ctx context.Context, key []byte) ([]byte, bool, error)
func (r *Reader) Scan(ctx context.Context, minKey, maxKey []byte) ([]KV, error)
func (r *Reader) ScanLimit(ctx context.Context, minKey, maxKey []byte, limit int) ([]KV, error)
func (r *Reader) NewIterator(ctx context.Context, opts IteratorOptions) (*Iterator, error)
func (r *Reader) Snapshot(ctx context.Context) (*Snapshot, error)
func (r *Reader) BootstrapView(ctx context.Context) (*BootstrapView, error)
func (r *Reader) ChangeHead(ctx context.Context) (ChangeCursor, error)
func (r *Reader) Prefetch(ctx context.Context, opts PrefetchOptions) (PrefetchStats, error)
func (r *Reader) PruneDiskCache(ctx context.Context) (int, error)
func (r *Reader) Refresh(ctx context.Context) error
func (r *Reader) RequestRefresh()
func (r *Reader) ViewPosition() ViewPosition
func (r *Reader) NextView(ctx context.Context) (*NextView, error)
func (r *Reader) DiskCacheStats() DiskCacheStats
func (r *Reader) BlockCacheStats() CacheStats
func (r *Reader) OpenSSTCacheStats() CacheStats
func (r *Reader) BloomCacheStats() CacheStats
func (r *Reader) ManifestPageCacheStats() CacheStats
func (r *Reader) Close() error

type KV struct {
    Key   []byte
    Value []byte
}

• Get returns found == false for a key that is missing, deleted or expired.
• Every range is half-open, [minKey, maxKey): Scan, ScanLimit, IteratorOptions, Snapshot.ScanLimit, snapshot iterators and PrefetchOptions.Range. A nil or empty bound leaves that side open. Because the upper bound is exclusive, PrefixRange(prefix) can be passed to any range API as is.
• Scan builds the whole result in memory. ScanLimit stops after limit results; zero or a negative limit means no limit. For large ranges, use an iterator.
• Refresh loads the latest commits now; see Freshness and outages.
• Prefetch downloads only into the disk cache's free space, so it never evicts what is already cached. SSTs that do not fit are counted in PrefetchStats.SkippedSSTs and load on demand.
• When a refresh retires SSTs, after a compaction, the reader deletes them from its disk cache in the background, at most 1,000 files a second so the deletes do not slow lookups, keeping the space for data a view can read. isledb_reader_dead_ssts_pending shows how many are still queued.

Iterate without loading the whole range

iter, err := reader.NewIterator(ctx, isledb.IteratorOptions{
    MinKey: []byte("user:"), // inclusive; nil means the beginning
    MaxKey: []byte("user;"), // exclusive; nil means the end
})
if err != nil {
    return err
}
defer func() { _ = iter.Close() }()

for iter.Next() {
    key, value := iter.Key(), iter.Value()
    _, _ = key, value
}
if err := iter.Err(); err != nil {
    return err
}

func (it *Iterator) Next() bool
func (it *Iterator) SeekGE(target []byte) bool
func (it *Iterator) Key() []byte
func (it *Iterator) Value() []byte
func (it *Iterator) Valid() bool
func (it *Iterator) Err() error
func (it *Iterator) Close() error

Consistent snapshots

A snapshot pins one loaded view. It does not move when its reader refreshes.

type Version struct { /* opaque */ }

func (v Version) String() string
func (v Version) IsZero() bool

func (s *Snapshot) Version() Version
func (s *Snapshot) ViewPosition() ViewPosition
func (s *Snapshot) Get(ctx context.Context, key []byte) ([]byte, bool, error)
func (s *Snapshot) ScanLimit(ctx context.Context, minKey, maxKey []byte, limit int) ([]KV, error)
func (s *Snapshot) NewIterator(ctx context.Context, opts IteratorOptions) (*Iterator, error)
func (s *Snapshot) Close() error

ViewPosition is the position of the view the snapshot reads, fixed when it was taken, unlike Reader.ViewPosition, which moves with refreshes. Report it beside a read to say exactly which view answered: a write whose WaitCommitted position is at or below it is included.

A snapshot and its iterators share the view's fixed deadline (MaxPinnedViewAge after it was loaded); opening another handle does not extend it. Past it, operations return ErrSnapshotExpired, ErrIteratorExpired or ErrReadViewExpired: take a new snapshot rather than retrying. Closing the reader invalidates its snapshots and iterators.

A snapshot of an older view keeps reading correctly after the reader moves on: object storage keeps its SSTs for MaxPinnedViewAge. Once a refresh retires those SSTs, the reader deletes them from its disk cache, so the snapshot may fetch them again from object storage. A long BootstrapView load running across a compaction is where that shows.

Load state, then follow the change feed

BootstrapView gives you a snapshot and the exact change-feed position that follows it, both from the same manifest:

type BootstrapView struct {
    Snapshot *Snapshot
    Cursor   ChangeCursor
    Version  Version
}

The snapshot holds every write committed before Cursor; Cursor is the first feed position the snapshot does not contain. Version matches Snapshot.Version() and is handy in checkpoint metadata. Build a local copy from the snapshot, save Cursor with it, and then consume only later changes:

view, err := reader.BootstrapView(ctx)
if err != nil {
    return err
}
defer view.Snapshot.Close()

iterator, err := view.Snapshot.NewIterator(ctx, isledb.IteratorOptions{})
if err != nil {
    return err
}
defer iterator.Close()

for iterator.Next() {
    if err := materialize(iterator.Key(), iterator.Value()); err != nil {
        return err
    }
}
if err := iterator.Err(); err != nil {
    return err
}

if err := saveCheckpointCursor(view.Cursor.String()); err != nil {
    return err
}

• Do not build this pair from Snapshot() and ChangeReader.Bounds() separately: a commit landing between the two calls gives a cursor newer than the snapshot, and you would skip that change.
• The change feed must be enabled, or this returns ErrChangeFeedDisabled.
• Finish before the snapshot's deadline, and keep change-feed retention long enough that Cursor is still there when you start consuming.
• Like Snapshot, it uses the reader's current view; call Refresh first if you need the very latest commit.
• ChangeHead returns only the cursor, without a snapshot: the first position the reader's view does not hold. A consumer at that cursor has every change this reader has loaded. It returns the same errors.

Prefetch SSTs to disk

type KeyRange struct {
    Min []byte // inclusive; nil means the beginning
    Max []byte // exclusive; nil means the end
}

func PrefixRange(prefix []byte) KeyRange

type PrefetchOptions struct {
    Range       KeyRange
    All         bool
    MaxSSTs     int
    MaxBytes    int64
    Concurrency int
}

type PrefetchStats struct {
    MatchedSSTs int
    CachedSSTs  int
    SkippedSSTs int
    BytesRead   int64
}

stats, err := reader.Prefetch(ctx, isledb.PrefetchOptions{
    Range:       isledb.PrefixRange([]byte("user:")),
    MaxBytes:    256 << 20,
    Concurrency: 4,
})

Prefetch downloads the selected SSTs (metadata, Bloom filter and data) into the disk cache, fetching only what is missing and sharing requests with concurrent reads.

• Use All: true to prefetch the whole keyspace.
• Zero MaxSSTs or MaxBytes means no limit beyond the disk cache itself. MaxBytes bounds what this call downloads, each SST's Bloom filter included, so repeated calls warm a large range in steps.
• It skips SSTs already on disk and selects only what fits in the disk cache's free space, each SST counted with its Bloom filter, so repeating a prefetch larger than the cache keeps what the last one cached. SSTs left out count in SkippedSSTs.
• CachedSSTs counts selected SSTs wholly on disk when it returns; BytesRead counts bytes it fetched.
• It uses the reader's current view and does not force a refresh.
• Reusing a CacheDir across restarts. The disk cache keeps its files, so a restarted reader can find it full of SSTs that compactions replaced while it was down. Prefetch never evicts, so it would cache nothing. Call PruneDiskCache after opening and before the start-up prefetch: it deletes every SST that neither the published view nor a view held for NextView names, and returns how many cache entries it deleted. It deletes without pacing, so call it before serving reads; a snapshot of an older view fetches again what it needs. The reader never prunes on its own, at open or later.
• Stale entries on a long-running reader. The reader deletes a retired SST's files once, at the publish that retires it. A snapshot, iterator or BootstrapView opened before that publish that reads the SST afterwards caches it again, and nothing deletes it a second time. Such entries are evicted first when lookups need space, so reads are unaffected, but until then they take free space that prefetch and NextView.Prefetch could use. A reader that serves long reads of older views across compactions and also warms views can call PruneDiskCache periodically; with nothing to delete it only scans the cache's index in memory.
• Partly cached SSTs count at full size. An SST with some parts already on disk, from an interrupted prefetch or from lookups, is selected only if its whole size fits in free space. On a nearly full cache it may be skipped and load on demand.

reader, err := db.OpenReader(ctx, opts)
if err != nil {
    return err
}
if _, err := reader.PruneDiskCache(ctx); err != nil {
    return err
}
stats, err := reader.Prefetch(ctx, isledb.PrefetchOptions{All: true})

• A reader runs one prefetch at a time, NextView.Prefetch included; another waits for it, or for its context to end. Each then sees the space the last one used, so two never fill the same free space.

Freshness and outages

A reader answers from a loaded view of the manifest. The view is refreshed in the background every Views.RefreshAfter; reads never start a refresh or wait for one.

So during an outage reads can be up to MaxPinnedViewAge old, and they stay correct: every SST a view names is kept until the view expires.

Spread-out refreshes. Each reader picks a random offset within RefreshAfter when it opens and refreshes exactly once per interval at that offset. Readers started together by a deploy stay spread evenly; neither a forced Refresh nor a failure moves the schedule, and retries after an outage stay on the reader's own offset. N readers read CURRENT a steady N / RefreshAfter times a second. The one exception is opening: each reader loads its first view as it opens.

An idle reader still reads CURRENT once per interval.

Refresh always reloads, waits and returns any failure, so a caller that needs the latest commits learns when it cannot have them. It reflects every commit made before the call, even when it joins a refresh already in progress.

RequestRefresh starts the background refresh now instead of at the next interval and returns at once. Use it when something tells you the data changed, such as a message from the writer, so readers pick up the change without polling more often. It runs the refresh the interval would run, so in manual mode the newer view is held for NextView and warmed before it is published. Requests within a second of the last are merged into one refresh at the end of that second, never dropped. Failures are logged and counted, as for any background refresh. Keep RefreshAfter as the safety net for a missed request.

Internals of how a reader fetches and caches data are in the appendix.

Prepare views before reads switch to them

A compaction rewrites data into new SSTs. When reads switch to a view naming them, the first lookups of hot keys each wait for object storage. In manual mode the application takes each newer view the reader loads, prepares it, for example by downloading its new SSTs, and only then switches reads to it.

type ViewPosition uint64

type SSTInfo struct {
    ID             string
    Level          int
    Size           int64 // in the manifest
    BloomSize      int64 // stored after Size in the same object
    MinKey, MaxKey []byte
}

func (v *NextView) Previous() ViewPosition
func (v *NextView) Next() ViewPosition
func (v *NextView) Version() Version
func (v *NextView) Added() []SSTInfo
func (v *NextView) Removed() []SSTInfo
func (v *NextView) Snapshot() (*Snapshot, error)
func (v *NextView) Prefetch(ctx context.Context, opts PrefetchOptions) (PrefetchStats, error)
func (v *NextView) Publish() error
func (v *NextView) Discard()

opts := isledb.DefaultReaderOpenOptions("/var/cache/isledb")
opts.Views.Manual = true
reader, err := db.OpenReader(ctx, opts)
if err != nil {
    return err
}
for {
    next, err := reader.NextView(ctx) // waits for the reader's next newer load
    if err != nil {
        return err // ctx ended or the reader closed
    }
    if _, err := next.Prefetch(ctx, isledb.PrefetchOptions{All: true}); err != nil {
        log.Printf("warming view %d: %v", next.Next(), err) // still correct, only colder
    }
    switch err := next.Publish(); {
    case err == nil, errors.Is(err, isledb.ErrViewChanged), errors.Is(err, isledb.ErrNextViewExpired):
        // Published, or another view was; take the next one.
    default:
        return err
    }
}

• The reader keeps loading. Every RefreshAfter it loads the manifest and renews an unchanged view, as in the default mode. A newer view is kept unpublished for the application instead of replacing the published one.
• NextView returns a loaded view newer than the published one, waiting for the reader's next such load if none is pending. Each load is handed out once: after a view is taken, published or not, the next call waits for a later load, at most RefreshAfter away, so two goroutines calling it get different loads. Outside manual mode it returns ErrNotManual.
• Positions. A ViewPosition is the manifest log position a view reflects; two views at the same position hold the same data. Reader.ViewPosition() returns the published one without a lock, so state prepared per view can be checked on every read. Previous is the published position when the view was handed out, Next the view's own.
• Added and Removed list the SSTs the view names that the published one does not, and the reverse. An SST moved between levels is in neither. Caching an SST downloads Size + BloomSize bytes.
• Prefetch downloads the Added SSTs, as Reader.Prefetch does, into the disk cache's free space only, so it never evicts what reads are using. Give the disk cache room for the largest compaction's output beside the live data, up to twice the store when a compaction rewrites all of it, or warming covers only part of a view and the rest loads on demand.
• Snapshot reads the view before it is published. It never refreshes.
• Publish is a compare-and-swap: it switches reads to the view only if the published view is still at Previous. Otherwise it returns ErrViewChanged. A view past its readable life, MaxPinnedViewAge from when it was loaded, returns ErrNextViewExpired. After Publish or Discard, every method that acts returns ErrNextViewDone. Discard releases nothing, so it is optional.
• Refresh publishes at once in manual mode too, for read-your-writes. A pending Publish then gets ErrViewChanged. To load a newer view early and still warm it, call RequestRefresh instead.
• The safety net. If the published view has been outdated for MaxLag, the reader publishes the newest view it loaded itself, at that time and without reading object storage, so a stuck application loop costs freshness, never availability. If the published view expires while a newer, valid view is loaded, the reader publishes that view, even when refreshes are failing.
• The cost is visibility. A change reaches reads after the next refresh plus the time to prepare it, at most MaxLag.

Change feed

The change feed is optional. Open the database with it, then open the writer: the writer turns it on.

db, err := isledb.Open(ctx, bucketURL, isledb.DBOptions{
    Prefix: "accounts",
    ChangeFeed: &isledb.ChangeFeedOptions{
        Payload: isledb.ChangeFeedFullValues,
    },
})

type ChangeFeedPayload uint8

const (
    ChangeFeedKeysOnly ChangeFeedPayload = iota + 1
    ChangeFeedFullValues
)

type ChangeFeedOptions struct {
    Payload ChangeFeedPayload
}

• ChangeFeedKeysOnly records the operation, key, sequence and expiry. It suits invalidation and "fetch the current value" consumers.
• ChangeFeedFullValues also records PUT values, for replaying history.

Only a writer turns the feed on: OpenWriter, on a DB opened with ChangeFeed, turns it on once it holds the database, before it takes any write. Opening a DB with ChangeFeed and no writer, for a reader, a change reader or maintenance, only checks the mode; it doesn't turn the feed on, so a running writer without the feed keeps committing.

The payload mode is stored when the feed is turned on and cannot change. Reopening with a different mode returns ErrChangeFeedPayloadMismatch; reopening with ChangeFeed == nil adopts the stored mode. Turning the feed on for an existing database starts it at the current head; older history is not added.

Types

type ChangeOperation uint8

const (
    ChangePut ChangeOperation = iota + 1
    ChangeDelete
)

type Change struct {
    Sequence  uint64
    Revision  ViewPosition
    Operation ChangeOperation
    Key       []byte
    Value     []byte
    HasValue  bool
    ExpiresAt time.Time
}

type ChangeCursor struct { /* opaque */ }

func ParseChangeCursor(value string) (ChangeCursor, error)
func (c ChangeCursor) String() string
func (c ChangeCursor) IsZero() bool
func (c ChangeCursor) Compare(other ChangeCursor) int
func (c ChangeCursor) MarshalText() ([]byte, error)
func (c *ChangeCursor) UnmarshalText(text []byte) error

type ChangeBounds struct {
    Oldest  ChangeCursor
    Head    ChangeCursor
    Payload ChangeFeedPayload
}

type ChangePage struct {
    Changes []Change
    Next    ChangeCursor
    Head    ChangeCursor
}

func (p ChangePage) CaughtUp() bool

In keys-only mode, a PUT has HasValue == false and a nil Value. In full-values mode, HasValue tells an empty value apart from a missing one. Deletes never carry a value.

Revision is the lowest view position that holds the change, that of the commit holding it; changes committed together share it. A Reader whose ViewPosition is at or past it sees the change. A write's WaitCommitted returns this position or a later one. In keys-only mode, read the key from a reader at or past Revision: one behind it can still return the old value.

Read pages

type ChangeReadOptions struct {
    MaxChanges int
    MaxBytes   int64
}

func DefaultChangeReadOptions() ChangeReadOptions

func (r *ChangeReader) Bounds(ctx context.Context) (ChangeBounds, error)
func (r *ChangeReader) Read(ctx context.Context, from ChangeCursor, opts ChangeReadOptions) (ChangePage, error)
func (r *ChangeReader) WaitForChanges(ctx context.Context, from ChangeCursor) error
func (r *ChangeReader) LoadedHead() ChangeCursor
func (r *ChangeReader) Close() error

• Default page limits: 1,024 changes and 16 MiB. MaxChanges is capped at 65,536. MaxBytes counts key and value bytes, not Go allocation overhead. A single change larger than MaxBytes is returned alone, so the cursor always moves.
• Zero fields select defaults; negative ones return ErrInvalidChangeReadOptions.
• A cursor is an opaque resume position: a manifest entry plus an index within its change batch. It is not Change.Sequence, and you cannot seek by sequence.
• Read does not wait for new writes. A page may be empty yet still move the cursor past manifest entries with no user writes. Keep reading until CaughtUp() is true, then call WaitForChanges with the page's Next cursor, or poll at your own interval.
• WaitForChanges returns once the view a Read would use has its head past the cursor. With a read cache it waits on the cache's long poll and returns within about half a second of a commit; with cache replicas that lag one another it waits, checking once a second, until the replica serving CURRENT has the commit too. Without a cache, or while the cache does not answer, it reads CURRENT every second. A cursor behind the view the reader holds returns at once, without a request. If a view cannot be loaded (storage failing while the cache does not answer), it returns that error; call it again to keep waiting. The head also moves for commits without user changes, so the next Read may return an empty page. It returns the context's error when the context ends, and ErrChangeReaderClosed when the reader closes.
• Read reads CURRENT only when the cursor has reached the head of the view the reader holds, or that view has expired (MaxPinnedViewAge after it was read). Catching up through a backlog reads it about once; a caught-up reader reads it on each Read, to see new commits. Retention that advances meanwhile shows at the next view reload, as Bounds does; the changes the held view names stay in storage until it expires.
• With a read cache, a cursor past the head of the cache's copy of CURRENT is checked against CURRENT in storage before Read returns ErrInvalidChangeCursor, so a cursor from a newer view, such as another reader's, is never refused.
• LoadedHead returns the head of the newest view the reader has loaded, without a request: every cursor up to it was in the feed. It is zero before the first load.
• Compare orders two cursors: -1, 0 or +1 as the first comes before, at or after the second. A zero cursor, which reads from the oldest change, comes first.

Read the feed through a read cache

type ChangeReaderOptions struct {
    ReadCache ReadCacheOptions
    Metrics   *ReaderMetrics
}

OpenChangeReader takes at most one ChangeReaderOptions. With ReadCache.URL set, a change reader reads through the regional read cache (see ReadCacheOptions under Read data) before storage:

• The change batches a page needs, their indexes and blocks. Many readers tailing the feed read the same recent batches, which the cache keeps once for all of them. Every index and block has a checksum: bytes from the cache that fail it are read again from storage.
• The feed's view, CURRENT, under a Reader's rules: a copy older than MaxCurrentAge, or behind the view the reader holds, is read from storage instead, and the view's lifetime is counted from when the cache read it from storage. A retry after a missing batch reads storage.
• The manifest pages holding older change entries, checked against their reference.
• WaitForChanges, through the cache's long poll.

Any read the cache cannot answer is read from storage, so the feed returns the same changes either way. Metrics counts the cache's reads and fallbacks (ReadCacheReads, ReadCacheFallbacks), its CURRENT copies (ReadCacheCurrent, ReadCacheCurrentAge) and the long poll's answers (ReadCacheDoorbells). A bad URL fails OpenChangeReader with ErrInvalidReaderOptions. Each change reader with a read cache keeps its own in-memory cache of manifest pages, of up to 1,024 pages, as each Reader does.

• Change batches are split into compressed, checksummed blocks; a read fetches only the blocks its page needs and reuses decoded blocks from a bounded cache.

Choose where to start

A zero cursor means "no saved position", not "start at the latest change". Pick the starting policy explicitly:

bounds, err := reader.Bounds(ctx)
if err != nil {
    return err
}

// Replay every change still retained.
replayCursor := bounds.Oldest

// Skip existing history; consume only changes published after Bounds.
tailCursor := bounds.Head

Oldest is the first retained position. Head is just past everything Bounds saw, so a first read from it is usually empty and caught up. (A zero cursor passed to Read starts at Oldest, but calling Bounds makes the choice explicit.)

Save the starting cursor before you poll. Otherwise a process that picks Head, crashes before saving it, and restarts would pick a newer head and silently skip changes. After that, save page.Next only once the whole page has been applied.

func drainChanges(
    ctx context.Context,
    reader *isledb.ChangeReader,
    savedCursor string,
    startAtHead bool,
    apply func(isledb.Change) error,
    saveCursor func(string) error,
) error {
    cursor, err := isledb.ParseChangeCursor(savedCursor)
    if err != nil {
        return err
    }

    if cursor.IsZero() {
        bounds, err := reader.Bounds(ctx)
        if err != nil {
            return err
        }
        if startAtHead {
            cursor = bounds.Head
        } else {
            cursor = bounds.Oldest
        }
        if err := saveCursor(cursor.String()); err != nil {
            return err
        }
    }

    options := isledb.DefaultChangeReadOptions()
    for {
        page, err := reader.Read(ctx, cursor, options)
        if err != nil {
            return err
        }
        for _, change := range page.Changes {
            if err := apply(change); err != nil {
                return err
            }
        }
        if err := saveCursor(page.Next.String()); err != nil {
            return err
        }
        cursor = page.Next
        if page.CaughtUp() {
            return nil
        }
    }
}

When a cursor expires

If retention deletes changes past a saved cursor, Read refreshes its view and returns ErrChangeCursorExpired: there is a gap between your position and the oldest retained one. You can:

• restart from Bounds().Oldest, accepting the gap;
• rebuild derived state from a KV snapshot, then resume the feed (see BootstrapView);
• or, if you need every historical event, stop and recover explicitly; a current-state snapshot cannot recreate intermediate updates or deletes.

Run maintenance

Maintenance compacts, checkpoints, applies change-feed retention and deletes retired objects. It is designed to run in its own process, against the same bucket and prefix:

writer process       -> buffers writes and commits them
reader processes     -> load views and read independently
maintenance process  -> prepares compaction, checkpoints and retention
reclamation workers  -> delete retired objects at their own bounded pace

How its work is published. Maintenance never edits the manifest itself. It does the heavy work (for example, writing compacted SSTs), then stages a command in a small mailbox object, maintenance/HEAD. The active writer checks the mailbox in the background every WriterOptions.Maintenance.PollInterval (one second by default, each check bounded to 5 seconds) and applies the command with its next commit, in order with data commits. Flush never waits on the mailbox, so a slow or hung mailbox cannot delay commits. Close checks it once more so a command staged just before shutdown is not left behind, and a command superseded by a later one is skipped. Without a flush interval, a fetched command is applied at the next Flush, Drain or Close.

Deletion runs separately. SSTs, change batches, manifest snapshots and manifest pages are deleted in independent lanes at their own pace, so slow deletes never block compaction.

func (m *Maintenance) Run(ctx context.Context) error
func (m *Maintenance) RunOnce(ctx context.Context) (MaintenanceStats, error)
func (m *Maintenance) Shape(ctx context.Context) (LSMShape, error)
func (m *Maintenance) Close(ctx context.Context) error

Run continues until its context ends, Close is called, or another process takes over maintenance. Close with a fresh context, since the run context is usually already cancelled:

func runMaintenance(ctx context.Context, db *isledb.DB) error {
    maintenance, err := db.OpenMaintenance(ctx, isledb.DefaultMaintenanceOptions())
    if err != nil {
        return err
    }

    runErr := maintenance.Run(ctx)

    closeCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()
    closeErr := maintenance.Close(closeCtx)

    if errors.Is(runErr, context.Canceled) {
        runErr = nil
    }
    return errors.Join(runErr, closeErr)
}

Shape reports the database's files by level now: how much newly saved data waits in L0 and how deep it is, and each level's size against its target. It reads only what changed since its last call, and it doesn't wait for a compaction in progress, so it can be called every few seconds to watch whether compaction keeps up. The types are in the appendix.

For a scheduled job, call RunOnce instead:

stats, err := maintenance.RunOnce(ctx)

RunOnce makes one control pass and one bounded pass of each deletion lane. If it returns MaintenanceWaitingForWriter, it has staged a command that the writer must apply; the next run picks up from there. One scheduled run is a bounded step of progress, not a promise to clear the whole backlog.

Maintenance options

type MaintenanceOptions struct {
    IdleInterval        time.Duration
    SSTCompaction       SSTCompactionOptions
    ManifestCheckpoint  ManifestCheckpointOptions
    ChangeFeedRetention *ChangeFeedRetentionOptions
    Reclamation         ReclamationOptions
    OnCycle             func(MaintenanceStats)
    OnReclamationCycle  func(ReclamationCycleStats)
    OnError             func(error)
}

type SSTCompactionOptions struct {
    ReadConcurrency   int
    ScratchDir        string
    L0TriggerSSTs     int
    BaseLevelBytes    int64
    LevelGrowthFactor int
    TargetSSTBytes    int64
    L0ToBaseRatio     float64
    L0MaxDepth        int
    L0MaxAge          time.Duration
}

type ManifestCheckpointOptions struct {
    TargetReplayPages uint64
    TargetReplayBytes uint64
}

type ReclamationOptions struct {
    MaxConcurrentDeletes int
    SST                  DeleterOptions
    ChangeFeed           DeleterOptions
    Manifest             ManifestDeleterOptions
}

type DeleterOptions struct {
    PollInterval      time.Duration
    MaxObjectsPerPass int
}

type ManifestDeleterOptions struct {
    DeleterOptions
    AuditInterval time.Duration
}

func DefaultMaintenanceOptions() MaintenanceOptions

The manifest lane also audits for orphaned objects once an hour (AuditInterval). MaxObjectsPerPass bounds normal work, but one SST retirement plan, already bounded, may finish in a single pass even if it goes over.

Scratch space. Compaction streams its inputs to ScratchDir, so a job is not limited by memory. Each store, role and fence epoch gets a private session directory below it. A clean close removes the session; a later epoch tries to remove abandoned ones, and a failed cleanup never stops maintenance from opening. IsleDB never deletes the configured directory itself or anything below it that it did not create.

How L0 is compacted. The writer saves a small L0 file at each flush. Once L0 holds L0TriggerSSTs files, each maintenance cycle does one of two things:

• Merge L0 into L1 when L0 holds at least L0ToBaseRatio of the L1 bytes it overlaps, when some key is covered by L0MaxDepth L0 files, or when L0's oldest data is L0MaxAge old. Files that overlap nothing in L1, as with keys written in order, move without being rewritten.
• Otherwise, merge the newest L0 files with each other into one L0 file, so L1 isn't rewritten for a little new data. L0 then stays a few files deep, and each byte is rewritten a few times in small merges on its way to L1.

With random keys, L0 overlaps nearly all of L1, so L0ToBaseRatio rarely triggers and L0MaxDepth decides when L0 is merged into L1.

While L0 has work and is changing, maintenance runs a cycle every 2 seconds (5 seconds on Cloud Storage) instead of every IdleInterval. SSTCompactionStats.L0Merges counts the merges within L0.

The first merge within L0 moves the database to layout version 3. Releases before this one refuse to open it from then on, so upgrade every process that opens the database before running this release's maintenance.

Compacted SSTs are encoded with DBOptions.SSTOutput.Compacted. The compaction planner's limits are described in the appendix.

OnCycle receives statistics for each control pass and OnReclamationCycle for each deletion pass; the types are in the appendix.

Change-feed retention

Retention is off until ChangeFeedRetention is set:

type ChangeFeedRetentionOptions struct {
    RetainFor time.Duration
}

func DefaultChangeFeedRetentionOptions() ChangeFeedRetentionOptions

options := isledb.DefaultMaintenanceOptions()
retention := isledb.DefaultChangeFeedRetentionOptions() // seven days
retention.RetainFor = 15 * 24 * time.Hour
options.ChangeFeedRetention = &retention

RetainFor is a minimum age, not an exact deletion time: publishing, view-safety rules and paced deletion can keep objects longer.

Prometheus metrics

func DefaultWriterMetrics(constLabels prometheus.Labels) *WriterMetrics
func DefaultReaderMetrics(constLabels prometheus.Labels) *ReaderMetrics

Set the result as WriterOptions.Metrics or ReaderOpenOptions.Metrics. The constructors create the collectors but do not register them; register them with your prometheus.Registerer.

Writer metrics cover puts, deletes, backpressure, flush counts, errors, latency and bytes, plus:

• isledb_writer_committed_sequence: the highest sequence committed and visible to readers. Set at open from the manifest and after each commit, so it never counts writes still in memory and continues across a failover.
• isledb_writer_oldest_uncommitted_timestamp_seconds: the Unix time the oldest uncommitted write was accepted, or 0 when everything is committed. Alert when commits keep failing:

time() - (isledb_writer_oldest_uncommitted_timestamp_seconds > 0) > 120

Change-feed lag. Every Change carries the writer's sequence, so a consumer that exports its last applied Change.Sequence gets its lag from Prometheus with no extra reads. With several databases, label each one's metrics through constLabels and match on it:

clamp_min(
  max by (db) (isledb_writer_committed_sequence)
    - on(db) group_right isledb_follower_applied_sequence,
  0)

clamp_min hides brief negative values caused by scrape timing. Lag counts changes committed but not yet applied; writes still buffered in the writer are not committed and not counted.

Reader metrics cover refreshes, point reads, scans and SST range reads, plus:

• stale_reads_total: reads answered from a view whose refresh failed (see Freshness and outages);
• view_loaded_timestamp_seconds: when the current view was loaded; time() - isledb_reader_view_loaded_timestamp_seconds is its age;
• dead_ssts_pending: SSTs a publish retired that are not yet deleted from the disk cache. It rises after a compaction and drains; one that keeps rising means deletes fall behind compactions.

In manual mode:

• next_view_publishes_total{result}: NextView.Publish calls by result, published, changed or expired. Calls on a view already done, or on a closed reader, are not counted.
• view_outdated_since_timestamp_seconds: when the published view became outdated, the load time of a newer view not yet published, or 0 when it is the newest loaded. Alert when preparation falls behind:

time() - (isledb_reader_view_outdated_since_timestamp_seconds > 0) > 60

• view_safety_publishes_total: views the reader published itself after MaxLag. A rising count means the application's loop is stuck or too slow.

Error reference

Check sentinel errors with errors.Is:

if errors.Is(err, isledb.ErrBackpressure) {
    // Retry according to the application's admission policy.
}

Storage and context errors can also be returned, wrapped. Log the complete error.

Database and configuration

Writer

Reader and snapshots

Change feed

Maintenance

Appendix: how a reader reads SSTs

Every SST is read from object storage by byte range, through four caches:

Fetching.

• An SST of up to 4 MiB is fetched whole, with its Bloom filter, in one request, and its SHA-256 is checked against the manifest.
• A larger SST's metadata and Bloom filter are fetched in one request each; its data in aligned 128 KiB chunks. A lookup fetches the chunk holding its block.
• A scan reads ahead: each request fetches twice as many chunks as the last, up to 4 MiB, and a seek elsewhere starts small again.
• No request repeats bytes already on disk or being fetched, and concurrent reads of the same part share one request.
• Data blocks of large SSTs are checked by their own checksums, which catch damaged bytes but not a different, internally valid object stored under the SST's name.

What gets cached. A point lookup caches the blocks it reads, in memory and on disk. A scan, and each iterator seek, caches only its first 64 KiB of keys and values in each SST, so short reads (a page, a prefix, a seek) are warm when repeated. Reading further uses buffers of its own, still drawing on what is cached: a long scan adds at most about 64 KiB per SST to memory and two chunks to disk, and never evicts what lookups reuse.

Speed and eviction. A read of an already-open SST parses no metadata; a warm lookup takes about 1 µs. Each cache evicts least recently used first. Nothing is dropped when an SST leaves the manifest, so a snapshot still reading it stays warm. A cached part that proves damaged is dropped and fetched again; a lookup that meets damaged cache bytes retries once from object storage instead of failing.

The disk cache survives restarts and holds no files open: each read opens, reads and closes, so the budget is exact.

• A fetched part is written before the read returns, to a temporary file renamed into place without fsync. On a local SSD that adds about 2–4% to a cold read and nothing to a warm one.
• Each file name carries its size, so startup only lists directories: about 0.1 s for a 10 GiB cache, a few seconds at 100 GiB.
• After a crash, startup drops unfinished writes. A file shorter than its name says fails the read that reaches past its end and is dropped then; checksums catch damaged bytes.
• Fetched bytes are never held in memory waiting to be written; if the disk cannot store a part, a later read fetches it again.
• Opening logs a warning when the filesystem cannot hold DiskCacheSize.

Block cache memory. The block cache is Pebble's. With cgo it lives outside the Go heap: it counts toward resident memory but not toward GOMEMLIMIT or heap profiles. Without cgo (CGO_ENABLED=0) it is ordinary Go heap, which the garbage collector lets grow to about twice the live heap under the default GOGC; set GOMEMLIMIT to bound the process.

Cache statistics

type CacheStats struct {
    Hits        int64
    Misses      int64
    Bytes       int64
    MaxBytes    int64
    EntryCount  int
    MaxEntries  int
    Evictions   int64
    Corruptions int64
    Bypasses    int64
    Failures    int64
}

type DiskCacheStats struct {
    Meta     CacheStats // SST metadata and Bloom filters
    Data     CacheStats // whole small SSTs and chunks of larger SSTs' data
    SSTDrops int64      // reads that failed on damaged bytes and dropped the SST
}

• BlockCacheStats counts hits and misses on index and data blocks. The metaindex and properties blocks, read on every SST open and never cached, are not counted. OpenSSTCacheStats reports how many reads found their SST already open.
• Byte-bounded caches report MaxEntries == 0; the open-SST cache, bounded by count, reports MaxBytes == 0.
• The disk cache has one budget, DiskCacheSize, shared by metadata and data: Meta.MaxBytes and Data.MaxBytes both report it, and Meta.Bytes + Data.Bytes is what is in use. Data is evicted first, and data is never stored by evicting metadata.
• For the disk cache: Corruptions counts entries it found damaged itself (a wrong size, or a Bloom filter failing its checksum); Bypasses entries not stored, because they are larger than the whole cache, are data that would fit only by evicting metadata, or belong to an SST whose files are being deleted; Failures entries that could not be written. Bypassed and failed entries are still served, just not kept, so a disk that keeps failing means each later read fetches them again.
• SSTDrops counts reads that failed on what looked like damaged bytes; each drops the SST from every layer, so the next read fetches it again. It counts drops, not distinct SSTs: concurrent readers of one damaged SST each count, a lookup whose retry also fails counts twice, and an SST read in chunks that is damaged in object storage, or fails to open the same way every time, counts on every read. A small SST damaged in object storage fails its whole-object checksum when fetched, so it fails reads without counting. A steadily rising count points at an SST in one of those states.

Appendix: SST output policy

How new SST files are encoded is a runtime setting. It applies to files written from now on and is not stored in the manifest; every file describes its own encoding, so readers handle a mix.

type SSTOutputOptions struct {
    L0        SSTEncodingOptions
    Compacted SSTEncodingOptions
}

type SSTEncodingOptions struct {
    Compression     string
    BlockBytes      int
    BloomBitsPerKey int
}

func DefaultSSTOutputOptions() SSTOutputOptions

Compression is "none", "snappy" or "zstd". Zero fields select the defaults:

The writer and maintenance can use different settings:

dbOptions := isledb.DBOptions{
    Prefix: "accounts",
    SSTOutput: isledb.SSTOutputOptions{
        L0: isledb.SSTEncodingOptions{
            Compression:     "snappy",
            BlockBytes:      4 << 10,
            BloomBitsPerKey: 10,
        },
        Compacted: isledb.SSTEncodingOptions{
            Compression:     "zstd",
            BlockBytes:      16 << 10,
            BloomBitsPerKey: 10,
        },
    },
}

Separate writer and maintenance processes each pass their own DBOptions. Different settings are safe, though one shared configuration makes performance easier to predict.

Appendix: compaction limits

One manifest entry can remove at most 128 objects and add at most 128. This is a fixed format rule, not a tuning option. The planner picks the widest source batch whose whole overlap with the next level fits. If even one source SST overlaps too many files below it, the planner first compacts that destination level further down and retries the original move on a later cycle.

If that drain reaches the bottom of the tree, IsleDB creates one deeper level. For sorted levels this is usually a metadata-only move: the SSTs keep their IDs and bytes, and only their level changes. It is a correctness escape hatch for the 128-object limit, not normal level sizing. Doing it repeatedly can deepen reads and delay merging tombstones and overwritten values, so it is counted in SSTCompactionStats.ForcedLevelCreations, with DeepestForcedLevel recording the deepest level reached. Ordinary creation of a new bottom level by size is not counted. As with every compaction, the statistic counts a staged job; the new layout is visible only once the writer applies it.

Appendix: maintenance statistics

type MaintenanceState uint8

const (
    MaintenanceIdle MaintenanceState = iota
    MaintenanceWaitingForWriter
)

type MaintenanceTask uint8

const (
    MaintenanceTaskNone MaintenanceTask = iota
    MaintenanceTaskSSTCompaction
    MaintenanceTaskManifestCheckpoint
)

type ReclamationFamily string

const (
    ReclamationSST        ReclamationFamily = "sst"
    ReclamationChangeFeed ReclamationFamily = "change_feed"
    ReclamationManifest   ReclamationFamily = "manifest"
)

MaintenanceState, MaintenanceTask, ChangeOperation, ChangeFeedPayload and WriterStatus implement String.

type MaintenanceStats struct {
    State               MaintenanceState
    WaitingSince        time.Time
    Scheduling          MaintenanceScheduleStats
    SSTCompaction       SSTCompactionStats
    SSTCleanup          SSTCleanupStats
    ChangeFeedRetention ChangeFeedCleanupStats
    ManifestCheckpoint  ManifestCheckpointStats
    ManifestCleanup     ManifestCleanupStats
    RejectedCommands    int
    Duration            time.Duration
}

// LSMShape is what Maintenance.Shape returns.
type LSMShape struct {
    L0      LevelShape   // newly saved files; TargetBytes is 0
    L0Depth int          // the most L0 files covering one key
    Levels  []LevelShape // L1 and below, in order
}

type LevelShape struct {
    Level       uint32
    Files       int
    Bytes       int64
    TargetBytes int64 // BaseLevelBytes for L1, LevelGrowthFactor times more per level below
}

type ReclamationCycleStats struct {
    Family     ReclamationFamily
    SST        SSTCleanupStats
    ChangeFeed ChangeFeedCleanupStats
    Manifest   ManifestCleanupStats
    Duration   time.Duration
}

type MaintenanceScheduleStats struct {
    Selected              MaintenanceTask
    CompactionSourceLevel uint32
    CompactionWorkUnits   uint32
    CompactionCritical    bool
    CheckpointEligible    bool
    CheckpointUrgent      bool
    ReplayPages           uint64
    ReplayBytes           uint64
}

type SSTCompactionStats struct {
    Jobs                 int
    InputSSTs            int
    OutputSSTs           int
    ForcedLevelCreations int
    DeepestForcedLevel   uint32
    OutputBytes          int64
    MovedJobs            int
    MovedBytes           int64
    ReadBytes            int64
    L0Merges             int
    RewrittenBytes       int64
}

type SSTCleanupStats struct {
    SSTsPlanned    int
    PlansPrepared  int
    PlansScanned   int
    PlansCompleted int
    DeleteAttempts int
    SSTsDeleted    int
    DeferredPlans  int
    Failures       int
}

type ManifestCheckpointStats struct {
    Staged      bool
    ReplayPages uint64
    ReplayBytes uint64
}

type ChangeFeedCleanupStats struct {
    EntriesRetired  int
    BatchesPlanned  int
    BatchesDeleted  int
    BlockedRetained int
    FailedDeletes   int
    Duration        time.Duration
}

type ManifestCleanupStats struct {
    Snapshots ManifestSnapshotCleanupStats
    Pages     ManifestPageCleanupStats
}

type ManifestSnapshotCleanupStats struct {
    SnapshotsMarked  int
    DeleteAttempts   int
    SnapshotsDeleted int
    Protected        int
    Deferred         int
    Failures         int
    MarkersScanned   int
    MarkersCleared   int
    ObjectsScanned   int
    Duration         time.Duration
}

type ManifestPageCleanupStats struct {
    PagesMarked      int
    PagesDeleted     int
    Protected        int
    Deferred         int
    Failures         int
    MarkersScanned   int
    MarkersCleared   int
    ObjectsScanned   int
    DeleteAttempts   int
    ReachabilityGETs int
    Duration         time.Duration
}

Appendix: the blobstore package

Most applications should use isledb.Open or isledb.OpenBucket. The github.com/ankur-anand/isledb/blobstore package is for storage adapters, integration tests and operational tools.

func blobstore.Open(ctx context.Context, bucketURL, prefix string) (*blobstore.Store, error)
func blobstore.New(bucket *blob.Bucket, bucketName, prefix string) *blobstore.Store
func blobstore.NewMemory(prefix string) *blobstore.Store

Object operations:

func (s *Store) Read(ctx context.Context, key string) ([]byte, Attributes, error)
func (s *Store) ReadStream(ctx context.Context, key string) (*blob.Reader, error)
func (s *Store) ReadRange(ctx context.Context, key string, offset, length int64) ([]byte, error)
func (s *Store) ReadRangeStream(ctx context.Context, key string, offset, length int64) (*blob.Reader, error)
func (s *Store) Attributes(ctx context.Context, key string) (Attributes, error)
func (s *Store) Exists(ctx context.Context, key string) (bool, error)
func (s *Store) Write(ctx context.Context, key string, data []byte) (Attributes, error)
func (s *Store) WriteReader(ctx context.Context, key string, r io.Reader, opts *blob.WriterOptions) (Attributes, error)
func (s *Store) WriteIfMatch(ctx context.Context, key string, data []byte, etag string) (Attributes, error)
func (s *Store) WriteIfNotExist(ctx context.Context, key string, data []byte) (Attributes, error)
func (s *Store) Delete(ctx context.Context, key string) error
func (s *Store) BatchDelete(ctx context.Context, keys []string) error

Listing operations:

type ListOptions struct {
    Prefix    string
    Delimiter string
}

type ObjectInfo struct {
    Key   string
    Size  int64
    IsDir bool
}

type ListResult struct {
    Objects []ObjectInfo
}

func (s *Store) List(ctx context.Context, opts ListOptions) (*ListResult, error)
func (s *Store) NewListIterator(opts ListOptions) *ListIterator
func (it *ListIterator) Next(ctx context.Context) (ObjectInfo, error)
func (s *Store) Walk(ctx context.Context, opts ListOptions, visit func(ObjectInfo) (bool, error)) error

List loads every matching object into memory; prefer NewListIterator or Walk for large listings.

type Attributes struct {
    Size       int64
    ETag       string
    ModTime    time.Time
    Generation int64
}

type BatchDeleteError struct {
    Failed map[string]error
}

• Sentinel errors: blobstore.ErrNotFound, blobstore.ErrPreconditionFailed and blobstore.ErrBucketNameRequired.
• Delete succeeds for an object that does not exist.
• BatchDeleteError names each key that failed.
• For backends without native conditional writes (the in-memory and file stores), Read returns the MD5 of the content as the ETag, which is what WriteIfMatch compares against.

Related documentation

• Project overview
• Object-store schema