Jesse A. Green / notebook

Live demo: the DocCompiler guide library as a static, read-only snapshot (October 2026). No backend, no accounts, nothing to submit.

LevelDB Admin & Architecture Guide

Generated June 2026 from google/leveldb at 7ee830d

AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it.

This guide is for the engineers, architects, and operators who embed LevelDB into a larger system and are responsible for its on-disk data, durability guarantees, build configuration, and long-term maintenance. Where the User Guide shows how to call LevelDB, this guide explains how LevelDB works: its log-structured merge-tree design, on-disk file formats, compaction machinery, build-time configuration, operational procedures, and a security analysis grounded in the source.

LevelDB is a single static (or shared) C++ library — there is no service to deploy, no ports to open, no configuration files to manage. "Administration" here means understanding the storage engine well enough to operate it safely inside your own process: choosing build options, sizing resources, backing up the data directory, recovering from corruption, and reasoning about failure modes.


Table of Contents

  1. Executive Summary
  2. Architecture Overview
  3. On-Disk Layout & File Formats
  4. The Write Path
  5. The Read Path
  6. Compaction
  7. Recovery & Durability
  8. Build & Dependency Configuration
  9. Deployment as a Dependency
  10. Operations
  11. Security Analysis
  12. Repository Structure & Module Reference
  13. Code Patterns & Conventions
  14. Testing Strategy
  15. Known Issues & Potential Improvements
  16. Quick Reference

1. Executive Summary

Attribute Value
Project LevelDB
Type Embeddable key-value storage library
Language C++ (requires C++17); C bindings included
Version 1.23 (kMajorVersion = 1, kMinorVersion = 23)
Build system CMake (≥ 3.22)
License BSD-style
Authors Sanjay Ghemawat, Jeff Dean (Google)
Storage model Log-structured merge-tree (LSM) with 7 levels
Concurrency Multi-threaded within a single process; one process per database
Network surface None — in-process library only
Required dependencies None (POSIX threads)
Optional dependencies snappy, zstd, crc32c, tcmalloc
Build artifacts leveldb library, leveldbutil tool, db_bench benchmarks

LevelDB persists an ordered map of byte-string keys to byte-string values. Writes are first appended to a write-ahead log and inserted into an in-memory memtable (a skip list); when the memtable fills, it is flushed to an immutable sorted table (SSTable) on disk. SSTables are organized into levels and merged by a background compaction process that keeps read amplification bounded and reclaims space from overwritten and deleted keys.

Note

The design is "similar in spirit to the representation of a single Bigtable tablet." Reading this guide alongside the Bigtable paper (section 5.3) gives the full context behind the on-disk structures.


2. Architecture Overview

                          Application (your process)
                                    |
                        leveldb::DB  (public API, db.h)
                                    |
            +-----------------------+-----------------------+
            |                       |                       |
        WRITE PATH               READ PATH            BACKGROUND
            |                       |                       |
            v                       v                       v
   +----------------+      +----------------+      +------------------+
   | WAL log file   |      | memtable (imm) |      | compaction       |
   | (db/log_*)     |      | + SSTables     |      | thread           |
   +----------------+      +----------------+      +------------------+
            |                       |                       |
            v                       v                       v
   +-------------------------------------------------------------------+
   |                In-memory: memtable (skiplist)                     |
   |                Maybe: immutable memtable being flushed            |
   +-------------------------------------------------------------------+
            |                                                  ^
            v                                                  |
   +-------------------------------------------------------------------+
   |  On disk (the database directory):                                |
   |    CURRENT  ->  MANIFEST-NNNNNN  (the version log)                |
   |    LOCK     (single-process lock)                                 |
   |    *.log    (write-ahead logs)                                    |
   |    *.ldb    (SSTables)  organized into levels 0..6                |
   |    LOG, LOG.old  (human-readable info logs)                       |
   +-------------------------------------------------------------------+

2.1 Key components

Component Source Role
DBImpl db/db_impl.{h,cc} Concrete DB implementation; orchestrates writes, reads, compaction, recovery
MemTable db/memtable.{h,cc} In-memory sorted buffer of recent writes (backed by a skip list + arena)
SkipList db/skiplist.h Lock-free-read, single-writer ordered map; the memtable's data structure
WAL log::Writer/Reader db/log_writer.*, db/log_reader.* Append-only write-ahead log for crash recovery
Table / TableBuilder table/table.cc, table/table_builder.cc Read/write immutable SSTables
TableCache db/table_cache.{h,cc} LRU cache of open SSTables and their index blocks
VersionSet / Version db/version_set.{h,cc} The set of SSTables per level; MVCC over the file set
VersionEdit db/version_edit.{h,cc} A delta to the version state, logged to the MANIFEST
Compaction db/version_set.{h,cc} Describes one compaction job (inputs, output level)
Cache (LRU) util/cache.cc Block cache for uncompressed data blocks
Env include/leveldb/env.h, util/env_posix.cc, util/env_windows.cc OS abstraction (files, threads, time)

2.2 Threading model

A database is safe for concurrent use by multiple threads within one process. Internally, mutable shared state is protected by a single port::Mutex (DBImpl::mutex_), with a port::CondVar to signal background-work completion. Thread-safety annotations (GUARDED_BY, EXCLUSIVE_LOCKS_REQUIRED) are used throughout db_impl.h and version_set.h, and the build enables -Wthread-safety when Clang supports it.

The skip list is designed for lock-free reads with a single synchronized writer: nodes are published with release-stores and read with acquire-loads, so concurrent readers always observe fully-initialized nodes. Allocated nodes are never freed until the whole list is destroyed.

Important

"Single process" is enforced by an OS file lock (LOCK file) acquired at open time. A second process attempting to open the same directory fails immediately. Within the owning process, share one DB* across threads — do not open the directory twice.


3. On-Disk Layout & File Formats

A database is a directory of files. Understanding them is essential for backup, recovery, and debugging.

3.1 File types

File Purpose
CURRENT Tiny text file naming the current MANIFEST-NNNNNN
MANIFEST-NNNNNN The version log: which SSTables make up each level, their key ranges, and metadata. A new one is written each time the DB is reopened.
LOCK The single-process lock
*.log Write-ahead logs (recent updates not yet in an SSTable)
*.ldb SSTables (immutable sorted tables). Legacy databases may use .sst.
LOG, LOG.old Human-readable informational log messages
*.dbtmp Temporary files used during operations

These names are produced by db/filename.cc and classified by ParseFileName into the FileType enum (kLogFile, kDBLockFile, kTableFile, kDescriptorFile, kCurrentFile, kTempFile, kInfoLogFile).

3.2 SSTable (.ldb) format

Defined in table/format.cc and documented in doc/table_format.md:

<beginning_of_file>
[data block 1]
[data block 2]
...
[data block N]
[meta block 1]
...
[meta block K]
[metaindex block]
[index block]
[Footer]        (fixed size; starts at file_size - sizeof(Footer))
<end_of_file>
  • Data blocks hold sorted key/value pairs, formatted by table/block_builder.cc, each individually (optionally) compressed.
  • Index block has one entry per data block: a separator key ≥ the block's last key, mapping to a BlockHandle (offset + size, both varint64).
  • Metaindex block maps meta-block names to their BlockHandles.
  • Footer is fixed-length and ends with a magic number 0xdb4775248b80fb57 (little-endian). It contains the handles for the metaindex and index blocks plus zero-padding to a fixed size.

Filter meta block

If a FilterPolicy is configured, each SSTable stores a filter meta block keyed filter.<Name> in the metaindex. Filters are sliced by file offset with a base of 2 KB: all keys in blocks whose offset falls in [i*base, (i+1)*base-1] are summarized into filter i. A trailing offset array plus lg(base) byte allow mapping a data-block offset to its filter quickly. (See table/filter_block.cc.)

3.3 Write-ahead log (.log) format

Defined in db/log_format.h and documented in doc/log_format.md. The log is a sequence of 32 KB blocks; each block holds a sequence of records:

record :=
  checksum: uint32   // crc32c of {type, data}, little-endian
  length:   uint16   // little-endian
  type:     uint8    // FULL=1, FIRST=2, MIDDLE=3, LAST=4 (kZeroType=0 reserved)
  data:     uint8[length]

The header is 7 bytes (kHeaderSize = 4 + 2 + 1). A user record that doesn't fit in the remaining block space is fragmented across FIRST/MIDDLE/LAST records. A record never starts in the last six bytes of a block; leftover bytes form an all-zero trailer that readers skip.

This framing makes resynchronization trivial after corruption: skip to the next 32 KB block boundary and resume. It also avoids confusion if one log file's contents are embedded inside another.

3.4 Internal key encoding

LevelDB does not store user keys directly in SSTables; it stores internal keys (db/dbformat.h). An internal key is the user key followed by an 8-byte tag packing a 56-bit sequence number and an 8-bit value type:

internal_key = user_key | (sequence << 8 | type)
Constant Value Meaning
kTypeValue 0x1 A value entry
kTypeDeletion 0x0 A deletion marker (tombstone)
kMaxSequenceNumber (1 << 56) - 1 Largest sequence number

Internal keys sort by user key ascending, then by sequence number descending (newest first), via InternalKeyComparator. This ordering is what lets a Get find the most recent version of a key efficiently. Value-type enum values are part of the on-disk format and must never change.

Caution

The ValueType and CompressionType enum values are embedded in persisted data. Changing them would silently corrupt the interpretation of existing databases. They are explicitly annotated "DO NOT CHANGE" in the source.


4. The Write Path

A Put, Delete, or Write(batch) flows through DBImpl::Write as follows:

  1. The operation is wrapped in a WriteBatch (a single Put/Delete is a one-entry batch).
  2. The writer is appended to a queue (writers_); the front writer builds a batch group (BuildBatchGroup) coalescing queued writes for throughput.
  3. MakeRoomForWrite ensures there is space in the active memtable, triggering a flush/compaction and possibly throttling (see below).
  4. The batch is assigned a sequence number and appended to the WAL via log::Writer::AddRecord. If WriteOptions::sync is set, the log file is flushed to stable storage.
  5. The batch is inserted into the active memtable (WriteBatchInternal::InsertInto).
  6. The sequence number is advanced and queued writers are signaled.

When the memtable reaches write_buffer_size (default 4 MB), it becomes the immutable memtable (imm_), a fresh memtable and log file are started, and a background thread flushes imm_ to a level-0 SSTable (WriteLevel0Table).

4.1 Write throttling and stalls

To keep read amplification bounded, the write path reacts to the number of level-0 files (constants in db/dbformat.h):

Constant Value Effect
kL0_CompactionTrigger 4 Start a level-0 → level-1 compaction
kL0_SlowdownWritesTrigger 8 Insert a 1 ms delay per write to let compaction catch up
kL0_StopWritesTrigger 12 Block writes entirely until compaction reduces the file count

Warning

Sustained write bursts faster than the disk's compaction throughput will accumulate level-0 files and eventually hit the stop-writes trigger, stalling your writers. Provision disk I/O for your write rate, and prefer batched writes.


5. The Read Path

DBImpl::Get resolves a key by consulting sources newest-to-oldest, stopping at the first match (a value or a deletion tombstone):

  1. The active memtable (mem_).
  2. The immutable memtable (imm_), if a flush is in progress.
  3. SSTables, level by level, via the current Version. Level-0 files may overlap, so all matching level-0 files are checked (newest first); for levels ≥ 1, files are non-overlapping, so at most one file per level is consulted (binary search via FindFile).

A lookup uses a LookupKey (db/dbformat.h) that packs the user key with the snapshot sequence number, so a read returns the newest version at or before that sequence. SSTable access goes through the TableCache, which keeps open file handles and parsed index blocks; data blocks are served from the block cache (LRU) when present.

5.1 Seek-driven compaction

Each FileMetaData carries an allowed_seeks counter (initialized to 1 << 30, reduced based on file size). When reads repeatedly seek through a file without finding data (because the key lives in a deeper level), Version::UpdateStats decrements the counter; hitting zero marks that file for compaction. This adaptively compacts files that are expensive to read through. Read samples are taken roughly every kReadBytesPeriod (1 MB) of data scanned during iteration.


6. Compaction

Compaction is the heart of an LSM engine: it merges SSTables to bound read cost and reclaim space. Logic lives in db/db_impl.cc (BackgroundCompaction, DoCompactionWork) and db/version_set.cc (PickCompaction, CompactRange).

6.1 Level structure

There are kNumLevels = 7 levels (0–6):

  • Level 0 files are SSTables flushed directly from memtables; their key ranges may overlap.
  • Levels 1–6 contain files with disjoint, sorted key ranges. Each level's total size target grows by 10× (≈10 MB for level-1, 100 MB for level-2, …).

A freshly flushed memtable is pushed as deep as level kMaxMemCompactLevel = 2 when doing so creates no overlap, avoiding expensive level-0→1 work and excess MANIFEST churn.

6.2 What a compaction does

When a level exceeds its size target (or a file is marked by seek stats), PickCompaction selects a file from level L plus all overlapping files from level L+1, and a background thread merges them into new level-(L+1) files. Output files roll over at max_file_size (2 MB) or when the key range would overlap more than ten level-(L+2) files (limiting the cost of the next compaction). During the merge:

  • Overwritten values are dropped (only the newest survives).
  • Deletion tombstones are dropped once no higher-numbered level contains an overlapping key (IsBaseLevelForKey).

Compactions for a level rotate through the key space using a per-level compact_pointer_, so work is spread evenly rather than always re-compacting the same hot range. A compaction that would simply move one file to the next level with no merging is detected as a trivial move (Compaction::IsTrivialMove) and skips the merge entirely.

6.3 Cost characteristics (from doc/impl.md)

  • A level-0 compaction reads up to four 1 MB level-0 files plus up to all level-1 files (~10 MB): ~14 MB read + 14 MB written.
  • A non-level-0 compaction picks one 2 MB file and ~12 overlapping next-level files: ~26 MB read + 26 MB written, ≈0.5 s at 100 MB/s.

Note

The MANIFEST records every change to the file set as a VersionEdit. After each compaction and after recovery, RemoveObsoleteFiles deletes log files that are no longer current and SSTables not referenced by any live version and not an active compaction output.


7. Recovery & Durability

7.1 Recovery sequence

On open, DBImpl::Recover / VersionSet::Recover (per doc/impl.md):

  1. Read CURRENT to find the latest committed MANIFEST.
  2. Read the MANIFEST to reconstruct the set of live SSTables per level.
  3. Clean up stale files.
  4. Replay each recovered *.log (RecoverLogFile), converting buffered records into a new level-0 SSTable.
  5. Begin directing new writes to a fresh log file with the recovered sequence number.

7.2 Durability semantics

Scenario Outcome with sync=false (default) Outcome with sync=true
Process crash (no reboot) No data loss — updates already in OS buffer No data loss
Machine reboot / power loss The last few updates may be lost Durable once Put/Write returns

sync=true calls fsync/fdatasync/F_FULLFSYNC (whichever the platform exposes — detected at build time) before returning. The WAL means that even with sync=false, a clean process crash recovers fully on next open.

Tip

For bulk loads, keep sync=false and restart from a checkpoint after a crash. For a hybrid guarantee, make every Nth write synchronous and record a restart marker.

7.3 Corruption handling

  • Options::paranoid_checks (default off): when on, the DB raises an error on the first detected corruption — possibly at open, possibly later. A single corrupt entry can render large portions unreadable.
  • ReadOptions::verify_checksums (default off): verifies CRC32C on every block read.
  • RepairDB (db/repair.cc): salvages a database that won't open by rebuilding the MANIFEST from the SSTables and logs it can find. Data may be lost.

Caution

RepairDB is a last resort. Always copy the database directory before repairing. The log framing (32 KB blocks, per-record CRC) is specifically designed so a reader can skip a corrupt region and resynchronize at the next block boundary, but repair still cannot recover data whose only copy was in a corrupted region.


8. Build & Dependency Configuration

Everything is driven by the top-level CMakeLists.txt.

8.1 Standards and platform detection

  • project(leveldb VERSION 1.23.0 LANGUAGES C CXX); C++17 required, C11 (gracefully decays to C89) for C code.
  • LEVELDB_PLATFORM_WINDOWS vs. LEVELDB_PLATFORM_POSIX is selected by WIN32. On Windows, _UNICODE/UNICODE are defined; the POSIX path uses util/env_posix.cc, the Windows path uses util/env_windows.cc.
  • Exceptions and RTTI are disabled (-fno-exceptions -fno-rtti, or the MSVC equivalents). Code that links LevelDB does not get exceptions thrown across the boundary.
  • Feature probes: HAVE_FDATASYNC, HAVE_FULLFSYNC, HAVE_O_CLOEXEC, HAVE_CXX17_HAS_INCLUDE, plus port/port_config.h generated from port/port_config.h.in.

8.2 Optional library auto-detection

check_library_exists probes for and conditionally links each of:

Symbol probed Library Linked target effect
crc32c_value crc32c Hardware-accelerated checksums
snappy_compress snappy kSnappyCompression support
zstd_compress zstd kZstdCompression support
malloc (tcmalloc) tcmalloc Faster allocator

If a library is absent, the corresponding feature is simply unavailable at runtime (e.g. Snappy-configured DBs store uncompressed blocks).

8.3 Build targets

Target Built when Notes
leveldb always The library; static unless BUILD_SHARED_LIBS=ON
leveldbutil always CLI tool (db/leveldbutil.cc)
leveldb_tests + per-file tests LEVELDB_BUILD_TESTS=ON (default) Uses GoogleTest submodule
db_bench, db_bench_sqlite3, db_bench_tree_db LEVELDB_BUILD_BENCHMARKS=ON (default) SQLite3/KyotoCabinet variants built only if those libs are found

Shared-library builds compile with -fvisibility=hidden and define LEVELDB_SHARED_LIBRARY so only LEVELDB_EXPORT-annotated symbols are exported (see include/leveldb/export.h).

8.4 Install & packaging

With LEVELDB_INSTALL=ON, CMake installs the library, the 15 public headers (to include/leveldb/), and a CMake package (leveldbConfig.cmake, leveldbConfigVersion.cmake with SameMajorVersion compatibility, and the leveldb:: namespaced targets export). Downstream projects consume it with find_package(leveldb) and link leveldb::leveldb.

Important

The project version in CMakeLists.txt (1.23.0) is kept in sync by hand with kMajorVersion/kMinorVersion in include/leveldb/db.h. When auditing a build, confirm both agree.


9. Deployment as a Dependency

LevelDB has no runtime service; "deployment" means how you ship it inside your application.

9.1 Linking strategies

Static (default)

The library is baked into your binary. No runtime library dependency to distribute. Larger binary; rebuild to pick up LevelDB fixes.

Shared (BUILD_SHARED_LIBS=ON)

Smaller binary; the .so/.dll must travel with your app. Only LEVELDB_EXPORT symbols are visible. Useful when multiple programs share one LevelDB build or for the C-ABI consumers of c.h.

9.2 Vendoring vs. system package

  • System package — find_package(leveldb) against an installed copy. Simple, but ties you to the distro's version.
  • Vendoring / submodule — add the LevelDB source tree to your build with add_subdirectory(leveldb) and link leveldb::leveldb. Pins an exact commit (this guide documents 7ee830d), which is valuable since LevelDB is under "very limited maintenance" and rarely releases.

9.3 Capacity planning

Resource Guidance
Disk Budget well above raw data size: levels overlap during compaction and obsolete data lingers until compacted. Compression (default Snappy) reduces footprint.
File descriptors ~1 open file per 2 MB of working set (max_open_files default 1000). Raise the OS ulimit -n accordingly.
Memory memtable(s) up to ~2×write_buffer_size, the block cache (default internal 8 MB, or your NewLRUCache size, holding uncompressed blocks), plus filter bits (~bits_per_key per key) and the table cache.
CPU/IO Background compaction competes with foreground I/O. Provision for your write rate to avoid write stalls (§4.1).

LevelDB has been validated up to ~1 billion 100-byte entries (~100 GB).


10. Operations

10.1 Backups

Because SSTables are immutable and the MANIFEST/CURRENT indirection is atomic, a backup is fundamentally "copy the directory consistently."

  • [ ] Quiesce writes, or take a Snapshot to pin a consistent read view for an online copy.
  • [ ] Copy the entire database directory (all *.ldb, *.log, MANIFEST-*, CURRENT, and the info logs).
  • [ ] Do not copy the LOCK file's lock state — copying the file is fine, but the destination must not be opened by two processes.
  • [ ] Validate the copy by opening it read-mostly in a throwaway process.

Tip

The immutability of SSTables makes incremental/file-level backup feasible: unchanged *.ldb files between two backups are byte-identical and need not be re-copied.

10.2 Inspecting files

Use the bundled leveldbutil tool to dump file contents in text form:

leveldbutil dump /path/to/db/000123.ldb
leveldbutil dump /path/to/db/MANIFEST-000045
Usage: leveldbutil command...
   dump files...         -- dump contents of specified files

dump is the only subcommand. It is implemented by routing each file through DumpFile (include/leveldb/dumpfile.h) into a StdoutPrinter.

10.3 Reading internal stats at runtime

Expose operational state through DB::GetProperty (see User Guide §8.5). For dashboards, scrape:

  • leveldb.stats — per-level file counts, sizes, read/write/compaction throughput.
  • leveldb.num-files-at-level0 — the leading indicator of impending write stalls (watch it against the triggers at 4/8/12).
  • leveldb.approximate-memory-usage — memory pressure from memtables, caches, and table cache.

10.4 Reclaiming space / triggering compaction

db->CompactRange(nullptr, nullptr);   // compact everything

Use after large deletions to force tombstones to be dropped and disk space reclaimed sooner than background compaction would.

10.5 Decommissioning a database

leveldb::DestroyDB("/path/to/db", leveldb::Options());

Removes the directory's contents. Ensure the database is closed (handle deleted) first.


11. Security Analysis

LevelDB's threat surface is unusual: it has no network listener, no authentication, no users, and no configuration files. It is a library that processes data your program hands it, plus the bytes already on disk. The relevant risks are therefore about untrusted input and operational handling, not remote attackers.

11.1 Trust boundaries

Boundary Risk
Untrusted database files on disk A crafted/corrupted SSTable, log, or MANIFEST can trigger parsing bugs. Opening attacker-supplied LevelDB directories is the main exposure.
Untrusted keys/values at the API Arbitrary byte arrays are accepted by design; the risk is in your code's handling, not LevelDB's.
Process privilege The DB runs with your process's privileges and writes wherever the path points; LevelDB has no sandbox.

11.2 Defensive features in the engine

  • CRC32C checksums on all stored data; verify_checksums (per-read) and paranoid_checks (per-DB) opt into verification/fail-fast.
  • Self-resynchronizing log format that bounds the blast radius of a corrupt log region to a 32 KB block.
  • Single-writer file lock preventing two processes from racing on the same directory.
  • Exceptions/RTTI disabled, reducing surprising control-flow across the library boundary.

11.3 Community-reported vulnerabilities and hardening notes

The public issue tracker contains several reports concerning parsing of malformed on-disk data — exactly the untrusted-file boundary above. As of commit 7ee830d the following are open and worth knowing for operators who might open untrusted databases:

Issue Class Operational mitigation
#1310 — ReadBlock block-size wraparound bypasses truncated-block validation → out-of-bounds access OOB read on crafted SSTable Do not open untrusted DBs; enable paranoid_checks + verify_checksums
#1318 — Unbounded allocation in ReadBlock via crafted SSTable block handle DoS (memory) Same; treat external DB files as hostile
#1311 — Bad internal key in MANIFEST crashes DB open Crash on open Validate provenance of MANIFEST; keep backups
#1298 — SIGSEGV in SkipList::FindGreaterOrEqual Crash Track upstream fix; isolate the process
#707 — Memory leak in status.cc enabling DoS Resource exhaustion Monitor memory; recycle long-lived processes
#1325 — UB: memcpy reads padding bytes (flexible array member) Undefined behavior Build with sanitizers in CI; pick up upstream fix
#1244 — WAL write succeeds but memtable insert fails → possible inconsistency Durability/consistency Use sync=true for critical writes; validate after crashes

Caution

Do not open LevelDB databases from untrusted sources. The parser was written for trusted, locally-produced files. The open issues above are concentrated in the SSTable/MANIFEST read paths, where adversarial input is the trigger. If you must process foreign data, do it in a sandboxed, resource-limited, restartable process with paranoid_checks and verify_checksums enabled — and keep up with upstream fixes.

Note

LevelDB receives "very limited maintenance" (per the README): only critical bug fixes (data loss/memory corruption) and changes needed by Google-internal clients are accepted. Plan your patching strategy accordingly — you may need to carry local fixes.

11.4 Secrets and data-at-rest

LevelDB does not encrypt data at rest and has no notion of secrets. If you store sensitive values, encrypt them in your application before Put, or place the database directory on an encrypted filesystem. File permissions on the database directory are governed by your process's umask and the Env.


12. Repository Structure & Module Reference

leveldb/
  include/leveldb/   Public API headers (the ONLY supported interface)
  db/                Database engine: DBImpl, memtable, WAL, versions, recovery
  table/             SSTable read/write, blocks, filters, iterators
  util/              Primitives: cache, arena, coding, crc32c, comparator, Env impls
  port/              Platform abstraction (port_stdcxx.h, thread annotations)
  helpers/memenv/    In-memory Env (testing helper; not public)
  benchmarks/        db_bench and comparison benchmarks
  issues/            Regression tests for specific GitHub issues
  doc/               Design docs: index.md, impl.md, table_format.md, log_format.md
  cmake/             Install/package config templates
  CMakeLists.txt     Build definition

12.1 include/leveldb/ — the public API (15 headers)

Header Contents
db.h DB (Open/Put/Get/Delete/Write/iterators/snapshots/properties/compact), DestroyDB, RepairDB, version constants — start here
options.h Options, ReadOptions, WriteOptions, CompressionType
slice.h Slice (pointer+length view)
status.h Status (success/error result type)
comparator.h Comparator interface, BytewiseComparator()
iterator.h Iterator interface, empty/error iterators
write_batch.h WriteBatch for atomic multi-op writes
cache.h Cache interface, NewLRUCache()
filter_policy.h FilterPolicy interface, NewBloomFilterPolicy()
env.h Env, file/logger abstractions, EnvWrapper
table.h, table_builder.h Lower-level SSTable access (most clients don't use directly)
dumpfile.h DumpFile() used by leveldbutil
c.h Complete C bindings (stable ABI)
export.h LEVELDB_EXPORT visibility macro

12.2 db/ — engine internals

File Role
db_impl.{h,cc} The DBImpl orchestrator: write queue, batch grouping, memtable flush, compaction scheduling, recovery, snapshots
dbformat.{h,cc} Internal key encoding, config:: constants, LookupKey, comparators
memtable.{h,cc} In-memory sorted buffer over a SkipList + Arena
skiplist.h Templated lock-free-read skip list (kMaxHeight=12, branching 1-in-4)
log_writer.*, log_reader.*, log_format.h WAL encode/decode
version_set.{h,cc}, version_edit.{h,cc} The level/file model, MVCC versions, MANIFEST application, compaction selection
table_cache.{h,cc} LRU of open SSTables
builder.{h,cc} BuildTable — flush an iterator to an SSTable
dumpfile.cc, leveldbutil.cc File dumper + CLI entry point
repair.cc RepairDB salvage logic
filename.{h,cc} Database file naming/parsing
write_batch.cc, write_batch_internal.h Batch serialization and memtable insertion
db_iter.{h,cc}, snapshot.h User-facing iterator over internal keys; snapshot list

12.3 table/ and util/

  • table/ — table.cc/table_builder.cc (SSTable I/O), block.cc/block_builder.cc (block format with restart points), filter_block.cc (filter meta block), format.cc (BlockHandle/Footer), merger.cc/two_level_iterator.cc/iterator.cc (iterator composition).
  • util/ — cache.cc (sharded LRU), arena.cc (bump allocator for memtable nodes), coding.cc (varint/fixed encoding), crc32c.cc, comparator.cc (bytewise default), bloom.cc (Bloom filter), hash.cc, logging.cc, status.cc, options.cc (defaults), env.cc + env_posix.cc/env_windows.cc (the two platform Envs), no_destructor.h, mutexlock.h, random.h.

13. Code Patterns & Conventions

LevelDB is a compact, idiomatic C++17 codebase. Patterns an integrator or contributor should recognize:

  • Status-return error handling. No exceptions (the build disables them). Every fallible call returns Status; the codebase threads if (s.ok()) checks everywhere.
  • Slice-based zero-copy. Keys/values move as Slice (pointer+length) to avoid copies; ownership of backing storage is the caller's responsibility.
  • Pimpl / opaque reps. Table, TableBuilder, and others hide a private struct Rep, allowing ABI-stable internal changes.
  • Reference counting for shared lifetimes: MemTable::Ref/Unref, Version::Ref/Unref keep objects alive while live iterators or background work reference them.
  • Arena allocation. The memtable allocates skip-list nodes from an Arena (bump allocator); nodes are never individually freed — the arena is released wholesale.
  • Explicit memory ordering. The skip list uses std::memory_order_acquire/release for lock-free reads with a single writer; "NoBarrier" variants are used where a subsequent publish provides the barrier.
  • Thread-safety annotations. GUARDED_BY, EXCLUSIVE_LOCKS_REQUIRED, LOCKABLE (in port/thread_annotations.h) enable Clang's -Wthread-safety static checking.
  • NoDestructor<T> wraps singletons (e.g. the default comparator) to avoid global-destructor ordering problems.
  • Style. Google C++ Style Guide; enforced with clang-format -i --style=file (config in .clang-format).

14. Testing Strategy

LevelDB is heavily tested with GoogleTest (built when LEVELDB_BUILD_TESTS=ON).

14.1 Test categories

Category Examples
Engine integration db/db_test.cc, db/autocompact_test.cc, db/recovery_test.cc, db/corruption_test.cc, db/fault_injection_test.cc
Format/units db/dbformat_test.cc, db/log_test.cc, db/skiplist_test.cc, db/version_edit_test.cc, db/version_set_test.cc, db/write_batch_test.cc, db/filename_test.cc
Table layer table/table_test.cc, table/filter_block_test.cc
Utilities util/arena_test.cc, util/bloom_test.cc, util/cache_test.cc, util/coding_test.cc, util/crc32c_test.cc, util/hash_test.cc, util/logging_test.cc, util/status_test.cc, util/no_destructor_test.cc
Platform Env util/env_posix_test.cc / util/env_windows_test.cc (selected per platform)
C ABI db/c_test.c — exercises the full C binding end-to-end
Regression issues/issue178_test.cc, issue200_test.cc, issue320_test.cc — pin fixes for specific reported bugs

A combined leveldb_tests target aggregates many of these; per-file targets are registered via the leveldb_test() CMake helper and run through ctest.

Note

Some tests build only for static library builds (NOT BUILD_SHARED_LIBS) because they reach into internal symbols not exported from the shared library. The C ABI test and a couple of others build in both modes.

14.2 Running the tests

cd build
ctest --output-on-failure
100% tests passed, 0 tests failed

14.3 Benchmarking

db_bench (built with LEVELDB_BUILD_BENCHMARKS=ON) measures fill/read/overwrite throughput; comparison harnesses db_bench_sqlite3 and db_bench_tree_db build only when SQLite3 / KyotoCabinet are detected. The README publishes a reference run (e.g. fillseq ~1.77 µs/op, readrandom ~16.7 µs/op cold) as a ballpark.


15. Known Issues & Potential Improvements

15.1 Maintenance posture

The repository is in limited-maintenance mode: only critical (data loss / memory corruption) fixes and changes required by Google-internal clients are accepted, and build-system contributions are generally declined. Integrators should plan to carry local patches if they need fixes that fall outside that scope.

15.2 Open correctness/security items (commit 7ee830d)

The parser-hardening issues in §11.3 are the most operationally significant: out-of-bounds reads and unbounded allocation on crafted SSTables (#1310, #1318), crash-on-open from a bad MANIFEST key (#1311), a SkipList SIGSEGV (#1298), a status.cc memory leak (#707), memcpy UB on a flexible array member (#1325), and a WAL-succeeds/memtable-fails consistency concern (#1244). Several are explicitly flagged as security/fuzzing findings (e.g. OSS-Fuzz #1006).

15.3 Self-identified improvement opportunities (from source TODOs/docs)

  • Stats meta block is unfinished — doc/table_format.md lists TODO(postrelease) stats (data/index/key/value sizes, entry/block counts) not yet recorded.
  • Compaction throttling heuristics — doc/impl.md proposes raising the log-switch threshold or artificially limiting write rate when level-0 files pile up, and reducing wide-merge cost — areas with room for tuning.
  • File-count strategy — the docs question whether sharding the file set across directories is still needed on modern filesystems, suggesting the fixed 2 MB file size could be revisited.
  • Skip-list insert optimization — a TODO(opt) notes Insert() could use a barrier-free FindGreaterOrEqual since it is externally synchronized.

15.4 Recommendations for integrators

  • [ ] Pin a specific commit (vendoring) given the slow release cadence.
  • [ ] Enable verify_checksums/paranoid_checks when ingesting any data of uncertain provenance.
  • [ ] Build CI with ASan/UBSan to catch the known UB classes early.
  • [ ] Monitor leveldb.num-files-at-level0 to anticipate write stalls.
  • [ ] Use sync=true (or batched synchronous writes) for durability-critical data.
  • [ ] Encrypt sensitive values at the application layer — there is no at-rest encryption.

16. Quick Reference

16.1 On-disk constants

Constant Value Source
Levels kNumLevels = 7 db/dbformat.h
L0 compaction trigger kL0_CompactionTrigger = 4 db/dbformat.h
L0 slowdown trigger kL0_SlowdownWritesTrigger = 8 db/dbformat.h
L0 stop trigger kL0_StopWritesTrigger = 12 db/dbformat.h
Max memtable push level kMaxMemCompactLevel = 2 db/dbformat.h
Read-sample period kReadBytesPeriod = 1 MB db/dbformat.h
Log block size 32768 bytes db/log_format.h
Log header size 7 bytes db/log_format.h
SSTable footer magic 0xdb4775248b80fb57 table/format.cc
Filter base 2 KB doc/table_format.md
Skip-list max height 12 db/skiplist.h
Max sequence number (1<<56)-1 db/dbformat.h

16.2 Default option values

Option Default
create_if_missing / error_if_exists / paranoid_checks false
write_buffer_size 4 MB
max_open_files 1000
block_size 4 KB
block_restart_interval 16
max_file_size 2 MB
compression kSnappyCompression
zstd_compression_level 1
internal block cache (if none supplied) 8 MB
ReadOptions::verify_checksums false
ReadOptions::fill_cache true
WriteOptions::sync false

16.3 File types in a database directory

CURRENT, MANIFEST-NNNNNN, LOCK, *.log, *.ldb (legacy *.sst), LOG, LOG.old, *.dbtmp.

16.4 CMake options

Option Default
LEVELDB_BUILD_TESTS ON
LEVELDB_BUILD_BENCHMARKS ON
LEVELDB_INSTALL ON
BUILD_SHARED_LIBS OFF

16.5 The one CLI command

leveldbutil dump <files...>