How it works

rubberduck combines Cloudflare services with DuckDB. Your application sends SQL to a rubberduck API.

Services

Service	What it is	What it does in rubberduck
Workers	Cloudflare's serverless runtime.	Receives API requests, checks auth, routes queries, and handles ingest.
D1	Cloudflare's managed serverless SQLite database.	Stores application and control data: orgs, API keys, and query placement. It is not the analytical store.
R2	Cloudflare's object storage service.	Holds each org's durable Parquet and CSV files. The data stays here as compute changes.
DuckDB	An analytical SQL engine.	Reads the files in R2 and runs filters, joins, and aggregations.
Containers	Cloudflare's container runtime.	Runs native DuckDB for the normal interactive path, with more memory than the Worker path.
DuckDB-WASM	DuckDB compiled to WebAssembly.	Runs small queries inside a Worker without waking a Container.

Request flow

YOUR APP
   │
   ├─ ingest ──► Worker ──► buffer ──► files in R2
   │
   └─ SQL ─────► Worker (auth + route)
                   ├─ small   → DuckDB-WASM
                   ├─ normal  → Container + DuckDB
                   └─ huge    → R2 SQL (when configured)

The Worker uses D1 to look up the org and its placement. It then sends the query to the selected engine. Every path reads the same files in R2, so stepping up compute does not require copying the data.

Words

Word	Meaning
Org	A tenant. Owns a folder in R2 (org_acme/…).
Duckling	One compute slot for an org. Place it on WASM or a Container.
Placement	Which engine runs the duckling. Does not move files.

Data layout

Bucket: rubberduck-analytics

org_acme/
  events/part-000.parquet
  customers/part-000.parquet

Samples (org_tpch, org_clickbench) use that layout. R2 Data Catalog is on for the bucket; Iceberg ATTACH needs a catalog token and real tables.

Query

curl -s -X POST https://rubberduck.b-christopher-3rd.workers.dev/v1/orgs/org_tpch/query \
  -H 'authorization: Bearer demo-token' \
  -H 'content-type: application/json' \
  -d '{"sql":"SELECT count(*) FROM lineitem"}'

• Normal: Container + native DuckDB
• Small: DuckDB-WASM (~64 MB ceiling; fine for SF0.1 / ~1M-row demos)
• Huge: "engine":"r2sql" once R2_SQL_TOKEN is set

Need more headroom? Set placement to container. Same R2 folder. See Scaling.

Ingest

• POST /v1/orgs/:orgId/ingest: stage rows, flush CSV to R2, compact
• POST /v1/orgs/:orgId/events: Pipelines path; needs EVENT_PIPELINE bound

Auth

Demo bearer demo-token, or per-org API keys. AUTH_MODE=prod turns off the demo token. SQL policy blocks dangerous statements and cross-org paths.

Gaps

Pipelines and R2 SQL need bindings / tokens. Iceberg ATTACH needs R2_CATALOG_TOKEN. Container cache clears on sleep. See Limits and Roadmap.