PostgreSQL

Load data from PostgreSQL into CrateDB.

Install

uv tool install --upgrade 'cratedb-toolkit[io-ingest]'

Synopsis

Invoke full-load operation.

ctk load \
    "postgresql://pguser:secret11@postgresql.example.org:5432/postgres?table=information_schema.tables" \
    "crate://crate:na@localhost:4200/testdrive/postgresql_tables"

Options

Batch size

The source URL accepts the batch_size option to configure pagination. For maximum throughput, you will need to experiment to find an optimal value. The default value is mostly 50_000, but might be different for individual adapters.

ctk load "protocol://?batch_size=5000"

Custom queries

The source URL accepts the ?table=query:<expression> option that can be used for partial table loading by using an SQL query expression on the source database. For example, limit the import to use just a subset of the source columns, a subset of all records, or for any other aggregation queries. See also custom queries for SQL sources. Example:

ctk load "protocol://?table=query:SELECT * FROM sys.summits WHERE height > 4000"