What We Learned Shipping the First Postgres Connector

Reading column types is easy. Reading column types and primary keys and foreign keys and nullability — and getting all of them consistently across Postgres versions — turned out to be the bulk of the work. Every assumption we made about the schema's stability was wrong at least once. We now run three separate queries to information_schema and reconcile them in code.

Our first cursor implementation was WHERE updated_at > last_seen. Worked beautifully on the demo dataset. Broke immediately against real Postgres because applications backdate inserts — a row arriving at 14:05 might have an updated_at of 14:04:30, which our sync had already advanced past. We now optionally pull a configurable overlap window on each poll. The cursor advances, but slightly slower than wall-clock time.

Every “this happened at time T” timestamp in a production database is lying by some amount. The connector either accommodates that or silently loses data.

It's tempting to format the cursor value straight into the SQL string — even when the cursor is a timestamp you control. Don't. The corner cases that break are a long list: special characters in timestamps, locale-sensitive formatting, and injection through field names you thought were trusted. Bind every cursor value as a parameter, validate any identifier you have to interpolate against a strict allowlist, and accept the small rigidity that comes with it.

The most-asked question wasn't about cursor semantics or schema drift. It was about whether the password we asked them to paste into the wizard was stored safely. The answer — connector credentials are held encrypted at rest, not in plaintext — was already true; the fix was making that explicit in the UI. Once people could see it, the question stopped.

The connector interface is intentionally small: anything that's a cursor-based pull from a structured source can implement it. Today the PostgreSQL connector ships production-validated — it even auto-discovers foreign keys and turns them into graph connections — alongside a REST API connector. MySQL, MongoDB, S3/Parquet, and Kafka are on the roadmap, staged for later releases. The framework was the hard part; each subsequent source is mostly a small, well-bounded adapter. (Connectors are an Enterprise feature.)