How do you get an AI system safe, reliable access to a customer's data without moving everything into a new store?
Answer it out loud first, then compare. That's what makes it stick.
I pick the lightest pattern that fits. For live lookups I use read-only access through their existing APIs or a read replica, so data stays put. For search or analytics I sync only the fields I need, incrementally, into an index they own. Every connector uses a narrow service account, secrets in their vault, and idempotent, retryable jobs. I log what was read, and I never copy sensitive columns I do not need.
- Query in place vs sync a copy: decide per use case
- Read-only, least-privilege service accounts
- Incremental sync with cursors, idempotent writes
- Secrets in the customer's vault
- Minimize fields, log access, define deletion
The question tests whether you know the difference between access and ownership of data.
Pick a pattern per use case
- Query in place (tool or API call at request time). Best for fresh, structured data such as an order status. Use their existing API or a read replica so you never load the production primary. Latency depends on their system, so set timeouts and caches.
- Sync to an index. Needed for semantic search or heavy analytics. Pull only required fields, store the source ID and last-modified time, and rebuild if the schema changes.
- Event stream (CDC or webhooks). Best when changes must appear within seconds; costs more to operate.
- Batch export. Simple, acceptable when daily freshness is fine.
Access hygiene
- One service account per integration, read-only, scoped to the tables or folders needed. Never a shared admin login.
- Credentials come from the customer's secret manager and rotate; nothing in code or prompts.
- Network path is private (VPC peering, private endpoints, or an on-prem agent that makes outbound-only calls).
Reliability
- Incremental cursors (updated_at or a change token) so a failed run resumes rather than restarts.
- Idempotent upserts keyed on source ID, so retries do not duplicate rows.
- Rate-limit yourself against the source system; a nightly crawl that slows their ERP will end the pilot.
- Dead-letter queue for records that fail parsing, reviewed daily.
Governance
- Keep a data map: field, source, purpose, retention. Drop columns you do not use; a sensitive field you never copied cannot leak.
- Define deletion: when a source record is removed, the derived chunk and embedding must be removed too.
Trade-off statement
In-place access is safest and freshest but slow and dependent on their uptime; a synced copy is faster and richer but adds a second place that must be secured, kept current and deleted.
- Their database is on-prem with no inbound access. How do you connect?
- How do you propagate a record deletion to your index?
- The source system slows down during your sync. What do you change?
- Forward Deployed Engineer System Design Interview (prep, 2026) Mentions legacy-source reads, schema normalization, idempotent writes and pipeline monitoring as themes.
- Generative AI model inference (AWS Security Reference Architecture) (docs, unknown) Supports private connectivity patterns for model inference using PrivateLink.