Real-time
Two-way sync
Changes in DuckDB or Egnyte instantly reflect in both systems. No stale data, no manual imports.
Database ⇄ Storage
DuckDB to Egnyte integration — real-time, two-way sync
Keep DuckDB and Egnyte in sync without custom scripts. Cut weeks of integration work, eliminate silent data drift, and give your team a single, reliable source of truth.
- SOC 2 and 6 other compliance frameworks
- POC with real engineers in minutes
Adopted by fast-scaling companies moving mission-critical data in real time
Case study
Migrated from MuleSoft
Case study
Migrated from Celigo
Migrated from Heroku Connect
Migrated from Matillion
Case study
Migrated from Fivetran
Case study
Migrated from Celigo
Why teams connect DuckDB and Egnyte
Keep DuckDB and Egnyte in step: a row in DuckDB for every file in Egnyte, with names, paths, metadata, and status staying consistent in real time, in both directions.
A database holds structured records; a storage system holds the files those records depend on, such as contracts, images, exports, uploads, and documents. The two describe the same things from opposite sides: a row in DuckDB says a file exists and carries its name, location, and status, while Egnyte holds the bytes. Linked only by a hand-kept path or a one-off script, the two drift the moment a file is renamed, moved, or deleted and the record still points at where it used to be.
Stacksync syncs Views, External files (Parquet/CSV/JSON), Attached databases, Database files in DuckDB with Links, Metadata, Events and Audit reports, Files in Egnyte bi-directionally and in real time. File attributes, including name, path or object key, size, type, modified time, owner, and tags or custom properties, map field by field to columns on the matching row, and a change on either side shows up on the other within seconds. New files appear as rows, metadata edits travel in the direction you choose, and deletes stay consistent, with conflict rules you set in place of nightly reconciliation scripts.
Common use cases
- 01 Sync SaaS data to Parquet on object storage and query it with DuckDB without standing up a warehouse.
- 02 Push aggregates computed in DuckDB out to a CRM or business tools so analysis results reach operational systems.
- 03 Two-way sync of Files and Folders between Egnyte and an operational database or another content store, keeping the folder hierarchy aligned.
- 04 Write custom Metadata such as classification tags and project fields onto Files and Folders from a governance or DMS system, and read it back for reporting.
Common sync patterns
File metadata kept in step
Tags, custom properties, owner, or status columns edited on a row in DuckDB write back to the matching file's metadata in Egnyte, and metadata changed in Egnyte updates the row, so the two never disagree about a file.
Records that point at documents
Where a record in DuckDB references a file in Egnyte, such as a contract, an image, an export, or an upload, the reference, path, and status stay consistent as files are renamed, moved, or replaced, so stored links keep resolving.
A landing zone for incoming files
Files that arrive in a folder or bucket in Egnyte become rows in DuckDB as they land, so a database-driven process can pick them up without polling the storage system's API.
What you can sync between DuckDB and Egnyte
Representative objects on each side — any object or custom field can map to any target. Schemas are auto-detected; types are converted between the two systems.
| DuckDB objects | Egnyte objects | How this pairing syncs | |
|---|---|---|---|
| Attached databases Additional database files or external systems attached into one session for cross-source queries. | Users Accounts managed via the Users API; provisioned, updated, and deactivated in sync with an HRIS or identity provider. | Attached databases is specific to DuckDB and Users to Egnyte — each maps to any object or custom field on the other side. | |
| Database files Single-file .duckdb databases that jobs read and write directly on disk or object storage. | Groups Groups and membership managed via the Groups API; kept aligned with directory or IdP groups that drive folder access. | Database files is specific to DuckDB and Groups to Egnyte — each maps to any object or custom field on the other side. | |
| Schemas Namespaces within a database used to organize tables in sync outputs. | Permissions Folder permissions listed, set, and removed via the Permissions API; effective permission for a user/folder is computable. | Schemas is specific to DuckDB and Permissions to Egnyte — each maps to any object or custom field on the other side. | |
| Tables Columnar tables created via SQL; the destination for materialized sync data. | Links Shared file and folder links created, listed, and revoked via the Links API; useful for governing and auditing external sharing. | Tables is specific to DuckDB and Links to Egnyte — each maps to any object or custom field on the other side. | |
| Views SQL views used to shape or filter data for downstream consumers. | Metadata Custom metadata keys and project fields on files/folders via the Metadata API; written for classification and read back for reporting. | Views is specific to DuckDB and Metadata to Egnyte — each maps to any object or custom field on the other side. | |
| External files (Parquet/CSV/JSON) Files DuckDB queries in place without loading, common as a sync interchange format. | Events and Audit reports File, login, permission, and sharing activity read via the Events API and Audit Reporting API; consumed read-only for compliance. | External files (Parquet/CSV/JSON) is specific to DuckDB and Events and Audit reports to Egnyte — each maps to any object or custom field on the other side. |
How changes propagate between DuckDB and Egnyte
Each direction of the sync is driven by what the source system can signal and what the destination accepts — detection, delivery, and expected latency below.
DuckDB Egnyte Interval-based propagation
DetectionStacksync polls DuckDB for changes on an incremental schedule, reading only records changed since the previous pass. Polling or full re-reads.
DeliveryEach detected change is written to Egnyte through its API, with automatic retries and rate-limit backoff.
Egnyte DuckDB Sub-second propagation
DetectionEgnyte notifies Stacksync of record changes through webhook events. Events API polled with an incremental cursor (Egnyte recommends an interval of 5 minutes or more), or Webhooks push notifications for fs, metadata,.
DeliveryEach detected change is applied to DuckDB as a row-level write, with types converted between the two schemas.
Rate-limit considerations
- DuckDB: No API rate limits; throughput is bounded by local compute and I/O.
- Egnyte: Auto-generated keys default to 2 requests/second and 1,000 requests/day, enforced per access token; monitor X-Accesstoken-Qps-Current/Allotted and X-Accesstoken-Quota-Current headers; higher limits on request.
What ships with DuckDB ⇄ Egnyte
Connect DuckDB and Egnyte for flexible, real-time data sync.
Real-time sync, workflow automation, event queues, EDI, and monitoring, for every DuckDB–Egnyte connection.
No-code + pro-code
Workflow automation
Trigger automated workflows whenever DuckDB or Egnyte data changes, update records, fire webhooks, or kick off sequences without brittle API scripts.
At scale
Event queues
Handle millions of events per minute without losing a single DuckDB or Egnyte record.
Observability
Monitoring
Track your DuckDB ⇄ Egnyte sync health, view errors, and replay failed events in one click.
Trading partners
EDI
Transform legacy EDI complexity into simple database interactions between DuckDB and Egnyte.
How the DuckDB and Egnyte connectors work
DuckDB
- Integration surface
- In-process SQL engine via client libraries (Python, Node.js, JDBC, CLI); no server or network API by default
- Authentication
- None built in; access control is file-system level (MotherDuck adds token auth for its hosted service)
- Change detection
- Polling or full re-reads; no change feed or transaction log API
- Capabilities
- read · write
- Rate limits
- No API rate limits; throughput is bounded by local compute and I/O
Egnyte
- Integration surface
- REST Public API (pubapi/v1) with File System, Users, Groups, Permissions, Links, Metadata, Events, and Webhooks endpoints
- Authentication
- OAuth 2.0 scoped access tokens (Authorization Code or Implicit for partner apps, Resource Owner Password for internal apps, Refresh Token flow); token sent in the Authorization header
- Change detection
- Events API polled with an incremental cursor (Egnyte recommends an interval of 5 minutes or more), or Webhooks push notifications for fs, metadata, permission, and comment events
- Capabilities
- read · write · webhooks
- Rate limits
- Auto-generated keys default to 2 requests/second and 1,000 requests/day, enforced per access token; monitor X-Accesstoken-Qps-Current/Allotted and X-Accesstoken-Quota-Current headers; higher limits on request
How it works
How to connect DuckDB to Egnyte — three steps, no code
Configure and sync within minutes, no code. Whether you sync 50k or 100M+ records, Stacksync handles the queues, infra, and plumbing. Integrations are non-invasive and need zero setup on your systems.
- 01
Connect your apps
Authenticate DuckDB and Egnyte with each platform's native method — OAuth, API keys, or service accounts — plus secure options like SSH tunneling, IP whitelisting, and VPC peering.
DuckDB
connected
Egnyte
connected
OAuth 2.0SSH tunnelSSL certificateVPC peering - 02
Choose tables
Pick the DuckDB and Egnyte objects to sync — Stacksync auto-detects both schemas, including custom fields where the platform exposes them. Sync to existing tables, or let Stacksync create new ones with ideal data types.
objects · DuckDB ⇄ EgnyteCustomers 12,480Sales Orders 8,213Invoices 5,902Items 1,344 - 03
Map fields
Fields map automatically even when names and types differ. Stacksync handles transformation and type casting for you, zero configuration required.
DuckDB EgnyteCompany company_name textEmail email textAmount amount numericCreated created_at timestamp
FAQ
DuckDB and Egnyte integration FAQ
-
Yes. Stacksync provides a managed, real-time two-way integration between DuckDB and Egnyte: authenticate both systems, choose the objects to sync (such as DuckDB's Attached databases and Database files), map fields visually, and changes propagate both ways in milliseconds — no code required.
-
On the DuckDB side: Views, External files (Parquet/CSV/JSON), Attached databases, Database files, plus custom fields where DuckDB exposes them. On the Egnyte side: Links, Metadata, Events and Audit reports, Files. Stacksync auto-detects both schemas and converts types between the two systems.
-
Yes. Each object mapping can be bidirectional or restricted to a single direction (both systems accept writes). Read-only mirrors, one-way pushes, and full two-way sync can be mixed in the same integration.
-
Common patterns for DuckDB and Egnyte: File metadata kept in step; Records that point at documents; A landing zone for incoming files. Tags, custom properties, owner, or status columns edited on a row in DuckDB write back to the matching file's metadata in Egnyte, and metadata changed in Egnyte updates the row, so the two never disagree about a file.
-
DuckDB: In-process SQL engine via client libraries (Python, Node.js, JDBC, CLI); no server or network API by default. Authentication: None built in; access control is file-system level (MotherDuck adds token auth for its hosted service). Egnyte: REST Public API (pubapi/v1) with File System, Users, Groups, Permissions, Links, Metadata, Events, and Webhooks endpoints. Authentication: OAuth 2.0 scoped access tokens (Authorization Code or Implicit for partner apps, Resource Owner Password for internal apps, Refresh Token flow); token sent in the Authorization header. Stacksync manages authentication, retries, and rate limits on both sides.
-
DuckDB: It queries Parquet, CSV, and JSON files directly without importing them, which makes file-based exchange a natural sync pattern. Egnyte: The Events API returns changes only after a supplied cursor and recommends polling no more often than every 5 minutes; Webhooks are the path for near-real-time. Stacksync's field mapping accounts for these differences between DuckDB and Egnyte without custom code.
SECURITY
Security teams trust Stacksync
As a data company, we understand the importance of keeping your data secure. Stacksync is built with security best practices to keep your data safe at every layer, and is DPF-certified for US, EU, UK and CH data transfers.
SOC 2 Type II
ISO 27001
HIPAA BAA
GDPR
CCPA
DPF US-EU-UK-CH
→ SECURITY WITH BENEFITS
SSO & SCIM
Let your users access Stacksync from your centralized user management systems. Works with Okta, Azure, Google SSO and more.
Alerts
Immediately get alerted about record syncing issues over email, Slack, PagerDuty and WhatsApp. Resolve issues from a centralized dashboard with retry and revert options.
Secure connection options
Securely connects to your systems with:
Related integrations
Every pair below is a real-time, two-way sync. Search all 439 integrations available for DuckDB and Egnyte.
Popular · 7 of 439
-
DuckDB ⇄ Amazon S3 -
DuckDB ⇄ Box -
DuckDB ⇄ Sharepoint -
Egnyte ⇄ Firebase -
Egnyte ⇄ SingleStore -
Egnyte ⇄ Amazon Aurora -
Egnyte ⇄ Amazon Lightsail
No integrations match .
