Real-time
Two-way sync
Changes in Elasticsearch or Google Pubsub instantly reflect in both systems. No stale data, no manual imports.
Database ⇄ Developer tools
Elasticsearch to Google Pubsub integration — real-time, two-way sync
Keep Elasticsearch and Google Pubsub in sync without custom scripts. Cut weeks of integration work, eliminate silent data drift, and give your team a single, reliable source of truth.
- SOC 2 and 6 other compliance frameworks
- POC with real engineers in minutes
Adopted by fast-scaling companies moving mission-critical data in real time
Case study
Migrated from MuleSoft
Case study
Migrated from Celigo
Migrated from Heroku Connect
Migrated from Matillion
Case study
Migrated from Fivetran
Case study
Migrated from Celigo
Why teams connect Elasticsearch and Google Pubsub
Keep Elasticsearch and Google Pubsub in step: the rows in your database and the Message attributes, Dead-letter topics, Ordering keys, Topics your engineering tools track stay consistent in real time, in both directions.
Elasticsearch is where your application's durable data lives; Google Pubsub is where engineering and operations teams track the issues, events, messages, or identities that run alongside it. The two overlap constantly, a row should open a ticket, an alert should land as a record, a user in one should exist in the other, but bridging them today means per-tool integration code: auth, webhooks, pagination, rate limits, and retries, built and maintained separately for every tool.
Stacksync syncs Documents, Index mappings, Aliases, Data streams in Elasticsearch with Message attributes, Dead-letter topics, Ordering keys, Topics in Google Pubsub field by field, in real time, and in both directions. You decide which system owns which fields; Stacksync keeps every copy consistent and resolves conflicts by rules you set, so the database and the tooling around it never drift apart.
Common use cases
- 01 Sync CRM accounts and contacts into an Elasticsearch index to power internal search across customer records.
- 02 Push product catalog data from an ERP or commerce database into Elasticsearch for storefront search.
- 03 Attach an Avro or Protocol Buffer schema to a topic so every message Stacksync publishes is validated against an agreed contract.
- 04 Publish new and changed database rows as messages to a Pub/Sub topic so multiple downstream subscriptions consume the change stream independently.
Common sync patterns
React to changes on either side in near real time
Updates in Google Pubsub arrive as row changes in Elasticsearch, and writes to Elasticsearch propagate to Google Pubsub within seconds, so triggers, jobs, and alerts fire without polling.
Where Google Pubsub manages users or groups: keep identity aligned
Directory and identity records in Google Pubsub stay matched to the users or owners table in Elasticsearch, so provisioning and de-provisioning flow from one source.
Turn rows into the records your tools track
A new or changed row in Elasticsearch creates or updates the matching record in Google Pubsub, whether that is an issue, an event, a message, or a user, so the tool reflects the database without a custom API job.
What you can sync between Elasticsearch and Google Pubsub
Representative objects on each side — any object or custom field can map to any target. Schemas are auto-detected; types are converted between the two systems.
| Elasticsearch objects | Google Pubsub objects | How this pairing syncs | |
|---|---|---|---|
| Indices Target containers for synced records; each holds a table-like collection of JSON documents. | Topics Named resource publishers send to; Stacksync publishes each record change as a message to a topic for downstream subscribers to consume. | Indices is specific to Elasticsearch and Topics to Google Pubsub — each maps to any object or custom field on the other side. | |
| Documents The unit of sync; JSON records created, updated, and deleted by _id. | Subscriptions A stream of messages from one topic; Stacksync consumes here via StreamingPull or a push endpoint, acknowledging each message after a successful write. | Documents is specific to Elasticsearch and Subscriptions to Google Pubsub — each maps to any object or custom field on the other side. | |
| Index mappings Field type definitions that determine how synced fields are indexed and queried. | Messages The synced unit: base64-encoded data plus attributes, ordering key, messageId and publishTime; capped at 10 MB each. | Index mappings is specific to Elasticsearch and Messages to Google Pubsub — each maps to any object or custom field on the other side. | |
| Aliases Stable read/write names that let a sync cut over between index versions without downtime. | Schemas Avro or Protocol Buffer definitions bound to a topic; validate that every published message matches the agreed structure. | Aliases is specific to Elasticsearch and Schemas to Google Pubsub — each maps to any object or custom field on the other side. | |
| Data streams Append-only targets for time-series or event data pushed from source systems. | Snapshots Captured subscription state for seek/replay; lets already-acknowledged messages be redelivered from a point in time. | Data streams is specific to Elasticsearch and Snapshots to Google Pubsub — each maps to any object or custom field on the other side. | |
| Ingest pipelines Server-side transforms applied to documents as a sync writes them. | Message attributes Up to 100 key-value pairs per message; carry routing metadata and drive subscription filter expressions. | Ingest pipelines is specific to Elasticsearch and Message attributes to Google Pubsub — each maps to any object or custom field on the other side. |
How changes propagate between Elasticsearch and Google Pubsub
Each direction of the sync is driven by what the source system can signal and what the destination accepts — detection, delivery, and expected latency below.
Elasticsearch Google Pubsub Interval-based propagation
DetectionStacksync polls Elasticsearch for changes on an incremental schedule, reading only records changed since the previous pass. Polling on timestamp or sequence fields.
DeliveryEach detected change is written to Google Pubsub through its API, with automatic retries and rate-limit backoff.
Google Pubsub Elasticsearch Sub-second propagation
DetectionGoogle Pubsub notifies Stacksync of record changes through webhook events. Consumes messages as they arrive on a subscription — StreamingPull (long-lived gRPC) or a push subscription delivering each message as an HTTPS POST.
DeliveryEach detected change is written to Elasticsearch through its API, with automatic retries and rate-limit backoff.
Rate-limit considerations
- Elasticsearch: No fixed request quota; throughput is bounded by cluster sizing, thread pools, and bulk queue capacity.
- Google Pubsub: Per-region throughput quotas (publish up to ~4 GB/s in large regions); 10 MB max per message and per publish request, 1,000 messages per request, and 1 MBps per ordering key.
What ships with Elasticsearch ⇄ Google Pubsub
Connect Elasticsearch and Google Pubsub for flexible, real-time data sync.
Real-time sync, workflow automation, event queues, EDI, and monitoring, for every Elasticsearch–Google Pubsub connection.
No-code + pro-code
Workflow automation
Trigger automated workflows whenever Elasticsearch or Google Pubsub data changes, update records, fire webhooks, or kick off sequences without brittle API scripts.
At scale
Event queues
Handle millions of events per minute without losing a single Elasticsearch or Google Pubsub record.
Observability
Monitoring
Track your Elasticsearch ⇄ Google Pubsub sync health, view errors, and replay failed events in one click.
Trading partners
EDI
Transform legacy EDI complexity into simple database interactions between Elasticsearch and Google Pubsub.
How the Elasticsearch and Google Pubsub connectors work
Elasticsearch
- Integration surface
- REST API (JSON over HTTP)
- Authentication
- API keys or basic authentication; Elastic Cloud also issues service account tokens
- Change detection
- Polling on timestamp or sequence fields; Elasticsearch does not expose a native change feed or webhooks
- Capabilities
- read · write
- Rate limits
- No fixed request quota; throughput is bounded by cluster sizing, thread pools, and bulk queue capacity
Google Pubsub
- Integration surface
- REST and gRPC (Cloud Pub/Sub API v1)
- Authentication
- Google Cloud IAM via OAuth 2.0 / service-account credentials (JSON key or workload identity); requires roles such as pubsub.publisher and pubsub.subscriber
- Change detection
- Consumes messages as they arrive on a subscription — StreamingPull (long-lived gRPC) or a push subscription delivering each message as an HTTPS POST; no modified-date polling
- Capabilities
- read · write · webhooks
- Rate limits
- Per-region throughput quotas (publish up to ~4 GB/s in large regions); 10 MB max per message and per publish request, 1,000 messages per request, and 1 MBps per ordering key
How it works
How to connect Elasticsearch to Google Pubsub — three steps, no code
Configure and sync within minutes, no code. Whether you sync 50k or 100M+ records, Stacksync handles the queues, infra, and plumbing. Integrations are non-invasive and need zero setup on your systems.
- 01
Connect your apps
Authenticate Elasticsearch and Google Pubsub with each platform's native method — OAuth, API keys, or service accounts — plus secure options like SSH tunneling, IP whitelisting, and VPC peering.
Elasticsearch
connected
Google Pubsub
connected
OAuth 2.0SSH tunnelSSL certificateVPC peering - 02
Choose tables
Pick the Elasticsearch and Google Pubsub objects to sync — Stacksync auto-detects both schemas, including custom fields where the platform exposes them. Sync to existing tables, or let Stacksync create new ones with ideal data types.
objects · Elasticsearch ⇄ Google PubsubCustomers 12,480Sales Orders 8,213Invoices 5,902Items 1,344 - 03
Map fields
Fields map automatically even when names and types differ. Stacksync handles transformation and type casting for you, zero configuration required.
Elasticsearch Google PubsubCompany company_name textEmail email textAmount amount numericCreated created_at timestamp
FAQ
Elasticsearch and Google Pubsub integration FAQ
-
Yes. Stacksync provides a managed, real-time two-way integration between Elasticsearch and Google Pubsub: authenticate both systems, choose the objects to sync (such as Elasticsearch's Indices and Documents), map fields visually, and changes propagate both ways in milliseconds — no code required.
-
On the Elasticsearch side: Documents, Index mappings, Aliases, Data streams, plus custom fields where Elasticsearch exposes them. On the Google Pubsub side: Message attributes, Dead-letter topics, Ordering keys, Topics. Stacksync auto-detects both schemas and converts types between the two systems.
-
Yes. Each object mapping can be bidirectional or restricted to a single direction (both systems accept writes). Read-only mirrors, one-way pushes, and full two-way sync can be mixed in the same integration.
-
Common patterns for Elasticsearch and Google Pubsub: React to changes on either side in near real time; Where Google Pubsub manages users or groups: keep identity aligned; Turn rows into the records your tools track. Updates in Google Pubsub arrive as row changes in Elasticsearch, and writes to Elasticsearch propagate to Google Pubsub within seconds, so triggers, jobs, and alerts fire without polling.
-
Elasticsearch: REST API (JSON over HTTP). Authentication: API keys or basic authentication; Elastic Cloud also issues service account tokens. Google Pubsub: REST and gRPC (Cloud Pub/Sub API v1). Authentication: Google Cloud IAM via OAuth 2.0 / service-account credentials (JSON key or workload identity); requires roles such as pubsub.publisher and pubsub.subscriber. Stacksync manages authentication, retries, and rate limits on both sides.
-
Elasticsearch: Writes are addressed by document _id, so upserts map directly onto the index API, and the _bulk endpoint batches many operations in a single request. Google Pubsub: Default message retention is 7 days, configurable up to 31 days; unacknowledged messages are redelivered until acked or the retention window expires. Stacksync's field mapping accounts for these differences between Elasticsearch and Google Pubsub without custom code.
SECURITY
Security teams trust Stacksync
As a data company, we understand the importance of keeping your data secure. Stacksync is built with security best practices to keep your data safe at every layer, and is DPF-certified for US, EU, UK and CH data transfers.
SOC 2 Type II
ISO 27001
HIPAA BAA
GDPR
CCPA
DPF US-EU-UK-CH
→ SECURITY WITH BENEFITS
SSO & SCIM
Let your users access Stacksync from your centralized user management systems. Works with Okta, Azure, Google SSO and more.
Alerts
Immediately get alerted about record syncing issues over email, Slack, PagerDuty and WhatsApp. Resolve issues from a centralized dashboard with retry and revert options.
Secure connection options
Securely connects to your systems with:
Related integrations
Every pair below is a real-time, two-way sync. Search all 332 integrations available for Elasticsearch and Google Pubsub.
Popular · 5 of 332
-
Elasticsearch ⇄ Jms -
Elasticsearch ⇄ Newrelic -
Elasticsearch ⇄ Opsgenie -
Elasticsearch ⇄ PagerDuty -
Google Pubsub ⇄ Neo4j
No integrations match .
