Mar 22, 12:17:13 GMT+0
Investigating -
We are currently investigating this incident. The Connector Builder is currently returning 502 Server Errors and is inaccessible. .
Mar 22, 14:59:30 GMT+0
Resolved -
This incident has been resolved..
Mar 20, 18:35:00 GMT+0
Identified -
We have seen an elevated number of sync failures with the following error messages:
`server 502 error message` or `Client error : 409 {"message":"Non-terminal Workload with mutexKey`
Our team is working on a fix and we apologize for any inconvenience caused and appreciate your patience as we work to resolve this issue..
Mar 20, 21:18:23 GMT+0
Resolved -
We implemented a fix and this issue has been successfully resolved.
Syncs should now be starting and running normally. If your most recent sync failed, your next scheduled sync should succeed.
Thank you for your patience as we investigated this issue and we apologize for any inconvenience..
Mar 13, 22:49:43 GMT+0
Investigating -
We are currently investigating an issue that is currently causing an increase in sync failures. Thank you for your patience as we investigate this issue..
Mar 13, 22:59:25 GMT+0
Monitoring -
We implemented a fix and are currently monitoring the result.
Thank you for your patience as we work to resolve this issue..
Mar 13, 23:30:00 GMT+0
Resolved -
The issue has been successfully resolved. Thank you again for your patience as worked this issue to resolution and we apologize for any inconvenience this issue might have caused..
Mar 11, 00:05:00 GMT+0
Identified -
A thread leak caused a pod to fail, leading to a subset of syncs in the EU data plane to fail. .
Mar 11, 01:35:00 GMT+0
Resolved -
This incident has been resolved..
Mar 5, 20:49:16 GMT+0
Identified -
We are having an internal issue where our job pods cannot write their output in \`gcp-us-west3-0\` dataplane. As a result, syncs are failing. The team is currently working on a fix..
Mar 5, 21:22:44 GMT+0
Identified -
We have a workaround allowing new jobs succeed. We are working on a fix for gcp-us-west3-0 dataplane.
Mar 5, 23:12:28 GMT+0
Monitoring -
The team has provided a fix and reenabled the affected data plane and is currently monitoring the situation..
Mar 6, 00:13:56 GMT+0
Resolved -
The team has monitored the `gcp-us-west3-0` data plane and has determined the issue has been resolved..
Mar 1, 23:24:35 GMT+0
Investigating -
We noticed a large number of users with failing connections and are currently investigating this incident..
Mar 1, 23:36:25 GMT+0
Monitoring -
All current syncs should now be running as expected. We have rolled back our most recent deployment and are continuing to investigate the root cause of this issue..
Mar 2, 02:31:17 GMT+0
Resolved -
The team did some monitoring and provided a fix. We are no longer seeing any issues..
Feb 25, 03:30:00 GMT+0
Monitoring -
We have identified an issue that prevents sync in the European data plane from running. A fix has been deployed, and we are monitoring..
Feb 25, 05:17:49 GMT+0
Resolved -
This incident is resolved. EU syncs are operational.
Feb 19, 19:57:19 GMT+0
Investigating -
We are currently investigating an issue that is currently affecting syncs running in EU data residency. Thank you for your patience as we dig into the issue..
Feb 19, 20:27:06 GMT+0
Resolved -
This incident has been resolved..
Feb 14, 00:00:00 GMT+0
Resolved -
Elevated job error rates from 4pm - 4:10pm..
Feb 12, 22:45:00 GMT+0
Identified -
We notice wide spread sync failure on the replication of the `user_interest` stream of the Google Ads source connector. Other streams are correctly replicated at the moment.
Google Ads API returns` 500 HTTP error codes` which characterizes a problem in the availability of their service.
No incident is yet declared on [Google Ads status page ](https://ads.google.com/status/publisher/)
We will reach out to Google Ads support..
Feb 14, 23:09:40 GMT+0
Monitoring -
We have implemented a workaround to allow connections with the user\_interests stream enabled to succeed. The user\_interest stream will show null tables while the API is experiencing this outage. We will continue monitoring Googles API status. .
Feb 20, 16:28:39 GMT+0
Resolved -
This incident is resolved with a workaround.
After monitoring the workaround we have seen connections return to normal. Once the API issue is resolved from Google we will update to connector to include the data from the USER\_INTEREST table. .
Jan 30, 05:33:55 GMT+0
Identified -
We are currently investigating an issue causing an increase in sync failures with connections using the Postgres destination connector.
We have found the root cause of the issue and are working hard on a fix. Thank you for your patience as we work towards resolving this issue..
Jan 30, 06:48:30 GMT+0
Monitoring -
A fix has been implemented and we are currently monitoring the update.
Thank you for your patience as we work to resolve this issue..
Jan 30, 08:31:53 GMT+0
Resolved -
This incident has been resolved..
Jan 19, 22:02:18 GMT+0
Resolved -
We have identified an issue that is causing new workspaces to not be created for users. A fix has been pushed to prod and tested. .
Jan 4, 17:46:57 GMT+0
Resolved -
Access to the Developer Portal has been fully restored.
Thank you for your patience as we worked to get this taken care of!.
Jan 4, 16:17:16 GMT+0
Investigating -
We are currently investigating an issue that is currently affecting access to the Dev Portal. Some users are reporting `invalid-api-keys` errors.
Thank you for your patience as we dig into the issue. As soon as we know more we will update here..
Jan 3, 02:07:09 GMT+0
Identified -
We are seeing an increase of sync failures from the Instagram source connector with the error: This metric engagement is no longer supported on version v18+.
This is likely due to Instagram Graph API's recent deprecation of some metrics on all versions of their API. Please see [this changelog from Sept 12, 2023](https://developers.facebook.com/docs/instagram-api/changelog#september-12--2023) for more information as well as the full list of affected metrics.
Our team is currently investigating a potential fix and we apologize for any inconvenience caused and appreciate your patience as we work to resolve this issue..
Jan 3, 15:38:25 GMT+0
Identified -
We are seeing an increase of sync failures from the Instagram source connector with the error: This metric engagement is no longer supported on version v18+.
**A temporary workaround to return a connection to syncing successfully is to disable the `media_insights` stream on the connection's replication tab. Our team is currently working on a proper fix to return all streams to functioning.
This is due to Instagram Graph API's recent deprecation of some metrics on all versions of their API. Please see this changelog from Sept 12, 2023 for more information as well as the full list of affected metrics..
Jan 8, 17:52:21 GMT+0
Resolved -
The issue has been successfully resolved.
The Instagram connector has been upgraded to use the latest version of the Instagram API (v18) following the deprecation of v11. Please see [our migration guide](https://docs.airbyte.com/integrations/sources/instagram-migrations#upgrading-to-300) for instructions on how to update your connector to the latest version (v3.0.0).
Apologies for any inconvenience and thank you again for your patience as we worked to resolve this issue. Please reach out to Airbyte Support if you have questions or concerns..
Nov 30, 18:48:48 GMT+0
Investigating -
We are currently investigating this incident..
Nov 30, 18:51:36 GMT+0
Resolved -
This incident has been resolved..
Nov 18, 20:00:00 GMT+0
Monitoring -
We have identified an issue that causes syncs to be delayed.
We have already deployed a fix and are monitoring the Airbyte platform..
Nov 18, 21:00:00 GMT+0
Resolved -
This incident has been resolved..
Nov 17, 14:51:23 GMT+0
Investigating -
We are currently investigating this incident where OAuth Connectors are unable to be set up. Users will see the OAuth button continually spin.
As soon as we have more information we will send communication here..
Nov 17, 15:03:07 GMT+0
Identified -
We believe to have identified the root cause and are working to implement a fix now.
Thank you for your patience..
Nov 17, 15:03:34 GMT+0
Monitoring -
We implemented a fix and are currently monitoring the result..
Nov 17, 15:08:16 GMT+0
Resolved -
This incident has been resolved..
Nov 8, 18:04:00 GMT+0
Identified -
We are currently investigating an issue causing an increase in sync failures with connections using the BigQuery destination connector.
We have found the root cause of the issue and are working on a fix. Thank you for your patience as we work towards resolving this issue..
Nov 8, 19:29:17 GMT+0
Monitoring -
A fix has been implemented and we are currently monitoring the update.
Thank you for your patience as we work to resolve this issue..
Nov 8, 20:12:09 GMT+0
Resolved -
After monitoring the implemented fix we have confirmed this issue has been resolved.
If your most recent sync failed with a `java.lang.NullPointerException` error, we expect the next sync to succeed.
Apologies for any inconvenience and thank you again for your patience as we worked to resolve this issue. Please reach out to Airbyte Support if you have questions or concerns..
Nov 7, 17:00:00 GMT+0
Resolved -
An update was rolled out for the S3 Destination on Friday, Nov 3rd, which led to the unexpected appending of UUIDs to file names, irrespective of the set S3 path format. We have reverted this change, ensuring that new files will now adhere to the naming structure configured within the connector settings.
If necessary, you can rename files created in your destination during this period and remove the appended UUIDs from the file names.
We sincerely apologize for any inconvenience this temporary discrepancy may have caused. Please reach out to Airbyte Support if you have any questions or concerns..
Nov 7, 00:32:25 GMT+0
Monitoring -
A fix has been implemented and we are currently monitoring the update.
Thank you for your patience as we work to resolve this issue..
Nov 7, 00:22:33 GMT+0
Identified -
We have identified a brief issue impacting the sync functionality for users of Snowflake v2 between 15:29 - 15:37 PT. Syncs are now running as expected, however any syncs that ran during this time period may not have synced some records.
Our team is actively engaged in rectifying any syncs that occurred during the aforementioned time frame. We apologize for any inconvenience caused and appreciate your patience as we work to resolve this issue..
Nov 7, 00:40:36 GMT+0
Resolved -
The issue has been successfully resolved. Thank you again for your patience as worked this issue to resolution and we apologize for any inconvenience this issue might have caused..
Nov 3, 15:17:47 GMT+0
Identified -
Setting up a connection with a Google Analytics GA4 source may fail without an explicit configuration error when users try to log in using an account with insufficient permissions.
We have found the root cause of the issue and are working hard on a fix. Thank you for your patience as we work towards resolving this issue..
Nov 3, 20:26:08 GMT+0
Monitoring -
A fix has been implemented and we are currently monitoring the update.
Thank you for your patience as we work to resolve this issue..
Nov 3, 21:32:30 GMT+0
Resolved -
The issue has been successfully resolved. Thank you again for your patience as worked this issue to resolution and we apologize for any inconvenience this issue might have caused..
Oct 12, 22:00:00 GMT+0
Identified -
**Maintenance:**
Enable the point-in-time recovery option on the main Postgres database. This should be a zero-downtime operation but we noticed that connections were dropping while testing in the Staging Environment. Our backend systems do reconnect but there could be an impact on transactions that are in-flight. We estimate that the operation will take 20min to 30min, but the connection dropping only occurred during a smaller time frame of a couple of minutes in the beginning.
**Why does it matter:**
- Reduces data loss from hours to seconds in case of a database failure
- Enables us to conduct database recovery exercises without downtime as required per SOC II and ISO 27001..
Oct 12, 22:00:01 GMT+0
Identified -
Maintenance is now in progress.
Oct 12, 22:30:00 GMT+0
Completed -
Maintenance has completed successfully.
Oct 4, 22:45:00 GMT+0
Resolved -
We have rolled back the changes successfully.
If your sync is failing due to a null field error, please reset just the stream that is has a null field causing the error.
Please reach out to Airbyte Support on our [Help Center](https://support.airbyte.com/hc/en-us) if you have any questions. .
Oct 4, 15:45:00 GMT+0
Identified -
There is an issue with BigQuery & Snowflake destination processing null columns typing and deduping.
This is causing failures for multiple users. The team is currently working on reverting the change that caused this bug..
Oct 2, 16:34:21 GMT+0
Identified -
Mixpanel is currently experiencing an outage that is affecting sync success.
We will be monitoring Mixpanels status page: https://www.mixpanelstatus.com/. .
Oct 2, 18:02:06 GMT+0
Monitoring -
Mixpanel has implemented a fix and is currently monitoring the results..
Oct 3, 00:00:00 GMT+0
Resolved -
This incident has been resolved..
Sep 28, 15:02:25 GMT+0
Monitoring -
The team has reverted the changes that caused issues with accessing the Status tab and users should be able to access their connections without issue..
Sep 28, 16:35:22 GMT+0
Resolved -
This incident has been resolved..
Sep 28, 13:08:42 GMT+0
Identified -
We noticed a number of users report issues when accessing their connection.
The issue is specifically happening when navigating to the Status tab of the connection. You should be able to navigate to another tab by including the table into your URL directly.
For example:
https://cloud.airbyte.com/workspaces/{WorkspaceID}/connections/{ConnectionID}/job-history.
Sep 13, 20:48:20 GMT+0
Identified -
We have noticed an increased failure rate with Facebook Marketing connections due to an outage of Meta's API.
They have posted on their [status page](https://metastatus.com/) and we will be monitoring their status page and update here once they have resolved the issue on their side. .
Sep 14, 03:00:00 GMT+0
Resolved -
Meta has resolved their [status page](https://metastatus.com/) and we are no longer seeing failures on user's syncs..
Apr 20, 14:26:13 GMT+0
Completed -
Maintenance has completed successfully. There should not be any noticeable impact to customers or to their sync jobs..
Apr 20, 14:00:00 GMT+0
Identified -
On Thursday April 20th from 7AM Pacific to 8AM Pacific we will be updating our GCS data plane.
If syncs are triggered during this time there is possibility for and interruption of sync jobs. Any interrupted sync job should automatically retry without need for intervention.
Please reach out to Cloud support via live chat or the Support webform in the platform for any questions.
Airbyte Team.
Apr 20, 14:00:01 GMT+0
Identified -
Maintenance is now in progress.
Mar 2, 10:00:00 GMT+0
Identified -
Maintenance didn't start yet..
Mar 2, 10:00:01 GMT+0
Identified -
Maintenance is now in progress.
Mar 2, 11:30:00 GMT+0
Completed -
Maintenance has completed successfully.
Feb 24, 16:00:00 GMT+0
Identified -
We are upgrading our production machines type hardware to more powerful one, this will help to speed up our syncs.
Because of the implemented retries mechanism we do not anticipate any action needed on the part of customers.
Thank you for your patience as we upgrade Airbyte..
Feb 24, 16:00:01 GMT+0
Identified -
Maintenance is now in progress.
Feb 24, 16:44:24 GMT+0
Completed -
Maintenance has completed successfully..
Feb 23, 16:00:00 GMT+0
Identified -
We are migrating from Airbtye.io to Airbyte.com on February 23rd. All traffic pointed to Airbyte.io should automatically be redirected to Airbyte.com. We are addressing a long tail of small problems where some hardcoded URLs must be changed. Most syncs should be successful. A small number of connectors and some EU syncs will fail for a short period as we resolve this.
Please reach out to Airbyte Cloud support if you have any questions..
Feb 23, 16:00:00 GMT+0
Identified -
We are migrating from Airbtye.io to Airbyte.com on February 23rd. All traffic pointed to Airbyte.io should automatically be redirected to Airbyte.com. We are addressing a long tail of small problems where some hardcoded URLs must be changed. Most syncs should be successful. A small number of connectors and some EU syncs will fail for a short period as we resolve this.
Please reach out to Airbyte Cloud support if you have any questions..
Feb 23, 19:28:06 GMT+0
Completed -
The migration has completed successfully.
All previously succeeding syncs that experienced a failure during the migration should succeed on the next run. ***The only exception is for the Snapchat Marketing, please re-authenticate your connector to allow further syncs to succeed.***
Thank you for your patience during this maintenance and reach out to Airbyte Cloud support with any questions. .
Feb 23, 16:00:00 GMT+0
Identified -
We are upgrading our production machines type hardware to more powerful one, this will help to speed up our syncs.
Because of the implemented retries mechanism we do not anticipate any action needed on the part of customers.
Thank you for your patience as we upgrade Airbyte..
Feb 23, 16:00:01 GMT+0
Identified -
Maintenance is now in progress.
Feb 23, 17:00:00 GMT+0
Completed -
Maintenance has completed successfully.
Feb 16, 14:00:00 GMT+0
Identified -
We are changing the machines type that runs our syncs. To do this some currently running syncs will be cancelled. Those syncs should all succeed on their next sync. Because of retries we do not anticipate any action needed on the part of customers.
Thank you for your patience as we upgrade Airbyte.
Feb 16, 14:00:44 GMT+0
Identified -
Maintenance is now in progress..
Feb 16, 15:01:49 GMT+0
Completed -
Maintenance has completed successfully..
Jan 17, 10:00:01 GMT+0
Identified -
Maintenance is now in progress.
Jan 17, 12:00:00 GMT+0
Completed -
Maintenance has completed successfully.
Jan 17, 10:00:00 GMT+0
Identified -
We are planning for a scheduled maintenance during that time..
Dec 13, 10:00:01 GMT+0
Identified -
Maintenance is now in progress.
Dec 13, 13:56:55 GMT+0
Completed -
Maintenance has completed successfully..
Dec 13, 10:00:00 GMT+0
Identified -
We are planning for a scheduled maintenance during that time..
Oct 18, 09:00:00 GMT+0
Identified -
Database upgrade. Service degradation maybe faced, e.g. planned syncs will not happen. .
Oct 18, 09:00:00 GMT+0
Identified -
Database upgrade. Service degradation maybe faced, e.g. planned syncs will not happen. .
Oct 18, 09:25:07 GMT+0
Completed -
Maintenance has completed successfully..
Jul 26, 09:00:00 GMT+0
Identified -
Kubernetes cluster update.
Maintenance didn't start yet..
Jul 26, 10:00:00 GMT+0
Identified -
Maintenance is now in progress.
Jul 26, 10:36:03 GMT+0
Completed -
Maintenance has completed successfully..
Jun 30, 09:00:00 GMT+0
Identified -
Maintenance didn't start yet..
Jun 30, 10:38:23 GMT+0
Completed -
Maintenance has completed successfully..
Jun 24, 04:00:00 GMT+0
Identified -
Scheduler upgrades for improved reliability..
Jun 24, 06:40:56 GMT+0
Completed -
Maintenance has completed successfully..
Jun 24, 04:00:00 GMT+0
Identified -
Scheduler upgrades for improved reliability - in progress.
Jun 21, 09:00:00 GMT+0
Identified -
Infrastructure update.
Maintenance didn't start yet..
Jun 21, 09:00:01 GMT+0
Identified -
Maintenance is now in progress.
Jun 21, 09:00:01 GMT+0
Identified -
Maintenance is now in progress.
Jun 21, 09:40:32 GMT+0
Completed -
Maintenance has completed successfully..
Jun 7, 10:30:00 GMT+0
Identified -
Maintenance didn't start yet..
Jun 7, 10:19:21 GMT+0
Identified -
Maintenance is now in progress..
Jun 7, 12:31:58 GMT+0
Completed -
Maintenance has completed successfully.
GCP roles for users on production env have been recreated successfully..
Mar 10, 10:00:00 GMT+0
Identified -
Maintenance didn't start yet..
Mar 10, 10:04:00 GMT+0
Identified -
Maintenance is now in progress..
Mar 10, 10:39:49 GMT+0
Completed -
Maintenance has completed successfully..
Dec 28, 10:00:00 GMT+0
Identified -
Database changes are planned to be applied..
Dec 28, 10:00:00 GMT+0
Completed -
Maintenance has completed successfully..
Dec 28, 10:00:00 GMT+0
Identified -
Maintenance is now in progress..
Dec 8, 22:00:00 GMT+0
Identified -
Maintenance is started..
Dec 9, 10:00:00 GMT+0
Identified -
Maintenance is now in progress..
Dec 9, 12:23:28 GMT+0
Completed -
Maintenance has completed successfully..
Dec 2, 10:02:57 GMT+0
Identified -
Maintenance is now in progress..
Dec 2, 10:00:00 GMT+0
Identified -
Improvements of a Database backup procedures will be conducted..
Dec 2, 10:30:45 GMT+0
Completed -
Maintenance has completed successfully..
Oct 26, 09:30:00 GMT+0
Completed -
Under maintenance..