Apr 22, 07:30 PDT
Resolved - This incident has been resolved.
Apr 22, 07:30 PDT
Resolved - Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring.
## Impact
Between April 22nd 14:25 UTC and April 23rd 8:50 UTC our search functionality operated on outdated data for some users, which resulted in potentially incorrect search results. In addition, some webhooks with custom field updates were not delivered. No data was lost during the incident.
## Root Cause and Resolution
A software update introduced a bug, where some events for custom field updates were malformed. As a result, the changes to custom fields weren’t reflected in search, and webhooks for these events were not sent. Close Engineering reverted the change, and reindexed all affected leads.
We are adding guardrails which will prevent us from sending such malformed events in future.
## Timeline
- 22 April 14:25 UTC - A software update is deployed that sends malformed events for some Custom Field updates
- 22 April 21:20 UTC - Close Engineering is notified of leads not populating correctly in smart views
- 23 April 04:36 UTC - Root cause is identified
- 23 April 05:49 UTC - Measures applied to stop further impact
- 23 April 08:46 UTC - Completed reindexing of affected leads
Apr 8, 21:35 PDT
Completed - The scheduled maintenance has been completed.
Apr 8, 21:07 PDT
Verifying - Verification is currently underway for the maintenance items.
Apr 8, 21:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 4, 13:20 PDT
Scheduled - We will be performing maintenance on our backend database on Tuesday, April 8, from 9:00 PM to 10:00 PM PDT (Wednesday, April 9, 04:00–05:00 UTC).
During this time, the Close App and API will remain operational, but some users may experience an elevated error rate.
The maintenance is expected to be complete by 10:00 PM PDT (Wednesday, April 9, 05:00 UTC).
Apr 8, 21:07 PDT
Verifying - Verification is currently underway for the maintenance items.
Apr 8, 21:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 4, 13:20 PDT
Scheduled - We will be performing maintenance on our backend database on Tuesday, April 8, from 9:00 PM to 10:00 PM PDT (Wednesday, April 9, 04:00–05:00 UTC).
During this time, the Close App and API will remain operational, but some users may experience an elevated error rate.
The maintenance is expected to be complete by 10:00 PM PDT (Wednesday, April 9, 05:00 UTC).
Apr 8, 21:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 4, 13:20 PDT
Scheduled - We will be performing maintenance on our backend database on Tuesday, April 8, from 9:00 PM to 10:00 PM PDT (Wednesday, April 9, 04:00–05:00 UTC).
During this time, the Close App and API will remain operational, but some users may experience an elevated error rate.
The maintenance is expected to be complete by 10:00 PM PDT (Wednesday, April 9, 05:00 UTC).
Apr 1, 07:17 PDT
Completed - This maintenance has been canceled.
Mar 26, 08:07 PDT
Update - This maintenance has been rescheduled to Tuesday, April 1 starting at 9:00 PM until 10:00 PM PDT (Wednesday, April 2, 04:00 - 05:00 UTC).
Mar 25, 09:53 PDT
Scheduled - We will be performing maintenance on our back end database on Wednesday, March 26 starting at 10:00 PM PDT until 11:00 PM PDT (Thursday, March 27, 05:00 - 06:00 UTC). The Close App and API will remain operational. Some users may experience an elevated error rate while using the Close App and API during the maintenance window. The maintenance is expected to be complete by 11:00 PM PDT (Thursday, March 27, 06:00 UTC).
Mar 25, 14:44 PDT
Resolved - This incident has been resolved.
Mar 25, 11:21 PDT
Identified - OpenAI are currently experiencing increased error rates on their side, which is impacting Lead Summaries and AI Search in Close. You can follow for updates on their status page here - https://status.openai.com/
Mar 25, 11:21 PDT
Identified - OpenAI are currently experiencing increased error rates on their side, which is impacting Lead Summaries and AI Search in Close. You can follow for updates on their status page here - https://status.openai.com/
Mar 25, 11:21 PDT
Identified - Open AI are currently experiencing increased error rates on their side, which is impacting AI features in Close. You can follow for updates on their status page here - https://status.openai.com/
Mar 10, 09:28 PDT
Resolved - This incident has been resolved.
Mar 10, 09:21 PDT
Monitoring - We are continuing to investigate the cause of the degraded performance of our Dialer system. Our Dialer system is now functioning normally. We are monitoring performance.
Mar 10, 09:02 PDT
Investigating - We've become aware of degraded performance of our Dialer service. We are investigating the issue. Updates will be posted as they become available.