Oct 26, 17:24 EDT
Resolved - From 2:46pm EST to 4:22pm EST we were experiencing intermittent issues with our API, which resulted in some public model failures. A fix has been applied for this and we are continuing to investigate the root cause.
Nov 21, 00:51 EST
Resolved - Due to a required database update, we needed to pause all requests involving database changes from Nov 21 00:00:00 -- Nov 21 00:45:00 US Eastern Time. During the maintenance window, users were unable to add, modify or delete inputs and concepts, and any such calls would have resulted in an HTTP code 401, error code 39998, and an error message of “Input writes are disabled for maintenance. Please try again in a few hours.”.
Predictions (public and custom) and searches with existing inputs remained unaffected during this period.
Dec 14, 19:12 EST
Resolved - The issue has now been resolved. We've confirmed that it randomly affected all database-related features (search, post inputs, etc.) between 5:28pm EST and 6:58pm EST.
Dec 14, 19:06 EST
Monitoring - We've identified that the problem is on the vendor's end and it appears that it has been resolved. We are continuing to monitor the situation.
Dec 14, 18:52 EST
Investigating - Around 6:25pm EST we discovered that database read/writes are randomly failing. We've opened a high priority ticket with our vendor and will continue to investigate.
Dec 20, 18:56 EST
Resolved - The issue should now be resolved.
Dec 20, 18:54 EST
Investigating - We are currently investigating some database issues that occurred from a recent migration. As a result you may not be able to access your Clarifai applications in the interim.
Jan 24, 18:12 EST
Resolved - The issue has been resolved and all search operations should be working again. Post-mortem to follow.
Jan 24, 16:05 EST
Investigating - We are currently experiencing some issues with our Search platform and are looking into the root cause. Apologies for the inconvenience.
Feb 28, 02:00 EST
Completed - The scheduled maintenance has been completed.
Feb 28, 01:30 EST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Feb 28, 01:15 EST
Scheduled - During this timespan Amazon will be making upgrades to their back-end systems, which we rely on for our own products. It is likely that some of our functionality will be affected because of this, but the upgrade should last no longer than 30 minutes. We apologize for this inconvenience.
Mar 27, 17:54 EDT
Resolved - We are still waiting on a root cause and will update with our postmortem accordingly. Apologies again for any inconveniences.
Mar 27, 17:20 EDT
Monitoring - Service was restored around 5:11 p.m. EST. We opened a ticket with our database vendor to determine the root cause and are still waiting to hear back from them.
Mar 27, 16:54 EDT
Investigating - Around 4:20pm EST we noticed a large spike in gateway timeout errors from our internal system logs. We are currently investigating this with a high priority and apologize for any inconvenience caused.
Affected platforms: Visual Search, Custom Training, Input Uploads, Explorer Tool
Apr 17, 19:27 EDT
Resolved - This incident has been resolved.
Apr 17, 19:25 EDT
Monitoring - We have successfully restored service to the affected areas and are monitoring to ensure that it remains stable. Detailed post-mortem to follow.
Apr 17, 19:04 EDT
Identified - We have identified the root cause of today's issues and are currently working on a fix.
Apr 17, 16:50 EDT
Investigating - Our Operations Team is currently looking into some issues with one of our third party vendors.
Affected platforms: Custom Training, Visual Search, Explorer Tool
We apologize for any inconvenience caused by this.
Apr 18, 16:42 EDT
Resolved - This incident has been resolved.
Apr 18, 16:14 EDT
Monitoring - A fix has been applied and all systems are running normally again.
Apr 18, 15:48 EDT
Investigating - Around 3:20p.m. EST we started to experience similar issues to yesterday's outage and are actively pursuing a fix. More information to come shortly.
Apr 20, 12:56 EDT
Resolved - Around 6:00am EDT we started to experience some issues with our Custom Training, Visual Search and Explorer UI tools that would have likely interrupted access to them. This was resolved around 10:40am EDT and we are still investigating the cause/future solutions with one of our vendors. Apologies for any inconveniences that this caused.