Webdock - Fault in Webdock dashboard/site database code – Incident details

All systems operational

Fault in Webdock dashboard/site database code

Resolved
Operational
Started 2 months agoLasted 37 minutes

Affected

Webdock Dashboard

Partial outage from 6:15 AM to 6:51 AM

Updates
  • Resolved
    Resolved

    The issue has been resolved. A strange sequence of events occurred at about 2AM this evening, which was ultimately caused by Cloudflare rate-limiting requests. This in turn caused our search indexer to break our search index. With a broken search indexer, all database listing filtering was failing causing exceptions to be thrown on many of our dashboard pages. Without going into too much detail, the resolution was simply to rerun the search indexer and we are now operational. We will be looking closely at how to prevent this from happening again.

  • Identified
    Identified

    We see a fault in a certain database subsystem which is causing our dashboard to be inaccessible at the moment. We are working hard on a resolution. This is not a hack, nor does this fault affect customer servers in anyway. We think it's some problem in a database caching system which is responsible.