Resolved -
Service is now restored in the EU. The Total Risk Score Over Time chart on the homepage is no longer 0 for customers. We have determined that it is normal and expected for Total Risk Score Over Time and Mean Time to Remediate charts to be 2 days behind. We apologize for the inconvenience this incident caused and thank you for your patience as we worked on resolving it.
Jun 15, 19:06 CDT
Update -
We are still working to resolve this issue and we thank you for your patience. Some Risk Meter scores continue to be at 0. The queue to update vulnerability scores went down to 0 last night, but is back up again and we are investigating why this happened. Some vulnerabilities in the UI temporarily had no data to show. The queue backlog is causing problems in processing updates to assets and vulnerabilities including delays and failures. For example, changes to asset and vulnerability statuses are not being reflected in the UI.
Jun 15, 10:29 CDT
Monitoring -
We are currently working on a resolution to the issue. EU instances should not have any more errors impacting new tasks but service is not completely restored. Dashboards will continue to show risk scores of 0 this evening. Once the daily recalculation of Risk Meter scoring completes in the early morning, instances should see their Risk Meter scores return. Unfortunately, there will be a gap between June 10th and June 15th, with no scores shown for the 11th, 12th, 13th and 14th. Furthermore, the re-scoring of individual vulnerabilities is backed-up. Updates to vulnerability scoring from Connector Runs and API calls is enqueued in our system, but significantly delayed. We anticipate that the queues will catch up by tomorrow. We apologize for this inconvenience and will continue to work towards a resolution.
Jun 14, 21:53 CDT
Investigating -
Starting at 3:00 AM UTC on Saturday, June 12 an error occurred with the elastic search index resulting in a drop in risk meter scores to 0 and Dashboards failing to load. We are still investigating this issue and will update you as we know more. Thank you for your patience.
Jun 14, 12:59 CDT