Increased Connector Run Queuing
Incident Report for Kenna Security
Resolved
A fix has been successfully deployed and all affected connectors have been restarted successfully, and are processing data correctly.
Posted Oct 07, 2018 - 22:28 CDT
Monitoring
A fix is currently being tested and will be deployed as soon as it is ready. We have determined that all affected connector runs will need to be restarted once this fix is in place, and will automatically perform those restarts at that time.

Additional updates will be provided as the fix is deployed.
Posted Oct 07, 2018 - 21:38 CDT
Identified
After further investigation, we have determined this additional queuing was not organic and is due to an unexpected Qualys API change in the paged detection data that is retrieved by Kenna's platform. This change resulted in our Qualys connector logic to continuously request duplicated detection data, erroneously attempting to process this data repeatedly. This duplicate processing resulted in large amounts of queued data.

Our engineering team is currently working on a change to our connector logic to account for the Qualys API change. Additional updates will be provided this evening.
Posted Oct 07, 2018 - 21:11 CDT
Monitoring
At 4pm CDT (9pm UTC) this afternoon our engineering team was notified of higher than usual connector run queues in our US platform, resulting in delayed importing of new client connector data. Additional processing capabilities have been deployed to handle this queuing, and our engineering team is continuing to monitor/tweak processing as needed.

Additional updates will be provided later this evening.
Posted Oct 07, 2018 - 18:42 CDT
This incident affected: US (Connector Runs).