Timeline of events

Around 01:00am CEST our monitoring system started recording devices not being able to reconnect to the server when they lost connection.
Early morning, about 06:30am CEST, investigation started by the technical team. At that time, only around 40% of devices were still connected.
As the connections kept dropping, a reboot was initiated around 07:30am CEST.
This reboot did not solve the issues, and our technical team kept searching for the root cause.
The problem was found in an unoptimized configured service and hot fixed at around 08:20am CEST.
A second reboot was initiated at 08:30am CEST.
Devices restored their connections over the next hours.
At around 1:30pm CEST normal operation was restored.

Impact

This issue was confined to the RL/mbCONNECT24 EU V2 Server.

Follow-up actions

Possible side effects of the hot fix are being tested and monitored. Afterward, the changes will be implemented into the next RSP release.

Posted Apr 30, 2024 - 15:40 CEST

Resolved

This incident has been resolved.

Posted Apr 30, 2024 - 14:04 CEST

Monitoring

A fix has been implemented and we are monitoring the results.
The devices are currently reconnecting to the portal.

Posted Apr 30, 2024 - 09:45 CEST

Investigating

We are experiencing temporary connectivity issues with our system.

The R&D team is informed and the situation is being investigated. Users of the system might have problems connecting to the system.

We will keep you updated

Posted Apr 30, 2024 - 06:30 CEST

This incident affected: mbCONNECT24 Servers (mbCONNECT24 Europe V2).