Temporary Disruption of Cloud Control Panel and API

After validation in our lab environment, we have successfully rolled out a permanent fix by applying the missing change of a previous rollout. In the next few days, we will review our rollout and QA process carefully and take the necessary measures to prevent such a situation from happening in the future.

Please accept our apologies for the inconvenience this incident may have caused you and your customers.

While we will roll out a permanent fix later today, we have successfully put a workaround in place. All services are operational again since 15:56 CEST.

Today at 15:23 CEST, we were notified by our monitoring system that specific OpenStack services were not running as expected. We started investigating and found out that these services were missing a specific change from an earlier rollout. We are currently resolving the issue and expect the Cloud Control Panel and API to be back soon.

Please note: Already running cloud servers are not affected.

We are sorry for the inconvenience this may cause. Rest assured that are working hard on restoring the service as soon as possible.