Staffbase Status - [App/Intranet] - Customers with apps on the German system may experience issues when accessing their apps

[App/Intranet] - Customers with apps on the German system may experience issues when accessing their apps

Incident Report for Staffbase

Postmortem

Summary
As our investigation of the incident is complete, we would like to provide you with more detailed information about the incident that occurred on November 5, between 06:40 AM and 08:20 AM (CET), and prevented users from accessing their Employee App & Front Door Intranet hosted on the German system.

What happened
Due to a misconfiguration of a required service component, Staffbase’s autoscaling had been blocked. This led to our servers no longer being able to handle the increasing traffic during morning hours which in turn led to an outage of all apps on the German system. By adjusting the faulty configuration and implementing additional resilience measures, we were able to unblock the autoscaling mechanism and fully restore the availability.

Impact
Between 06:40 AM and 08:20 AM (CET), our customers reported issues when accessing their Employee App & Front Door Intranet hosted on the German system. All users could no longer log in and received a connectivity error message. Additionally, third-party services could not interact with customer’s instances during the incident.

Future improvements
We will implement additional checks for the operation of the affected service component and optimize the subsequent alerting to ensure that no more outages will occur due to potential misconfigurations. Thank you for your patience and understanding while we work to improve our services and prevent future incidents. If you have any questions or concerns, please do not hesitate to reach out to our customer support team at support@staffbase.com.

Posted Nov 06, 2024 - 16:54 CET

Resolved

The issue is resolved and all apps are reachable again. We apologize for the inconvenience this has caused. Please get in touch with us at support@staffbase.com in case you're still having any problems. We’re happy to help!

Posted Nov 05, 2024 - 09:48 CET

Monitoring

After the rollout of a fix, our teams are seeing improvements in app performance and believe customers should be able to use their apps and intranet again. We will continue to monitor the situation and update you shortly.

Posted Nov 05, 2024 - 08:18 CET

Investigating

Our teams are investigating issues with apps on the German system that are no longer accessible and displaying a connectivity error instead. We will keep you updated on the progress as we learn more.

You can see on which system your application is hosted by going to the Studio > Settings > API Access and checking the tag next to the title.

We apologize for the interruption. In case you have any questions about this topic, please let us know at support@staffbase.com.

Posted Nov 05, 2024 - 07:07 CET

This incident affected: Employee App & Front Door Intranet (Core API (Europe) [SLA], Web Application (Europe) [SLA]).

Welcome to Staffbase Status