Welcome to Staffbase Status
We continuously monitor outages, slowdowns, and other issues. Should there be any interruptions in service, we'll provide status updates here.
Service disruption - slow process of news publishing / missing push notification
Incident
Report for Staffbase
Resolved
After the technical issues throughout the last week, we further observed our infrastructure and are meanwhile confident to say that our backend system has generally stabilized. Therefore, we will close this incident.
Summary: Customers using our German infrastructure experienced several service disruptions including the following behaviours:
- Delayed or failing push and email notifications
- Slow operation of publishing news posts and comments
- Slow operating user management
We diagnosed that our backend system had difficulties processing the notification requests under certain circumstances which led to a slow-down of the entire infrastructure.
Once we identified the root cause, we implemented several hotfixes, which eventually had a lasting effect by Wednesday afternoon (September 1st). Ever since our German system has been operating as expected.
Due to the frequency of similar incidents in recent times, we will take further prioritized actions that should clearly improve the performance of the affected functions.
We would like to apologize for the unstable situation and the troubles this has caused you. We understand that these functions are central to your use of the app and we were not able to provide the necessary reliability lately.
If you have further questions about this incident, you can contact us at support@staffbase.com and we will be happy to answer them.
Summary: Customers using our German infrastructure experienced several service disruptions including the following behaviours:
- Delayed or failing push and email notifications
- Slow operation of publishing news posts and comments
- Slow operating user management
We diagnosed that our backend system had difficulties processing the notification requests under certain circumstances which led to a slow-down of the entire infrastructure.
Once we identified the root cause, we implemented several hotfixes, which eventually had a lasting effect by Wednesday afternoon (September 1st). Ever since our German system has been operating as expected.
Due to the frequency of similar incidents in recent times, we will take further prioritized actions that should clearly improve the performance of the affected functions.
We would like to apologize for the unstable situation and the troubles this has caused you. We understand that these functions are central to your use of the app and we were not able to provide the necessary reliability lately.
If you have further questions about this incident, you can contact us at support@staffbase.com and we will be happy to answer them.
Monitoring
Since 5 pm (CET) the performance of our system has recovered including push notifications working as expected again.
Therefore we set the status back to monitoring.
Therefore we set the status back to monitoring.
Identified
Unfortunately, we are experiencing another service disruption with push and email notifications on our German system - you may experience a delayed arrival of push notifications.
We implemented the first fixes to mitigate the effects.
In case you have any questions, please get in touch with our Customer Care team under support@staffbase.com who will be happy to help you out.
We implemented the first fixes to mitigate the effects.
In case you have any questions, please get in touch with our Customer Care team under support@staffbase.com who will be happy to help you out.
Update
Our last measurements led to a clear improvement of the notification function on our German system and we believe customers should be able to use the push notification without further delays or timeouts.
We will continue to monitor the situation until tomorrow at least before resolving.
We will continue to monitor the situation until tomorrow at least before resolving.
Update
Our resolution process is still ongoing.
At the current stage, our infrastructure around notifications should generally work as intended but short timeouts may occur throughout the day which can result in delayed push messages.
In the event of a timeout, you can approach us under support@staffbase.com for direct feedback.
At the current stage, our infrastructure around notifications should generally work as intended but short timeouts may occur throughout the day which can result in delayed push messages.
In the event of a timeout, you can approach us under support@staffbase.com for direct feedback.
Update
We have implemented a fix to resolve the issue and notifications are slowly getting back to normal. Unfortunately, we still occasionally see delays in push notifications for our German system.
We will continue to work on an overall solution. Therefore, if you still experience any related issues, please contact us at support@staffbase.com
We will continue to work on an overall solution. Therefore, if you still experience any related issues, please contact us at support@staffbase.com
Monitoring
We see a recovery of our German system. All affected functions should work as intended again.
We continue to monitor our systems.
If you still experience any related issues, please contact us at support@staffbase.com
We continue to monitor our systems.
If you still experience any related issues, please contact us at support@staffbase.com
Investigating
Following further investigations, we currently see more functions of the application affected by the incident.
According to reports, some of the following functions may not work or might be delayed:
- Publishing news posts and comments
- Push and email notification
We suggest not to publish content several times in order to minimize the effects on our systems as well as prevent multiple publications at once.
If you encounter any other faulty behaviours today - please let us know.
Due to the bigger scope of the disruption we set the status back to “Investigating”.
According to reports, some of the following functions may not work or might be delayed:
- Publishing news posts and comments
- Push and email notification
We suggest not to publish content several times in order to minimize the effects on our systems as well as prevent multiple publications at once.
If you encounter any other faulty behaviours today - please let us know.
Due to the bigger scope of the disruption we set the status back to “Investigating”.
Identified
We are experiencing a service disruption with our news publishing process as well as sending out push and email notifications.
Publishing posts and sending out notifications may either be delayed or interrupted.
We have meanwhile identified the cause and work on a solution as fast as possible.
Further status updates will follow soon.
Publishing posts and sending out notifications may either be delayed or interrupted.
We have meanwhile identified the cause and work on a solution as fast as possible.
Further status updates will follow soon.
This incident affected: Employee App & Front Door Intranet (all regions) (Core API (Germany) [SLA]).