[App/Intranet] User Search and related features don’t work

Incident Report for Staffbase

Postmortem

We have completed our investigation of the incident and would like to provide you with more detailed information regarding the issue that occurred on January 26, at 09:09 AM CET.

Summary of the issue:

On January 26, between 09:09 AM CET and 03:36 PM CET, some customers reported that the user search function and related functionalities (such as ghostwriting, post reach, and mentions) weren't available.

What happened:

While trying to implement an internal fix, we unintentionally triggered a previously unnoticed configuration issue, which broke the user search indexing. We resolved the issue by initiating a full synchronization of our search index database.

Impact:

The user search function and related functionalities were unavailable throughout the duration of the incident.

‌

Future improvements:

To prevent future incidents, we have implemented additional checks during updates to the search engine to avoid faulty indexing.

We thank you for your patience and understanding as we continue to improve our systems and prevent future incidents. If you have any questions or concerns, please contact our support team at support@staffbase.com.

Posted Feb 01, 2024 - 08:51 CET

Resolved

We have closely monitored the system and haven't observed the same behavior again. Therefore, we are pleased to inform you that the issue has been resolved. We sincerely apologize for any inconvenience this may have caused.
If you are still experiencing any problems, please don't hesitate to contact us at support@staffbase.com. We are more than happy to assist you further.

Posted Jan 29, 2024 - 13:57 CET

Monitoring

We have successfully implemented a solution to fix the issue so that the search function and related features (such as post reach, user export, mentions and ghostwriting) are working again. Rest assured that we will be monitoring our systems closely to ensure that the issue remains fully resolved. Should you have any further difficulties, please don't hesitate to contact our support at support@staffbase.com.

Posted Jan 26, 2024 - 15:36 CET

Update

Although we are still in the process of completing the implementation, we would like to inform you that several features are currently affected. These include the user search functionality in both the studio and the app, as well as the user API and related features (such as post reach, user export, mentions and ghostwriting).
If you have any questions regarding this topic, please reach out to our support at support@staffbase.com.

Posted Jan 26, 2024 - 14:47 CET

Identified

We have identified the root cause of the problem and are currently working on implementing a solution.

Posted Jan 26, 2024 - 09:58 CET

Investigating

Some customers on the German System may be experiencing issues with the user search and related features like user reach analytics.

Our teams are investigating and will work to resolve this as fast as we can. We will share more information soon.

We apologize for the inconvenience. If you have any questions regarding this topic, please reach out to our support at support@staffbase.com.

Posted Jan 26, 2024 - 09:09 CET

This incident affected: Employee App & Front Door Intranet (all regions) (Core API (Germany) [SLA]).

Welcome to Staffbase Status