Performance Issues Denial of Service (DoS) attack

Incident Post-Mortem: DDoS Attack on Calxa Application

Summary

On Wed the 17th of July, between 3.30am and 1pm (AEST), the Calxa Application experienced a Distributed Denial of Service (DDoS) attack. This incident resulted in temporary service disruption for users, particularly affecting trial sign-ups and sign-ins from outside Australia.

What is a Distributed Denial of Service (DDoS) Attack?

A Distributed Denial of Service (DDoS) attack is an attempt to make an online service unavailable by overwhelming it with a flood of internet traffic from multiple sources. Unlike attempts to infiltrate or extract data from a system, a DDoS attack aims to disrupt service and prevent legitimate users from accessing the service. In this case, the attack involved a flood of phony trial sign-ups designed to overload our system.

Customer Data Safety

We want to reassure our customers that at no point was any customer data compromised. This attack was not an attempt to infiltrate our system or steal data but rather to disrupt service by overwhelming it with traffic. Your data remains secure and protected.

Incident Details

At approx. 3:30 AM on the 17th of July, our monitoring systems alerted us to an unusual spike in traffic. Upon investigation, our global team discovered that our services were being overwhelmed by a flood of phony trial sign-ups, indicating a DDoS attack. The attack exploited our trial sign-up registration process to provision new trial accounts and later configure SMS 2-Step Verification. Both actions were intended to overload our systems and cause service disruption.

Impact

Over the duration of the incident:

• There was an initial period of performance degradation
• 3 hours of application downtime
• Logins limited to Australia and New Zealand for 9 hours
• Trial sign ups blocked for 16 hours
• Some users experienced issues sending SMS 2-Step verification codes (email or authenticator app codes continued to be available)

Response

Once system alerts were triggered, the security incident response team were advised and an initial investigation confirmed a DDoS attack. The team promptly disabled access to the Calxa application to limit the impact of the incident. The attack was from multiple locations outside Australia and New Zealand and only affected trial sign ups.

In collaboration with Microsoft support, mitigating measures were implemented. These included restricting access by region and blocking trial sign ups, which allowed us to restore access during business hours for the Australian and New Zealand regions. Later further mitigations were added via a custom login domain, with advanced traffic filtering and blocking rules that enabled us to safely remove all login and trial registration restrictions.

Preventative Measures

To prevent similar incidents, we have implemented the following measures:

• New domain, “login.calxa.com” with options to block malicious traffic more effectively
• Introduced rate-limiting on trial sign-up requests to prevent abuse
• Improved our monitoring systems to detect and respond to abnormal traffic patterns more quickly

Conclusion

We apologise for any inconvenience this incident may have caused and appreciate your patience and understanding. Our teams worked in collaboration with Microsoft to recover services quickly and prevent future attacks.

We are committed to learning from this experience and continue to enhance our systems to ensure such incidents are less likely in the future.

Thank you for your continued trust in Calxa.

Overnight, the trial sign-up services have been restored, and we have implemented additional measures to further reduce our exposure to similar attacks.

We will provide a full post-mortem on this status page in the coming days once our internal investigation is complete.

After a period of monitoring, we feel confident the Calxa app is now stable for users.

The denial of service attack was intended to cause performance issues and deny our service. At no time was our customer data compromised.

We are now implementing additional safeguards to enable re-opening trial sign ups overnight.

We are pleased to report that the Calxa Application is stable for users in Australia and New Zealand. We are now implementing additional strategies to allow us to re-enable trial sign-ups and remove sign-in restrictions.

The DoS Attack is still ongoing; however, we are pleased to inform you that the Calxa Application is now back online with a few temporary caveats. To maintain stability, trial sign-ups are currently blocked, and sign-ins from outside Australia are restricted. Our team, in collaboration with Microsoft Support, continues to monitor the situation closely and work towards a full resolution.

Thank you for your patience and understanding.

We are continuing to experience a Denial of Service (DoS) attack, resulting from a flood of phony trial sign-ups. To limit the impact, we have temporarily turned off access to the application. Our team is actively working to mitigate the attack, and we are now collaborating with Microsoft Support to expedite the resolution process. We are committed to restoring normal service as quickly as possible.

Thank you for your patience and understanding.

We are currently experiencing a Denial of Service (DoS) attack. Our services are being flooded with an excessive number of phony trial sign-ups, which is causing performance issues. Our team is actively working to mitigate the attack and restore normal service as quickly as possible. We apologise for any inconvenience this may cause and appreciate your patience during this time.

Thank you for your understanding.