CrowdStrike, a well-known cybersecurity firm, has released a detailed report on what caused the widespread crash of its Falcon Sensor software, affecting millions of Windows devices around the world.
What Happened?
The issue, referred to as the “Channel File 291” incident, was due to a problem in a software update aimed at improving security. This update was supposed to help detect new types of cyber attacks that use Windows communication methods, but it ended up causing a major crash instead.
The Root Cause
The crash was triggered by a mismatch in the software update. The update was meant to handle 21 pieces of information, but the system only provided 20. This mismatch wasn’t caught during testing because the tests used a wildcard setting that didn’t highlight the missing piece of information.
The problematic update was released on July 19, 2024, and it was the first time the system needed to use all 21 pieces of information. Since there wasn’t a specific test for this scenario, the issue went unnoticed until the update was already in use.
The Impact
Once the update was installed, it caused a problem known as an “out-of-bounds read” in the system’s memory. When the system tried to read the missing piece of information, it accessed an area of memory it shouldn’t have, leading to crashes.
Fixes and Improvements
To fix the problem, CrowdStrike has made several changes:
- They now check the number of inputs during the software’s compile time to ensure they match.
- They’ve added a runtime check to prevent the system from accessing memory incorrectly.
- They corrected the number of inputs provided by the update.
Additionally, they’ve increased the scope of their testing to include cases that don’t use wildcard settings. They’ve also made the following updates:
- Modified the Content Validator to prevent mismatches in the number of inputs.
- Limited wildcard settings to prevent similar issues.
- Updated testing procedures to ensure every new update is thoroughly tested.
- Improved deployment checks and added more layers of testing.
Future Steps
CrowdStrike has also engaged two independent security firms to review their code and quality processes. They are working with Microsoft to enhance security measures and have pledged to improve how they deliver security updates to avoid future issues.
Response to Criticism
Delta Air Lines, which experienced significant disruptions due to the crash, is seeking compensation from CrowdStrike and Microsoft, claiming losses of around $500 million. Both companies have responded, stating they offered assistance which was declined by Delta, suggesting the airline’s issues may extend beyond the software update problems.
Conclusion
CrowdStrike’s report sheds light on the complexities of cybersecurity and the importance of thorough testing and validation in software updates. They are taking steps to prevent similar incidents in the future, aiming to provide robust security without compromising system stability.
Cookie Consent with Real Cookie Banner