How Microsoft tackles the 30,000 bugs its 47,000 developers generate each month
Microsoft is detailing how it handles bugs in its software and services using machine learning models. “47,000 developers generate nearly 30,000 bugs a month,” explains Scott Christiansen, a senior security program manager at Microsoft. The software maker tracks these bugs across GitHub and AzureDevOps repositories, but it’s a lot of issues to track with just traditional labeling and prioritization.
Microsoft is now using nearly 20 years of historical data across 13 million work items and bugs to create a machine-learning model that can separate security and non-security bugs 99 percent of the time. It’s a model that’s designed to help developers accurately identify and prioritize critical security issues that need fixing.
“Our goal was to build a machine learning system that classifies bugs as security / non-security and critical/non-critical with a level of accuracy that is as close as possible to that of a security expert,” explains Christiansen. Microsoft fed its machine learning model bugs that were labeled security and non-security to train it and make sure the data wasn’t too noisy. The model then learned how to classify security bugs and apply severity labels like critical, important, or low-impact to each.
Security experts and data scientists worked together at Microsoft to create the model, ensuring that it could be monitored in production and that a random sampling of bugs are manually reviewed. The model is also continually re-trained with new data that’s reviewed by Microsoft’s security experts. This machine learning model means Microsoft now accurately identifies security bugs 99 percent of the time, and labels them correctly 97 percent of the time.
It’s unusual for a company the size of Microsoft to reveal how many bugs its developers generate on a monthly basis, let alone how it tackles these. Microsoft is now planning to open source its methodology to GitHub, allowing other companies with similar data sets to implement a similar model. If you’re interested in learning more about Microsoft’s machine learning techniques, the company has published a academic paper with all the details.
Microsoft is detailing how it handles bugs in its software and services using machine learning models. “47,000 developers generate nearly 30,000 bugs a month,” explains Scott Christiansen, a senior security program manager at Microsoft. The software maker tracks these bugs across GitHub and AzureDevOps repositories, but it’s a lot of…
Recent Posts
- iOS 17.5 is reportedly resurfacing once deleted photos for some users
- Arizona accuses Amazon of being a monopoly and deceiving consumers with ‘dark patterns’
- Android 15’s second beta release lets users lock down access to private apps
- Apple’s next accessibility features let you control your iPhone and iPad with just your eyes
- Microsoft announces the Proteus Controller, a gamepad for Xbox gamers with disabilities
Archives
- May 2024
- April 2024
- March 2024
- February 2024
- January 2024
- December 2023
- November 2023
- October 2023
- September 2023
- August 2023
- July 2023
- June 2023
- May 2023
- April 2023
- March 2023
- February 2023
- January 2023
- December 2022
- November 2022
- October 2022
- September 2022
- August 2022
- July 2022
- June 2022
- May 2022
- April 2022
- March 2022
- February 2022
- January 2022
- December 2021
- November 2021
- October 2021
- September 2021
- August 2021
- July 2021
- June 2021
- May 2021
- April 2021
- March 2021
- February 2021
- January 2021
- December 2020
- November 2020
- October 2020
- September 2020
- August 2020
- July 2020
- June 2020
- May 2020
- April 2020
- March 2020
- February 2020
- January 2020
- December 2019
- November 2019
- December 2011