Microsoft is detailing how it handles bugs in its software and services using machine learning models. “47,000 developers generate nearly 30,000 bugs a month,” explains Scott Christiansen, a senior security program manager at Microsoft. The software maker tracks these bugs across GitHub and AzureDevOps repositories, but it’s tons of issues to trace with just traditional labeling and prioritization.
Microsoft is now using nearly 20 years of historical data across 13 million work items and bugs to make a machine-learning model which will separate security and non-security bugs 99 percent of the time. It’s a model that’s designed to assist developers inaccurately identify and prioritize critical security issues that require fixing.
“A machine learning model involves the rescue”
“Our goal was to create a machine learning system that classifies bugs as security / non-security and critical/non-critical with A level of accuracy that’s as close as possible thereto of a security expert,” explains Christiansen. Microsoft fed its machine learning model bugs that were labeled security and non-security to coach it and confirm the info wasn’t too noisy. The model then learned the way to classify security bugs and apply severity labels like critical, important, or low-impact to every.
Security experts and data scientists worked together at Microsoft to make the model, ensuring that it might be monitored in production in which a sampling of bugs is manually reviewed. The model is additionally continually re-trained with new data that are reviewed by Microsoft’s security experts. This machine learning model means Microsoft now accurately identifies security bugs 99 percent of the time and labels them correctly 97 percent of the time.
It’s unusual for a corporation the dimensions of Microsoft to reveal what percentage bugs its developers to generate on a monthly basis, including how it tackles these. Microsoft is now getting to open source its methodology to GitHub, allowing other companies with similar data sets to implement an identical model.