{"id":38842,"date":"2020-04-07T07:19:49","date_gmt":"2020-04-07T14:19:49","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/premier-developer\/?p=38842"},"modified":"2020-03-23T07:34:29","modified_gmt":"2020-03-23T14:34:29","slug":"bias-in-machine-learning","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/premier-developer\/bias-in-machine-learning\/","title":{"rendered":"Bias in Machine Learning"},"content":{"rendered":"

Dev Consultant Ashley Shorter<\/a> examines the dangers of bias and importance of ethics in Machine Learning.<\/p>\n

\n

In our digital <\/span>era<\/span>, efficiency is expected. We can instantly find the fastest route to a destination, make purchases with our voice, and get recommendations based on our previo<\/span>us purchases<\/span>. <\/span>T<\/span>hese examples of machine learning have impacted our lives <\/span>positively,<\/span> saving customers time and companies money.<\/span>\u00a0<\/span><\/p>\n

Machine learning is the scientific study of algorithms and statistical models that result in devices automatically learning and improving from experiences <\/span>without being explicitly programmed<\/span><\/b>. With so much success integrating machine learning into our everyday lives, the <\/span>obvious <\/span>next step is to <\/span>integrate <\/span>machine learning into even more systems. <\/span>Unfortunately,<\/span> the collected data used to train machine learning models is often riddled with bias.\u00a0\u00a0<\/span>\u00a0<\/span><\/p>\n

Every time a dataset includes human decisions there is bias. Bias is learned stereotypes or prejudice in favor or against one thing or another. It can be conscious or unconscious. Datasets are vulnerable to bias during the data cleaning process where data is <\/span>manipulated <\/span>to increase overall effectiveness and accuracy. Both missing data and outliers need to be handled during data cleaning. If done inaccurately, bias may be introduced into the <\/span>model and machine<\/span>.\u00a0<\/span>\u00a0<\/span><\/p>\n

Missing Data<\/span>\u00a0<\/span><\/h3>\n

There are three main types of missing <\/span>data:<\/span> missing completely at random (MCAR), missing at random (MAR), and missing not at random or non-ignorable (MNAR\/NI). In MCAR, the cause of missing data is independent of any values and\/or variables in the dataset (i.e. a broken test tube). This is uncommon but ideal because analysis remains unbiased. MAR is where the cause of the missing data is related to some of the non-missing values of other variables. Bias may be introduced when analyzing if an observation is dependent on a missing value (MCAR or MAR) and how to handle it. For example, a dataset of patient information includes age and blood pressure. A data scientist notices multiple blood pressure observations missing from patient data. During data cleaning, she concludes that the missing blood pressure observations are MCAR and deletes <\/span>all<\/span> the patient information for anyone who is missing a blood pressure reading. What she doesn’t know is that doctors are less likely to take the blood pressure of younger patients, so the missing data is MAR, not MCAR. The data scientist has now unconsciously introduced bias toward younger patients into her dataset and, eventually, her model. A similar problem can happen with MNAR data because it is missing data whose cause is related to other missing data (i.e. people with higher and lower income are less likely to provide their income information). It is important to correctly identify and handle missing data to prevent bias.<\/span>\u00a0<\/span><\/p>\n

Outliers<\/span><\/h3>\n

Sometimes there are observations that are significantly distant from other observations in a dataset. These are called outliers and are caused by errors within a dataset or natural variability. While outliers can reveal valuable information about a dataset, it can also skew your model and lead to less than optimal results. It is up to the data scientist to determine if outliers need to be removed, replaced, or remain in the dataset. If natural variability is treated as an error by mistake, a data scientist will remove the observation and valuable information will be lost. For example, in the early stages of the Flint Water Crisis in Flint, Michigan, water samples were taken from select homes in the community and tested for lead. The results revealed that <\/span>a majority of<\/span> homes had safe levels of lead in their water supply, but a couple of homes came back with results of dangerously high levels of lead in the water supply. The cause of the outliers was deemed human error and they were consequently removed from the dataset. The revised dataset was then sent back to the appropriate authorities and no action was taken to improve the water quality. In this case, the outlier was not dealt with appropriately and, as a result, introduced bias into the dataset, putting the health of people at risk.<\/span>\u00a0<\/span><\/p>\n

Bias can have dangerous consequences. The effects of writing our unconscious bias into machine learning models can make a machine, whose task is efficiency, just as flawed as human beings. \u00a0As flawed as we are \u2212 How can we fix a problem that we cannot see? \u00a0While Machine Learning is a powerful tool that brings values to\u00a0 many industries and problems, it\u2019s critically important to be aware of the inherent bias humans bring to the table.\u00a0 This is also a key reason that ethical principles must be considered in the future of AI.\u00a0 To learn more about what Microsoft is doing in this space, visit Microsoft AI Principals<\/a>.<\/p>\n

\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Machine learning is the scientific study of algorithms and statistical models that result in devices automatically learning and improving from experiences without being explicitly programmed. With so much success integrating machine learning into our everyday lives, the obvious next step is to integrate machine learning into even more systems.<\/p>\n","protected":false},"author":582,"featured_media":38844,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[14],"tags":[15,9459,265,3],"class_list":["post-38842","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-ai","tag-ethics","tag-machine-learning","tag-team"],"acf":[],"blog_post_summary":"

Machine learning is the scientific study of algorithms and statistical models that result in devices automatically learning and improving from experiences without being explicitly programmed. With so much success integrating machine learning into our everyday lives, the obvious next step is to integrate machine learning into even more systems.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/38842","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/users\/582"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/comments?post=38842"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/38842\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media\/38844"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media?parent=38842"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/categories?post=38842"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/tags?post=38842"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}