AI Weekly: Algorithmic discrimination highlights the need for regulation

How open API standards will transform financial services Open standards will have a huge impact on driving innovation in banking. Learn the status in the U.S. and the bold new opportunities open standards are set to usher in. Register here The Transform Technology Summits start October 13th with Low-Code/No Code: Enabling Enterprise Agility. Register now! This week, a piecefrom The Makeup uncoveredbiases in U.S. mortgage-approval algorithms that lead lenders to turn down people of color more often than white applicants. A decisioning model called Classic FICO didnt consider everyday payments like on-time rent and utility checks, among others and instead rewarded traditional credit, to which Black, Native American, Asian, and Latino Americans have less access than white Americans. The findings arent revelatory: back in 2018, researchers at the University of California, Berkeley found that mortgage lenders charge higher interest rates to these borrowers compared to white borrowers with comparable credit scores. But they do point to the challenges in regulating companies that riskily embrace AI for decision-making, particularly in industries with the potential to inflict real-world harms. The stakes are high. Stanford and University of Chicago economists showed in a June report that, because underrepresented minorities and low-income groups have less data in their credit histories, their scores tend to be less precise. Credit scores factor into a range of application decisions, including credit cards, apartment rentals, car purchases, and even utilities. In the case of mortgage decisioning algorithms, Fannie Mae and Freddie Mac, home mortgage companies created by Congress, told The Markup that Classic FICO is routinely evaluated for compliance with fair lending laws internally and by both the Federal Housing Finance Agency and the Department of Housing and Urban Development. But Fannie and Freddie have over the past seven years resisted efforts by advocates, the mortgage and housing industries, and Congress to allow a newer model. Algorithmic discrimination The financial industry isnt the only party guilty of discrimination by algorithm, equality and fairness laws be damned. Last year, a Carnegie Mellon University study found that Facebooks ad platform behaves prejudicially against certain demographics, sending ads related to credit cards, loans, and insurance disproportionately to men versus women. Meanwhile, Facebook rarely showed credit ads of any type to users who chose not to identify their gender, the study showed, or who labeled themselves as nonbinary or transgender. Laws on the books including the U.S. Equal Credit Opportunity Act and the Civil Rights Act of 1964 were written to prevent this. Indeed, in March 2019, the U.S. Department of Housing and Urban Development filed suit against Facebook for allegedly discriminating against people based upon who they are and where they live, in violation of the Fair Housing Act. But discrimination continues, a sign that the algorithms responsible and the power centers creating them continue to outstrip regulators. The European Unions proposed standards for AI systems, released in April, come perhaps the closest to reigning in decisioning algorithms run amok. If adopted, the rules would subject high-risk algorithms used in recruitment, critical infrastructure, credit scoring, migration, and law enforcement to strict safeguards and ban outright social scoring, child exploitation, and certain surveillance technologies. Companies breaching the framework would face fines of up to 6% of their global turnover or 30 million euros ($36 million), whichever is higher. Piecemeal approaches have been taken in the U.S. to date, such as a proposed law in New York City to regulate the algorithms used in recruitment and hiring. Cities including Boston, Minneapolis, San Francisco, and Portland have imposed bans on facial recognition, and Congressional representatives including Ed Markey (D-Mass.) and Doris Matsui (D-CA) have introduced legislation to increase transparency into companies development and deployment of algorithms. In September, Amsterdam and Helsinki launched algorithm registries to bring transparency to public deployments of AI. Each algorithm cited in the registries lists datasets used to train a model, a description of how an algorithm is used, how humans use the prediction, and how algorithms were assessed for potential bias or risks. The registries also provide citizens a way to give feedback on algorithms their local government uses and the name, city department, and contact information for the person responsible for the responsible deployment of a particular algorithm This week, China became the latest to tighten its oversight of the algorithms companies use to drive their business. The countrys Cyberspace Administration of China said in a draft statement that companies must abide by ethics and fairness principles and shouldnt use algorithms that entice users to spend large amounts of money or spend money in a way that may disrupt public order, according to Reuters. The guidelines also mandate that users be given the option to turn off algorithm-driven recommendations and that Chinese authorities be provided access to the algorithms with the choice of requesting rectifications, should they find problems. In any case, its becoming clear if it wasnt already that industries are poor self-regulators where AI is concerned. According to a Deloitte analysis, as of March, 38% of organizations either lacked or had an insufficient governance structure for handling data and AI models. And in a recent KPMG report, 94% of IT decision makers said they feel that firms need to focus more on corporate responsibility and ethics when developing their AI solutions. A recent study found that few major AI projects properly address the ways that technology could negatively impact the world. The findings, which were published by researchers from Stanford, UC Berkeley, the University of Washington, and University College Dublin Lero, showed that dominant values were operationalized in ways that centralize power, disproportionally benefiting corporations while neglecting societys least advantaged. A survey by Pegasystems predicts that if the current trend holds, a lack of accountability within the private sector will lead to governments taking over responsibility for AI regulation over the next five years. Already, the results seem prescient. For AI coverage, send news tips toKyle Wiggers and be sure to subscribe to the AI Weekly newsletterand bookmark our AI channel,The Machine. Thanks for reading, Kyle Wiggers AI Staff Writer VentureBeat VentureBeats mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access: up-to-date information on the subjects of interest to you our newsletters gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More networking features, and more Become a member .article-content .boilerplate-after { background-color: #F5F8FF; padding: 30px; border-left: 4px solid #000E31; line-height: 2em; margin-top: 20px; margin-bottom: 20px; } .article-content .membership-link { background-color: #000E31; color: white; padding: 10px 30px; font-family: Roboto, sans-serif; text-decoration: none; font-weight: 700; font-size: 18px; display: inline-block; } .article-content .membership-link:hover { color: white; background-color: #0B1A42; } .article-content .boilerplate-after h3 { margin-top: 0; font-weight: 700; } .article-content .boilerplate-after ul li { margin-bottom: 10px; } @media (max-width: 500px) { .article-content .boilerplate-after { padding: 20px; } } The post AI Weekly: Algorithmic discrimination highlights the need for regulation appeared first on Patabook Technology.

source https://patabook.com/blogs/108238/AI-Weekly-Algorithmic-discrimination-highlights-the-need-for-regulation