Computing Bias and Crowdsourcing: Unfair Data, Shared Effort

Q: What is check your understanding in Python?

Exam topics: 5.3 Computing Bias (IOC-1.D), 5.4 Crowdsourcing (IOC-1.E)

Beginner30 min3 exercises40 XP

Prerequisites

Python Lists: Create, Access, Modify, and Loop Through Lists Python Dictionaries: The Complete Guide with Real Examples Python Functions: Define, Call, and Return Values

0/3 exercises

A program can treat people unfairly even when nobody meant it to, because people write its rules and choose the data it learns from.

Computing innovations can reflect existing human biases in two main ways. Bias can be written into the algorithm, through the rules a programmer chooses. Bias can also sit in the data the innovation uses or learns from.

How does bias get into a computing innovation?

Bias in the rules often starts as a choice that seemed harmless. A sign-up form that rejects any name containing an apostrophe or a hyphen locks out people named O'Brien or Smith-Jones. A form that demands both a first and a last name fails people who have only one name, which is common in Indonesia. Nobody set out to exclude anyone; the programmer pictured names like their own.

Bias in the data is harder to spot. In 2018 news reports revealed that Amazon had scrapped an experimental tool for rating job applicants. It had learned from ten years of past résumés, most of them from men, and it marked down résumés that included the word "women's", as in "women's chess club captain". The same year, researchers Joy Buolamwini and Timnit Gebru tested commercial face-analysis software and found that it guessed gender wrong for up to about a third of darker-skinned women, but for less than 1% of lighter-skinned men. Popular collections of face photos used to build and test such systems were made up mostly of lighter-skinned faces. A follow-up test months later found that all three companies named in the study had narrowed the gap, a sign that measuring bias in public pushes programmers to fix it.

Bias can be embedded at every level of software development:

Choosing the problem: whose problems get solved first, and whose get ignored.

Collecting data: who appears in the data, and who is missing.

Designing the algorithm: which rules, and which measure of "success", the programmer picks.

Testing: who the testers are. An app tested only by its own team is tested by people who think like that team.

Releasing it: who can afford it, and which languages and devices it supports.

How does skewed data make a program unfair?

Automatic soap dispensers and faucets use a sensor that shines invisible light and measures how much bounces back off a hand. Darker skin reflects less of that light, and some dispensers became famous online for not noticing darker-skinned hands.

The toy model below "learns" a detection rule from sample readings, the way a very simple machine learning system would: it places the cutoff halfway between a typical "no hand" reading and a typical "hand" reading. Its training data has ten hands with lighter skin and only one with darker skin. Every number is invented.

A toy sensor trained on skewed data

# Toy hand sensor. All readings are INVENTED, on a 0-100 scale.

no_hand = [5, 8, 6, 7, 4]

training_hands = [80, 85, 78, 90, 82, 88, 84, 79, 86, 83, # lighter skin

40] # darker skin

def learn_threshold(no_hand, hands):

"""Put the cutoff halfway between the average no-hand and hand readings."""

no_hand_avg = sum(no_hand) / len(no_hand)

hand_avg = sum(hands) / len(hands)

return (no_hand_avg + hand_avg) / 2

def detection_rate(readings, threshold):

detected = [r for r in readings if r >= threshold]

return 100 * len(detected) / len(readings)

threshold = learn_threshold(no_hand, training_hands)

print(f"Learned threshold: {threshold:.1f}")

lighter_test = [81, 77, 92, 84, 88]

darker_test = [38, 45, 41, 50, 36]

print(f"All test hands detected: {detection_rate(lighter_test + darker_test, threshold):.0f}%")

print(f"Lighter skin detected: {detection_rate(lighter_test, threshold):.0f}%")

print(f"Darker skin detected: {detection_rate(darker_test, threshold):.0f}%")

Output

Learned threshold: 42.8
All test hands detected: 70%
Lighter skin detected:   100%
Darker skin detected:    40%

An overall rate of 70% might pass a quick check. Split by group, the problem is obvious: the sensor works every time for one group and fails more often than it works for the other. That is why results should always be measured separately for each group.

Collecting more of the same skewed data does not remove the bias: adding ten more lighter-skin samples would leave the darker-skin rate at 40%. What fixes it is a balanced sample, which moves the cutoff to a place that works for everyone:

The same toy sensor trained on balanced data

# Same toy sensor, balanced training sample (still INVENTED numbers)

no_hand = [5, 8, 6, 7, 4]

balanced_hands = [80, 85, 78, 90, 82, # lighter skin

40, 44, 37, 47, 42] # darker skin

no_hand_avg = sum(no_hand) / len(no_hand)

hand_avg = sum(balanced_hands) / len(balanced_hands)

threshold = (no_hand_avg + hand_avg) / 2

print(f"Learned threshold: {threshold:.1f}")

test_groups = {"Lighter": [81, 77, 92, 84, 88], "Darker": [38, 45, 41, 50, 36]}

for group, readings in test_groups.items():

detected = sum(1 for r in readings if r >= threshold)

print(f"{group} skin detected: {100 * detected / len(readings):.0f}%")

Output

Learned threshold: 34.2
Lighter skin detected: 100%
Darker skin detected: 100%

How can programmers reduce bias?

Programmers should act to reduce bias in the algorithms they build. Otherwise computing repeats existing human biases, only faster and for millions of people at once. Steps that help:

Build diverse teams, and ask a wide range of users for feedback while designing.

Check who is in the data, and collect data from the groups that are missing.

Measure results separately for each group, as the demo did, instead of trusting one overall number.

Question every rule: does a name field really need to reject apostrophes?

Keep checking after release, and give users a way to report unfair results.

Exercise 1: Measure a Detection Rate

Write Code

Write detection_rate(readings, threshold) that returns the percent of readings that are at or above threshold, rounded to 1 decimal place. If readings is empty, return 0.0 instead of crashing.

detection_rate([50, 30, 45, 20], 40)  ->  50.0

def detection_rate(readings, threshold):

# Return the percent of readings >= threshold, rounded to 1 decimal

pass

Exercise 2: Accuracy for Each Group

Write Code

A face-unlock feature was tested on many people. Each result is a tuple (group, was_correct). Write accuracy_by_group(results) that returns a dictionary mapping each group to the percent of its results that were correct, rounded to 1 decimal place. Groups should appear in the order they first show up in results.

accuracy_by_group([("A", True), ("A", True), ("B", False), ("B", True)])
->  {"A": 100.0, "B": 50.0}

def accuracy_by_group(results):

# results is a list of (group, was_correct) tuples

accuracy = {}

return accuracy

Check your understanding

A company trains a program to rank job applicants using records of the people it hired over the past ten years, most of whom went to the same few colleges. The program now ranks applicants from other colleges lower. What is the most likely cause?

AThe program learned a bias that was already present in its training data.BThe program has a syntax error in its ranking procedure.CApplicants from other colleges typed their information incorrectly.DThe company did not have enough computing power to rank everyone fairly.

Which two actions would most help programmers reduce bias in a speech-recognition app? Select two answers.

Select 2 answers.

ATesting the app with speakers of many accents and measuring accuracy for each groupBCollecting many more recordings from the same group of speakers already in the dataCIncluding team members and test users with a range of backgrounds and accentsDKeeping the training data secret so nobody can criticize it

Which statement about bias in computing innovations is accurate?

ABias can only enter a program through its training data.BBias can be embedded at any level of development, from choosing the problem to testing and release.CA program that passes all of its test cases cannot be biased.DBias is only possible in programs that use machine learning.

A hand sensor detects a hand when its reading is at least 45. The list holds the readings from five hands in a group that was under-represented in the training data. What is displayed?

readings ← [52, 38, 61, 44, 35]
count ← 0
FOR EACH r IN readings
{
  IF(r ≥ 45)
  {
    count ← count + 1
  }
}
DISPLAY(count)

A1B2C3D5

Run this pseudocode

How do people solve problems together at scale?

When information and public data are easy to reach, many more people can spot problems, build solutions, and share what they find. Many cities publish open data about potholes, bus delays and air quality, and anyone can download it and build tools from it. Open data helps at every step: spotting a problem (a map showing asthma cases clustered near a highway), developing a solution (a volunteer-built app that texts riders when their bus is late), and sharing the results where anyone can check them. During the COVID-19 pandemic, a dashboard built at Johns Hopkins University combined case counts from public sources around the world, and news outlets, governments and researchers checked it every day.

Science in particular has been changed by distributed computing and citizen science: research done in whole or in part by many people spread out in different places, most of them not scientists, who contribute data using their own computing devices.

On eBird, birdwatchers log what they see in a phone app, and scientists at the Cornell Lab of Ornithology use the records to track how bird populations move and change.

On iNaturalist, people photograph plants and animals, and other users help identify them, building a map of where species live.

In the online game Foldit, players fold virtual proteins. In 2011 players worked out the structure of a virus protein within weeks, a problem that had stumped scientists for about 15 years.

Folding@home borrows unused processing power from volunteers' computers to simulate proteins. In 2020, as volunteers joined to study the virus behind COVID-19, it passed an exaflop, a quintillion calculations per second, which was more raw computing power than the fastest supercomputer of the time.

Crowdsourcing is getting input or information from a large number of people over the Internet. Wikipedia is written and corrected by volunteers. Map apps such as Waze ask drivers to report crashes and slowdowns, then reroute everyone else. After the 2010 earthquake in Haiti, OpenStreetMap volunteers around the world mapped the damaged streets from satellite photos within weeks, and rescue teams used those maps. Even the old "type the words you see" puzzles were crowdsourcing: early reCAPTCHA had people type words from scanned books and newspapers that computers could not read.

Collaboration through computing enhances what humans can do. No single birdwatcher can follow a migration across a continent, but a hundred thousand birdwatchers with phones can.

Crowdsourcing also creates new ways to connect businesses and social causes with money. On crowdfunding sites, many people each give a small amount toward a goal. In 2012 the makers of the Pebble smartwatch raised over $10 million on Kickstarter from about 69,000 backers. On DonorsChoose, teachers post classroom projects, from microscopes to robotics kits, and anyone can chip in.

Crowds can also be surprisingly accurate when their answers are combined. The simulation below asks 200 made-up people to guess how many jelly beans are in a jar, then adds two joke answers. The median, the middle value, ignores the jokes.

A simulated crowd guesses

import random

import statistics

random.seed(42) # same "random" guesses every run

true_count = 850 # jelly beans in the jar

# 200 simulated people each make a rough guess, plus two joke answers

guesses = [round(random.gauss(true_count, 250)) for _ in range(200)]

guesses += [99999, 1]

typical_miss = statistics.median(abs(g - true_count) for g in guesses)

print(f"A typical person misses by about {typical_miss:.0f} beans")

print(f"Mean of all guesses: {statistics.mean(guesses):.0f}")

print(f"Median of all guesses: {statistics.median(guesses):.0f}")

Output

A typical person misses by about 152 beans
Mean of all guesses:   1365
Median of all guesses: 886

A typical person misses by about 150 beans, but the crowd's median misses by under 40. One joke answer of 99,999 drags the mean far off, which is why crowdsourcing projects check and clean what people send in.

Exercise 3: The Crowd's Estimate

Write Code

Write crowd_estimate(guesses) that returns the median of a non-empty list of guesses. Sort the guesses; the median is the middle value. When there is an even number of guesses, the median is the average of the two middle values.

crowd_estimate([40, 52, 48])        ->  48
crowd_estimate([40, 52, 48, 1000])  ->  50.0

def crowd_estimate(guesses):

ordered = sorted(guesses)

# Return the middle value (or the average of the two middle values)

pass

Check your understanding

Which of these is an example of citizen science?

AStudents count birds in a park and upload their sightings to a research database using a phone app.BA university lab hires three ornithologists to count birds at a wildlife refuge.CA company surveys its own employees about their favorite lunch spots.DA scientist reads published research papers about bird migration.

Which two are examples of crowdsourcing? Select two answers.

Select 2 answers.

ADrivers report a crash in a map app, which then warns and reroutes other drivers.BThousands of volunteers write and edit articles for an online encyclopedia.CA team of five engineers writes the code for a map app.DA weather service builds its forecast using data from its own sensors only.

A musician raises the money for her first album by collecting small pledges from 2,000 fans on a website. Which statement best describes this?

AIt is a crowdsourcing model that connects a project with funding from many people.BIt is citizen science, because fans contribute data to research.CIt is an example of the digital divide.DIt is a form of targeted advertising.

Exam topics: 5.3 Computing Bias (IOC-1.D), 5.4 Crowdsourcing (IOC-1.E)

How does bias get into a computing innovation?

How does skewed data make a program unfair?

How can programmers reduce bias?

Check your understanding

How do people solve problems together at scale?

Check your understanding

Related Tutorials