Differential Privacy
& Federated Learning

The Illusion of Invisibility

Set up the core tension of the whole unit: the usefulness of data versus the harm that detail creates. Don't belabor specific examples — let the students connect it to things they already care about.

Transition: "The field that tries to resolve this paradox is called privacy-enhancing technology, and to understand where it is now, we have to go back to 1996."

1996 — Setting the Scene

Ground the students in the era. The internet is new, spreadsheets are the primary tool, and the standard approach to "anonymization" is: delete names, delete addresses, delete Social Security numbers, and you're done.

This matters because the failure we're about to see isn't a failure of the people involved — they did what everyone thought was enough. It's a failure of the *assumption* that deletion equals anonymity.

Direct Identifiers

The Group Insurance Commission wanted to help researchers study healthcare costs. Their approach to anonymization was the standard of the era: delete the fields that *obviously* identify someone — name, address, SSN — and publish the rest.

Click "Remove Direct Identifiers" to show what the released version looked like. The direct-ID columns go black. Weld's portrait blurs — because now he's just another row in a sea of "anonymous" patients.

This looked safe. It wasn't. The next few slides show why.

Latanya Sweeney — Proving the Point

Sweeney, then an MIT researcher (now one of the world's leading privacy scholars), saw the GIC release and saw an obvious hole. She decided to make the vulnerability concrete by picking the most visible possible target: the governor.

She walked into the Cambridge city office and paid $20 for the public voter registration list. Now she had two spreadsheets. One had diagnoses. The other had names. Both had three fields in common: birth date, ZIP code, and gender.

Walk the class through the voter table on screen. Note Weld's row: 1945-07-31, male, 02138. Those three values will come back in the next two slides.

Payoff: Sweeney mailed Weld his own medical records to prove the vulnerability. This is one of the most cited demonstrations in the history of privacy research.

The Linkage Attack

The technique has a name: linkage attack. You take two datasets that each look safe on their own, find the columns they share, and use one to re-identify records in the other.

The key insight: anonymity isn't a property of a single dataset. It's a property of a dataset *in the context of everything else that's publicly known*. And in 1996, as now, an enormous amount of data is publicly known.

Interactive: Finding the Governor

Walk through the filters one at a time. Let the class call out what they notice. After the first filter, we're down to 6 matches on each side. After gender, 3. After ZIP, 1.

The mathematical inevitability of that final "1" is the whole lesson. There was only ever going to be one match.

87%

Let that number breathe. Sweeney's 2000 paper quantified what we just saw: three public fields are enough to re-identify the vast majority of Americans.

The crowd we were supposed to be hiding in — it isn't that big. Or maybe we're all much more unique than we thought.

The Spartacus Strategy — k-Anonymity

The first serious attempt to fix the linkage attack was k-anonymity, introduced by Sweeney herself a few years after the Weld incident.

The intuition is the Spartacus scene. If everyone in the group looks the same, the Romans can't pick out the real target.

Mechanically, k-anonymity uses two tools: generalization (turning age 32 into "30–40", turning ZIP 12345 into "123**") and suppression (deleting outliers entirely — if you're the only 104-year-old in the dataset, you get cut).

Interactive: k-Anonymizing the Medical Records

Same 20 records students saw in the Finding the Governor demo. At k=1 it's the raw table. Click k=3 and the birth dates become 20-year buckets, ZIPs collapse to 021**, and the two records that can't find a group of 3 get suppressed.

At k=5 the buckets widen further. More rows fit into each group, but the data is even blurrier.

Emphasize the two techniques explicitly: generalization (replace with a bucket) and suppression (drop the outlier). These are the only tools k-anonymity has.

Set up the next slide: "So is this safe? Let's look at one more example and see what still leaks."

Interactive: k-Anonymity

Step through the three k values. At k=1 we see raw data. At k=3, age and ZIP are generalized but everyone is still present. At k=5, the generalization is heavier and the two outliers are suppressed.

Then hit Reveal. The homogeneity attack: the five males in the first group all have Heart Disease. The attacker doesn't need to know *which* male is the target. They know what condition he has, because everyone in his hiding group has the same condition.

k-anonymity was patched with l-diversity and t-closeness, but the bigger problem was the curse of dimensionality: in modern high-dimensional data, you'd have to suppress so much that the dataset becomes useless.

Differential Privacy (2006) — Frosted Glass

Dwork, McSherry, Nissim, and Smith published the foundational paper that created the field. The key move was a change of framing: instead of trying to sanitize a dataset so it was "safe to release," they defined a mathematical property of the release mechanism itself.

Plausible deniability: the output of a query should be effectively the same whether or not you, specifically, are in the database.

The frosted-glass metaphor is the intuition that makes DP click for most people. Through frosted glass, you can see the big picture — beach vs. forest, crowd vs. empty street — but not individual faces. Use the slider to show: low epsilon = heavy blur, high epsilon = clear. Same image, same underlying data; what changes is the privacy budget.

The mechanism: compute the true answer, then add random noise drawn from a carefully chosen distribution (Laplace, for the basic case). The noise scale depends on epsilon.

Interactive: The Epsilon Dial

Low epsilon = thick frosted glass = strong privacy, weak utility. High epsilon = thin glass = weak privacy, strong utility.

Click re-roll a few times so students see the noise is actually random. This is the key distinguishing feature from k-anonymity: there is no single "anonymized dataset" — each query draws fresh noise.

Emphasize: you cannot have perfect privacy *and* perfect accuracy. Someone has to decide where to set the dial.

DP in the Wild

Four real deployments. Apple's local DP is the one students are most likely to have encountered — noise added on-device before anything is transmitted.

The 2020 Census is the most consequential. Reconstruction attacks had become feasible enough that the Bureau switched methodology entirely. We'll come back to the consequences of that decision on the next slide.

The Dark Side — Minority Erasure

This is the honest reckoning. DP isn't a free lunch. The noise doesn't affect all groups equally.

Large groups absorb the noise without trouble. Small groups — which in demographic data usually means minority communities — can be noised into oblivion. A group of 3 + a random noise draw of -4 = reported as 0.

This isn't hypothetical. The 2020 Census noise levels were controversial for exactly this reason.

Interactive: Minority Erasure

Lower the epsilon and re-roll the noise a few times. The majority group (950) is untouched. The minorities (12, 7, 5, 3) collapse to zero at low epsilon, often.

Census data decides school placement, voting districts, and federal aid distribution. If the noise washes out a small community, they lose funding and political representation. The paradox: the people who most need protection from identification are also the ones most harmed by the loss of accuracy.

Researchers call the active work in this space "equitable differential privacy." It's unsolved.

Minority Erasure — What's at Stake

Two concrete examples of how noise in released counts becomes harm in the real world.

Funding and representation. Federal funding allocations (Title I schools, Medicaid matching, highway funds, — all tied to population counts), political boundary drawing, and where new public infrastructure gets built all run off census data. Small communities noised to zero become invisible to those processes.

Voting Rights Act Section 203. Requires bilingual ballots where a language-minority population exceeds 5% of eligible voters OR 10,000 people in a jurisdiction. The trigger depends entirely on the count being accurate. DP noise on the small end of those populations can push them below the threshold and strip the legal protection.

The memo on the link is from 2020, when the Bureau was finalizing its DP approach. Virginia's state demographer laid out concrete distortion cases and asked the governor to push back. Worth a click after class.

The Matthew Effect

The name comes from sociology of science, ultimately from Matthew 25:29: the rich get richer, the poor get poorer. When applied to DP training, it captures the failure mode exactly.

Mechanically: DP training adds noise at each gradient step, and clips per-example gradient norms so any single example can't influence the model too much. Both moves disproportionately damage signal from rare classes and outliers.

Walk through the four cards. Land on the last one: the fairness story and the privacy story are coupled. You can't fix one without thinking about the other.

Compliance & Privacy Theater

Regulation is catching up. GDPR (2018) is the landmark: it creates an explicit legal category for "anonymized" data that no longer identifies anyone, and carves that category out of the regulation entirely. So there is a large compliance incentive to achieve real anonymization.

Quebec's Law 25 (2022) goes further and specifies irreversibly anonymized. Under that wording, the old delete-the-names approach doesn't meet the bar. Penalties are severe: up to CAD $25M or 4% of global revenue. Companies operating in Quebec are effectively pushed toward techniques like DP.

The catch: the law mandates the technique but usually not the parameters. A company can adopt DP in name, configure it with a huge epsilon, and be formally compliant while leaking freely. This is the privacy theater trap — the move the law was trying to block reappears in a new costume.

Pedagogical payoff: privacy-as-math-guarantee only works if someone is watching the parameter, not just the label.

The Reframe

We've gone from the black marker to injecting measured chaos. From redaction to perturbation. We've accepted that we have to damage the data a little to save the people in it.

The key conceptual shift: privacy isn't a binary. It's a dial that balances three things pulling against each other.

Leave them with the question: is there a point where being *too* private actually hurts? Can you be so hidden from the systems that are supposed to help you that you get left behind? That's a real tradeoff. It doesn't have a clean answer.

Intro to Federated Learning

Transition slide into the federated learning half of the unit. Open the Google PAIR explorable live and walk through the first scroll-interactive together. Return to the deck afterward for the FL content.