Colin Carroll, Karin C. Knudson - Fighting Gerrymandering with PyMC3

At the end of 2017, there were seven states with ongoing redistricting litigation. We will discuss a statistical model that the United States Supreme Court declared to be appropriate in cases of racial gerrymandering, and show how it can be implemented and used with the library `PyMC3`. We will also discuss what the model tells us about racial gerrymandering in North Carolina.

https://us.pycon.org/2018/schedule/presentation/110/

Transcript

1. FIGHTING GERRYMANDERING WITH PYMC3 Dr. Karin Knudson and Dr. Colin Carroll
2. PLAN FOR THE DAY History of gerrymandering What do we need to model? Ecological inference Modelling fake data Modelling real data
3. WHERE I’M COMING FROM
4. THE VOTING RIGHTS ACT OF 1965 “ I want you to write me the goddamndest, toughest, voting rights act that you can devise” - LBJ to Attorney General Nicholas Katzenback
5. "So long as I do not firmly and irrevocably possess the right to vote I do not possess myself. I cannot make up my mind — it is made up for me. I cannot live as a democratic citizen, observing the laws I have helped to enact — I can only submit to the edict of others.” - Martin Luther King
6. VOTING RIGHTS ACT Section 2 - Prohibits racial discrimination in voting, vote denial and vote dilution Section 3 - Bail-in Section 4 - Coverage formula Section 5 - Pre-clearance http://www.nytimes.com/interactive/2013/06/23/us/voting-rights-act-map.html
7. SHELBY V. HOLDER (2013) Section 4 coverage formula unconstitutional Leaves section 5 preclearance intact but unenforceable Section 2 intact - challenges still possible and still slow and expensive Will congress write a new coverage formula? Credit Pablo Martinez Monsivais/Associated Press “the Act imposes current burdens and must be justified by current needs.”
8. THORNBURG V. GINGLES (1986) WHAT DOES A VRA CHALLENGE REQUIRE? AMONG OTHER GUIDANCE…. 1. The history of official discrimination in the jurisdiction that affects the right to vote; 2. The degree to which voting in the jurisdiction is racially polarized; 3. The extent of the jurisdiction's use of majority vote requirements, unusually large electoral districts, prohibitions on bullet voting, and other devices that tend to enhance the opportunity for voting discrimination; 4. Whether minority candidates are denied access to the jurisdiction's candidate slating processes, if any; 5. The extent to which the jurisdiction's minorities are discriminated against in socioeconomic areas, such as education, employment, and health; 6. Whether overt or subtle racial appeals in campaigns exist; 7. The extent to which minority candidates have won elections; 8. The degree that elected officials are unresponsive to the concerns of the minority group; and 9. Whether the policy justification for the challenged law is tenuous. 1. compactness - racial/language minority group is “sufficiently numerous and compact to form a majority in a single- member district” 2. the minority group is “politically cohesive” 3. “majority votes sufficiently as a block to enable it…usually to defeat the minority’s preferred candidate” SENATE FACTORS (1982)
9. QUESTIONS OF INTEREST - ECOLOGICAL INFERENCE GROUP A GROUP B Group 1 bi11 = ?? bi12 = ?? X1,i Group 2 bi21 = ?? bi22 = ?? X2,i T1,i T2,i i = 1,…, p i
10. precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White precinct 2: 40% Dem, 60% Rep, 35% Black, 65% White PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i T1,i T2,i QUESTIONS OF INTEREST i = 1,…, p i
11. QUESTIONS OF INTEREST i = 1,…, p PRECINCT DEMOCRAT REPUBLICAN NO VOTE Black bi11 = ?? bi12 = ?? bi13 = ?? X1,i White bi21 = ?? bi22 = ?? bi23 = ?? X2,i Other bi31 = ?? bi32 = ?? bi33 = ?? X3,i T1,i T2,i T3,i i
12. WISHLIST Give accurate estimates when possible Give possible estimates (e.g. rates between 0% and 100%!) Quantify uncertainty Make assumptions explicit Results clearly communicable to courts (!)
13. GOODMAN’S ECOLOGICAL REGRESSION "requires, in all but the smallest of jurisdictions, reliance on computers to perform the calculations." PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i i = 1,…, p Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic i
14. METHOD OF BOUNDS precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White Note: 0.9 = .85b11 + .15b12 .33 < b12 ≤ 1 PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i
15. KING’S ECOLOGICAL INFERENCE precinct 1: 70% Dem, 30% Rep, 65% Black, 35% White precinct 2: 50% Dem, 50% Rep, 25% Black, 75% White .7 = .65b11 + .35b12 .5 = .25b11 + .75b12 b11 b12
16. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL c1~ Exponential(λ) d1~ Exponential(λ) c2~ Exponential(λ) d2~ Exponential(λ) bi1 | c1,d1 ~ Beta(c1,d1) i.i.d. bi2 | c2,d2 ~ Beta(c2,d2) i.i.d. θi=Xi bi1 + (1-Xi)b2i T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) λ c1 d1 c2 d2 bi1 T’i p bi2 King, Rosen, and Tanner
17. WITH COVARIATES p(α) = p(β) = p(γ) = p(δ)∝1 d1 ~ Exponential(λ) d2 ~ Exponential(λ) bi1 | Zi, d1, α, β ~ Beta(d1exp(α+βZi ), d1) bi2 | Zi, d2, γ, δ ~ Beta(d2exp(γ+δZi ), d2) θi=Xi b1i + (1-Xi) bi2 T’i ~ Binomial(Ni,θi) Note: log (b1i)/( 1-(b1,i)) =α+βZi
18. WITH MORE CATEGORIES (RXC) rc ~ Exponential(λ1) i.i.d. bir | r ~ Dirichlet(r) i.i.d. r = 1,…,R θic=Xir bi1c + XiRbiRc c = 1… C T’ic ~ Multinomial(Ni,θi) bi1 biR T’i p 1 R λ vectors of length C
19. WITH MORE CATEGORIES AND COVARIATES dr ~ Exponential (λ) i.i.d. r = 1….R r = (dr exp( r1 + r1 Zi),…,dr exp( rC-1 + rC-1 Zi), dr ) r = 1,…,R bri | r ~ Dirichlet(r) i.i.d. r = 1,…,R θic=Xi1 bi1c + XiRbiRc c = 1,…,C T’ic ~ Multinomial(Ni,θi) Note: log (birc)/( 1-(birc)) = rc + rc Zi λ d1 dR T’i p Zi R R bi1 biR 1 1
20. A HIERARCHICAL BAYESIAN APPROACH - SIMULATION STUDY
21. SIMULATING DATA # Hidden minority_pct_vote_for_pete = np.random.rand(n_precincts) * 0.3 + 0.7 majority_pct_vote_for_pete = np.random.rand(n_precincts) * 0.3
22. SIMULATING DATA # Observed pct_minority = np.random.rand(n_precincts) pct_voting_for_pete = (pct_minority * minority_pct_vote_for_pete + (1 - pct_minority) * majority_pct_vote_for_pete)
23. SIMULATING DATA # Observed voting_population = np.random.randint(100, 10000, size=n_precincts) num_voting_for_pete = (pct_voting_for_pete * voting_population)
24. MODELLING!
25. VISUALIZING RESULTS
26. NORTH CAROLINA
27. NORTH CAROLINA
29. NORTH CAROLINA • 6.8M out of 7.1M voting-age people • 3.1M out of 4.6M votes • 2,592 voting divisions
30. NORTH CAROLINA source: cnn.com
31. politico.com
32. politico.com
33. politico.com
35. OVERALL VOTE FOR DEMOCRATIC CANDIDATES
36. ESTIMATED NON-WHITE VOTE FOR DEMOCRATIC CANDIDATES
37. ESTIMATED WHITE VOTE FOR DEMOCRATIC CANDIDATES
38. IT’S A GREAT TIME… TO CARE ABOUT GERRYMANDERING 2010 2013 2020 2021 Census Census Major redistricting Shelby vs. Holder Supreme Court on gerrymandering in: WI, NC, …. Now-ish
39. THANK YOU!
40. • King, Gary and Roberts, Molly.(2016). ei: Ecological Inference. R package version 1.3-3. • King, Gary, Ori Rosen and Martin A. Tanner. Ecological Inference: New Methodological Strategies • King, Gary, Ori Rosen, and Martin A. Tanner. "Binomial-beta hierarchical models for ecological inference." Sociological Methods & Research 28.1 (1999): 61-90. • King, Gary (1997). A Solution to the Ecological Inference Problem. Princeton, NJ: Princeton University Press. • Rosen, Ori, et al. "Bayesian and frequentist inference for ecological inference: The R× C case." Statistica Neerlandica 55.2 (2001): 134-156. • James Greiner, “Ecological Inference in Voting Rights Act Disputes: Where Are We Now, and Where Do We Want to Be?”, 47 Jurimetrics J. 115-167 (2007). • Bullock, Charles S. et al. The Rise and Fall of the Voting Rights Act. Oklahoma University Press: Norman, 2006. • Thornburg v. Gingles, 478 U.S. 30,49 (1986). • Shelby County v. Holder, No. 570 U.S 12-96, (2013) • Metric Geometry and Gerrymandering Group https://sites.tufts.edu/gerrymandr/ (particular acknowledgment to Dr. Mira Bernstein of the MGGG and Dr. Megan A. Gall of the Lawyers’ Committee for Civil Rights Under Law) • NC data from: OpenElections http://www.openelections.net/ and the US Census