Mayoral supporters and their votes for Council At-Large

Jonathan Tannen, Sixty Six Wards
Seth Bluestein, Philadelphia City Commissioner

Today, Sixty Six Wards is partnering with Philadelphia City Commissioner Seth Bluestein to look at individual voter patterns in the 2023 Democratic Primary.

The 2023 Democratic Primary saw mayoral candidates running in distinct lanes (Sixty Six Wards’ recap, preview). Parker won handily–in retrospect–as Rhynhart and Gym came in distant second and third. In Council, all of the winners were endorsed by the Democratic City Committee, with Santamoor, McIllmurray and Almirón–in 6th, 7th, and 8th–also apparently splitting various non-machine coalitions.

In February, Sixty-Six Wards teamed up with Commissioner Bluestein to look beyond division-aggregated data at individual voter level patterns for correlations and diversity within divisions. We’re doing it again.

The Commissioner’s office has generated that dataset with anonymized, aggregated counts of candidate combinations at the Division level by vote mode–election day, mail-in, provisional–for the 2023 primary.

Rather than a gigantic piece like last time, we will provide a series of short pieces with interesting insights from the data.

How did Mayoral supporters vote for Council?

Today, we examine how each mayoral candidate’s supporters voted for council. Among Divisions, there were strong correlations between Parker and Thomas performance, or Gym and Landau. Some of that may be ecological: supporters tend to live in the same places, but might not be the same people. Just how strong is the voter-level trend?

View code
library(dplyr)
library(ggplot2)
library(tidyr)

pairs <- readr::read_csv(
  "../../data/raw_election_data/pairs_new.csv",
  locale = readr::locale(encoding = "Latin1")
)
pairs_backup <- pairs
# pairs <- pairs_backup

toplines <- readr::read_csv(
  "../../data/raw_election_data/primary_2023_combinations_toplines.csv",
  locale = readr::locale(encoding = "Latin1")
)
voters <- readr::read_csv(
  "../../data/raw_election_data/primary_2023_combinations_voters.csv"
)
histogram <- readr::read_csv(
  "../../data/raw_election_data/primary_2023_combinations_histogram.csv",
  locale = readr::locale(encoding = "Latin1")
)

toplines$office_candidate <- factor(toplines$office_candidate)

levels_df <- data.frame(
  office_candidate=levels(toplines$office_candidate)
) |>
  separate(office_candidate, into=c("office_orig", "candidate_orig"), sep="::", remove=FALSE) |>
  mutate(
    office_orig = stringr::str_trim(office_orig),
    office = stringi::stri_trans_totitle(office_orig),
    office = gsub("(Dem|Rep)$", "(\\1)", office),
    candidate_orig = stringr::str_trim(candidate_orig),
    candidate = gsub("\\s*(DEM|REP) \\([0-9]+\\)$", "", candidate_orig),
    candidate = stringi::stri_trans_totitle(stringr::str_trim(candidate)),
    candidate = gsub("\\bMc([a-z])", "Mc\\U\\1", candidate, perl=TRUE),
    candidate = gsub("\\b([Ii]+)\\b", "\\U\\1", candidate, perl=TRUE),
  )

pairs <- pairs |> filter(candidate.x != "overvote")
pairs <- pairs |> filter(candidate.y != "overvote")

pairs$office.x <- factor(pairs$office.x)
pairs$office.y <- factor(pairs$office.y)
pairs$candidate.x <- factor(pairs$candidate.x)
pairs$candidate.y <- factor(pairs$candidate.y)

levels(pairs$office.x) <- levels_df$office[match(levels(pairs$office.x), levels_df$office_orig)]
levels(pairs$office.y) <- levels_df$office[match(levels(pairs$office.y), levels_df$office_orig)]
levels(pairs$candidate.x) <- levels_df$candidate[match(levels(pairs$candidate.x), levels_df$candidate_orig)]
levels(pairs$candidate.y) <- levels_df$candidate[match(levels(pairs$candidate.y), levels_df$candidate_orig)]

testthat::expect_false(any(is.na(pairs$office.x)))
testthat::expect_false(any(is.na(pairs$office.y)))
testthat::expect_false(any(is.na(pairs$candidate.x)))
testthat::expect_false(any(is.na(pairs$candidate.y)))

histogram$office_candidate <- factor(histogram$office_candidate, levels=levels(toplines$office_candidate))
histogram$office <- levels_df$office[histogram$office_candidate]
histogram$candidate <- levels_df$candidate[histogram$office_candidate]

toplines$office <- levels_df$office[toplines$office_candidate]
toplines$candidate <- levels_df$candidate[toplines$office_candidate]

Council Results by Mayor

View code
mayor_council <- pairs |>
  filter(
    office.y == "Mayor (Dem)", 
    office.x == "Council At-Large (Dem)"
  ) |> 
  rename(mayor = candidate.y, council=candidate.x) |>
  group_by(mayor, council, Type) |>
  summarise(counts = sum(n), .groups="drop")

# Add candidate, Type pairs that are missing
mayor_council <- expand.grid(
      mayor=unique(mayor_council$mayor), 
      Type=unique(mayor_council$Type),
      council=unique(mayor_council$council)
    ) |>
  left_join(mayor_council) |>
  mutate(counts = replace_na(counts, 0)) |>
  left_join(
    toplines |> 
      filter(office == "Mayor (Dem)") |> 
      group_by(candidate, Type) |> 
      summarise(topline_votes=sum(votes), .groups="drop"), 
    by=c("mayor" = "candidate", "Type"="Type")
  ) |>
  mutate(pvote = counts / topline_votes)

get_order <- function(office){
  toplines |>
    filter(office == !!office) |>
    group_by(candidate) |>
    summarise(votes = sum(votes), .groups="drop") |>
    arrange(desc(votes)) |>
    with(candidate)
}

council_order <- get_order("Council At-Large (Dem)")
mayor_order <- get_order("Mayor (Dem)")

mayor_council <- mayor_council |>
  mutate(
    council = factor(council, levels=council_order),
    mayor = factor(mayor, levels=mayor_order)
  )

ggplot(
  mayor_council |> 
    group_by(mayor, council) |> 
    summarise(topline_votes=sum(topline_votes), counts=sum(counts)),
  aes(x=council, y=100*counts/topline_votes)
) + 
  geom_bar(stat="identity") +
  facet_wrap(~mayor) +
  theme_minimal() %+replace% 
  theme(axis.text.x = element_text(angle=90, vjust=0.5, hjust=1, size=5)) +
  labs(
    title="Votes for Council At-Large by Mayoral Vote (Dem)",
    x=NULL,
    y="Percent of Mayoral voters who cast\nvote for Council candidate"
  ) 

Among Parker voters, the five eventual topline winners were on top. But the order was starkly different: third-place Rue Landau finished a distant fifth among this group. Conversely, Landau was the highest vote-getter among Gym voters, and among Rhynhart voters was on par with Thomas and Gilmore-Richardson. There are a few other noticeable peaks in the plot above: Santamoor and Itzkowitz received 36% and 25% of votes from Rhynhart voters, and McIllmurray and Almirón whopping 48% and 46% from Gym voters.

Amusingly, voters who wrote in choices for Mayor were vastly more likely to write in for council, too.

Those patterns didn’t change across vote mode, with Election Day voters and Mail-In voters showing similar preferences (the plots below show only the top four mayoral candidates, for sanity).

View code
ggplot(
  mayor_council |> 
    filter(mayor %in% c("Cherelle L Parker", "Rebecca Rhynhart", "Helen Gym", "Allan Domb")),
  aes(x=council, y=100*counts/topline_votes)
) + 
  geom_bar(stat="identity") +
  facet_grid(Type~mayor) +
  theme_minimal() %+replace% 
  theme(axis.text.x = element_text(angle=90, vjust=0.5, hjust=1, size=5)) +
  labs(
    title="Votes for Council At-Large by Mayoral Vote and Vote Type",
    x=NULL,
    y="Percent of Mayoral voters who cast\nvote for Council candidate"
  ) 

Notice that the average heights of the bars are different between facets; some mayoral candidates’ supporters cast more votes for Council. The heights of the bars would sum to 500 if every voter used all five votes, but only 400 if they used on average four, 300 if three, etc.

View code
mayor_council |>
  group_by(mayor, Type, topline_votes) |>
  summarise(counts = sum(counts), .groups="drop") |>
  mutate(votes_per_voter = counts/topline_votes) |>
  ggplot(
    aes(x=mayor, y=votes_per_voter)
  ) +
  geom_bar(stat="identity") +
  facet_grid(Type~.) +
  theme_minimal() %+replace% 
  theme(axis.text.x = element_text(angle=60, vjust=1, hjust=1)) +
  labs(
    title="Council At-Large votes per voter, by Mayoral choice",
    x=NULL,
    y="Average number of votes cast\nfor Council At-Large"
  )

Among Election Day voters, Gym and Rhynhart supporters cast 3.4 and 3.3 votes for Council At-Large, respectively. Parker supporters cast 2.7, and the rest of voters about 2.5. That difference vanished among Mail-In voters: all candidates’ supporters cast between 3.6 and 4.1 At-Large votes. An obvious explanation is that mail-in voters have time to look up down-ballot candidates with the ballot in hand that Election Day voters do not.

Suppose we recreated the above plot at the Division level: categorize Divisions by which mayoral candidate won, and calculate each council candidate’s percent (we’ll use total votes for Mayor as the denominator). The patterns are reasonably similar to the person-level results.

View code
div_mayor <- toplines |> filter(office == "Mayor (Dem)") |>
  group_by(Division, candidate) |> 
  summarise(votes = sum(votes)) |>
  group_by(Division) |>
  summarise(
    winner = candidate[which.max(votes)],
    pvote = votes[which.max(votes)] / sum(votes),
    mayoral_votes = sum(votes)
  ) |>
  mutate(mayor = factor(winner, levels=mayor_order))

div_council <- toplines |> filter(office == "Council At-Large (Dem)") |>
  group_by(Division, candidate) |> 
  summarise(votes = sum(votes)) |>
  mutate(council = factor(candidate, levels=council_order))

div_council |> 
  left_join(div_mayor, by="Division") |>
  group_by(mayor, council) |>
  summarise(
    votes = sum(votes),
    mayoral_votes = sum(mayoral_votes)
  ) |>
  ggplot(
    aes(x=council, y=100*votes / mayoral_votes)
  ) +
  geom_bar(stat="identity") +
  facet_wrap(~mayor) +
  theme_minimal() %+replace% 
  theme(axis.text.x = element_text(angle=90, vjust=0.5, hjust=1, size=5)) +
  labs(
    title="Votes for Council At-Large by Division's Mayoral Winner",
    x=NULL,
    y="Percent (Votes / Total votes for office of Mayor)"
  ) 

There are some interesting, small differences between the plots: Rhynhart Divisions supported McIllmurray over Harrity by 3.2pp, but Rhynhart voters supported Harrity over McIllmurray by 2pp. This is due to ecological correlations: Rhynhart voters live disproportionately in Divisions where the other voters were more likely to support McIllmurray (presumably Gym voters). Similarly, Santamoor beat Harrity by 3.9pp in Gym Divisions, but only by 0.6pp among Gym voters.

Regression Analysis

A form of ecological effects could permeate even the voter-level results above. Maybe Rhynhart voters came from Divisions more likely to support Santamoor, but within each Division they were just like the others. We can solve this by fitting a regression, using Division fixed effects. This allows us to say, for example, that within a given Division, Cherelle Parker supporters were on average 3.2 percentage points more likely to support Jim Harrity, and less likely to support Amanda McIllmurray and Rue Landau (8.8 and 7.9pp, respectively).

View code
mayor_council_div <- toplines |> 
    filter(office == "Mayor (Dem)") |>
    rename(voters=votes, mayor=candidate) |>
    select(Type, Division, mayor, voters) |>
    # Need this cross join in case there are zero votes for a council candidate.
    cross_join(
        data.frame(council=unique(mayor_council$council))
    ) |>
    left_join(
      pairs |> 
        filter(
          office.y == "Mayor (Dem)", 
          office.x == "Council At-Large (Dem)"
        ) |>
        rename(mayor = candidate.y, council=candidate.x, votes = n) |>
        select(mayor, council, Type, Division, votes),
      by=c("mayor", "council", "Type", "Division")
    ) |>
  mutate(votes=replace_na(votes, 0)) |>
  group_by(Division, mayor, council) |>
  summarise(
    voters=sum(voters),
    votes=sum(votes)
  )

RERUN <- FALSE

library(purrr)

if(RERUN){
  df_cand <- mayor_council_div |>
    ungroup() |>
    filter(council != "Write-In") |>
    # filter(council == !!council_candidate) |>
    cross_join(data.frame(vote_for=c(0,1))) |>
    mutate(votes = ifelse(vote_for==1, votes, voters-votes))
  fits <- list()
  
  for(council in unique(df_cand$council)){
    print(council)
    if(!council %in% names(fits)){
      fits[[council]] <- lm(
        vote_for ~ Division + mayor, 
        w=votes,
        data=df_cand |> filter(council==!!council)
      )
    }
  }  
  coefs_full <- lapply(fits, broom::tidy) |> bind_rows(.id="council")
  saveRDS(coefs_full, file="coefs_full.RDS")
} else {
  coefs_full <- readRDS("coefs_full.RDS")
}

mayor_coef <- coefs_full |> 
  filter(substr(term, 1,5) == 'mayor') |>
  mutate(mayor = gsub("^mayor", "", term))  |>
  select(council, mayor, estimate, std.error) |>
  bind_rows(
    data.frame(
      council = council_order,
      mayor = "Allan Domb", # Reference mayor
      estimate = 0,
      std.error = NA
    )
  ) |>
  left_join(
    toplines |> filter(office == "Mayor (Dem)") |>
      group_by(candidate) |>
      summarise(total_votes = sum(votes)),
    by=c("mayor"="candidate")
  ) |>
  group_by(council) |>
  mutate(weighted_coef = estimate - weighted.mean(estimate, w=total_votes))|>
  mutate(
    council = factor(council, levels=council_order),
    mayor = factor(mayor, levels=mayor_order)
  ) 


ggplot(
  mayor_coef,
  aes(x=council, y=weighted_coef)
) +
  geom_bar(stat='identity') +
  facet_wrap(~mayor) +
  theme_minimal() %+replace% 
  theme(axis.text.x = element_text(angle=90, vjust=0.5, hjust=1, size=5)) +
  labs(
    title="Mayoral supporters' relative vote for council",
    subtitle="Coefficient of mayoral support on council votes, controlling for Division fixed effects",
    x=NULL,
    y="Coefficient of Mayoral Support on Council Vote"
  )

Within a Division, Gym supporters were vastly more likely to support Thomas, Gilmore-Richardson, Landua, McIllmurray, and Almirón, and Rhynhart supporters more likely to support Santamoor, Itzkowitz, and then Gilmore-Richardson, Landau, and Ahmad.

Parker voters were relatively similar to their broader Divisions, but less likely to support Landau, McIllmurray, and Almirón.

Supporters of all mayoral candidates from Domb on were much less likely to support the top four council candidates–Thomas, Gilmore Richardson, Landau, and Ahmad–than supporters of the top three mayoral candidates.

Turnout didn’t decide the election. Preferences did.

Ahead of the election, I was chatting with a reporter. They mentioned that everyone was talking about turnout. Philadelphia had seen weak relative turnout in the last two years, they pointed out, and that would decide this race.

I pushed back. Turnout is not the story here. We know basically what turnout will be. The big open question is preferences.

I wish I had gone on the record. I would have looked like a genius.

Put another way, as I explained on that Friday ahead of the election, if there were one piece of information that would help me predict the outcome, it would not be the relative turnouts of Voting Blocs. It would be Helen Gym’s performance in the Black Wards. If she gets 30%, she wins. If she gets 10%, she loses. Both seemed in play. She ended up at 14.

This brings up an important point about this Municipal Primary:

Preferences decided the election, not turnout

First, let’s be clear what I mean. It is trivially true that if zero of a candidate’s supporters turned out to vote, she would have lost. So in a completely uninteresting sense, turnout mattered.

A more useful statement is that the plausible range of turnouts in Voting Blocs had much less impact on the final result than the plausible range of preferences. The Black Voters Divisions were certainly going to represent between 35-45% of the City’s votes.

But with only limited polling, we had only a rough guess at voters’ preferences. Support for Cherelle Parker in those Divisions could have plausibly been anywhere from 30% to 60% in this crowded race. She ended up at 57.

The plausible range of turnouts in Black Voter Divisions could have swung the topline result +/- 2.5 percentage points. The plausible range of preferences in those Black Voter Divisions could have swung it +/- 6. Cherelle Parker won because she did extremely well among people we always expected to vote, and not by achieving an extreme turnout among her base.

Some Math

Let’s formalize this. A candidate’s overall proportion of the vote is the average of their proportions \(p_i\) in each geography \(i\), weighted by turnout \(t_i\).

\[ p = \frac{1}{\sum t_i} \sum t_i p_i \]

We can normalize turnout using \(\tilde{t}_i = \frac{t_i}{\sum t_i}\), so that \(\tilde{t}_i\) is each geography’s proportion of total turnout. Then

\[ p = \sum \tilde{t}_i p_i \]

Ahead of the election, we don’t know what each \(\tilde{t}_i\) and \(p_i\) is. Instead, we have priors with variances. The Law of Total Variance tells us

\[ Var(p) = E_p[Var( \sum \tilde{t}_i p_i) |\vec{p}] + Var_p(E[ \sum \tilde{t}_i p_i] |\vec{p}) \] where \(\vec{p}\) is the vector of all \(p_i\).

The expectations in the second term simply add. And we will assume that a candidate’s \(p_i\) is uncorrelated across geographies (that is, of course, the definition of my Voting Blocs).

\[\begin{align*} Var(p) &= E_p[Var( \sum \tilde{t}_i p_i) |\vec{p}] + Var_p( \sum E[\tilde{t}_i] p_i |\vec{p}) \\ &= E_p[Var( \sum \tilde{t}_i p_i) |\vec{p}] + \sum E[\tilde{t}_i] Var(p_i) \end{align*}\]

The variances in the first term are a little bit complicated, since the normalization of \(\tilde{t}\) means they will negatively covary across Divisions.

If we simplify to assuming only two geographies (for example, two Voting Blocs), then the Blocs will have a perfect -1 correlation in \(\tilde{t}\).

\[ Var(\tilde{t}) = \left[\begin{array}{rr} \sigma_t^2 & -\sigma_t^2 \\ -\sigma_t^2 & \sigma_t^2 \end{array}\right] \]

In this case, the uncertainty in the topline result reduces to

\[ Var(p) = \sigma_t^2 E[(p_1 – p_2)^2] + (E[\tilde{t}_1] Var(p_1) + E[\tilde{t}_2] Var(p_2)) \]

Which of these terms is bigger? In this past election, the second was much bigger than the first.

The standard deviation of uncertainty in turnout proportions (\(\sigma_t\)) was maybe 0.04, with expected performance differences between Blocs of maybe \(p_1 – p_2 \approx 0.4\), to be generous. That gives a contribution of \(0.04^2 \times 0.4^2 = 0.016^2\).

The standard deviation of uncertainty in candidate preferences (\(\sqrt{Var(p_i)}\)) was maybe 0.10. Since \(\sum{\tilde{t}} = 1\), that contributes the full \(0.10^2\).

The result: uncertainty in candidate’s performance in each Bloc contributed six times as much uncertainty as that of Blocs’ relative turnout!

So yeah, while it’s trivially true that with zero turnout, a candidate can’t win, it was Cherelle Parker’s high percentage among those who we expected to vote, and not a surprisingly high turnout among any group, that won the day.

A caveat

It’s important to note this analysis is for a municipal election with many candidates and sparse, uneven polling. Uncertainty in \(p\) was huge! In a Presidential election, comparatively, we know a lot more about \(p\). Uncertainty in \(t\) is probably more comparable in size to uncertainty in \(p\), though I doubt enough to actually become more important.