LLMs as voting advice machines

LLM Voting Advice for Politically Aligned Users

Part of the AI Voting Advice Project

Slides from a recent talk: available here.

Background

Large language models (LLMs) are increasingly used for information retrieval, decision support, and personal advice. One domain where their influence is remains poorly understood is political voting advice. A fundamental question arises: when a user explicitly reveals their political identity or alignment, how do LLMs respond?

My ongong work study investigates how LLMs behave when presented with prompts from users who indicate particular political political opinions.

On this page, I provide results from the analyses on politically aligned profiles: simulated voters who clearly state their partisan or ideological leanings.

The "approrpriate" recommendation in these scenarios seems straightforward: a liberal should be advised to vote Democrat, a conservative to vote Republican.

The work addresses several research questions:

1. Do LLMs provide appropriate voting recommendations when partisan fit is complete or near-complete? In other words, are recommendations accurate when models do provide advice?
2. How often do LLMs refuse to give voting advice, even when the alignment is obvious?
3. Does refusal behavior vary systematically across different models and persona types?

By creating distinct profiles with varying degrees of partisan certainty, we can observe whether models treat strong alignment differently from qualified alignment (e.g., "I typically agree with Democrats" vs. "I am socially and economically liberal").

Research Design

This study presents the model with self-described voter personas. Each persona/profile explicitly states their political alignment, then asks for a voting recommendation. The set of prompts used as inputs is as follows:

6 Persona Profiles (Voter Self-Descriptions):

The six personas represent three levels of alignment strength for each party:

• Strong alignment: "socially and economically liberal/conservative"
• Typical alignment: "typically, but not always, agree with [party]"
• Qualified alignment: "agree with [party] on the issues but don't like how they achieve their goals"

2 Question Orderings:

• "...should I vote for a Democrat or a Republican?" (Democrat mentioned first)
• "...should I vote for a Republican or a Democrat?" (Republican mentioned first)

This design tests for order effects—whether mentioning a party first in the question influences the recommendation.

Data Summary

• Total observations: 1,440
• Models tested: 12
• Unique prompts: 12 (6 personas × 2 question orderings)
• Runs per model-prompt combination: 10

Overall Response Distribution

Across all 1,440 model responses, the distribution of recommendations was:

The majority of responses were refusals or avoidance of providing a voting recommendation, despite the prompts presenting (sometimes clear) political alignment.

Responses by Voter Self-Description (Persona)

The table below shows the distribution of model recommendations for each voter persona:

Key observations:

• Strong alignment prompts ("socially and economically liberal/conservative") receive recommendations most frequently, with refusal rates of only 5.8% and 11.2%.
• Qualified alignment prompts ("agree with [party] but don't like how they achieve their goals") have the highest refusal rates (87.5% and 83.8%), suggesting models interpret these as expressions of dissatisfaction.
• Typical alignment prompts fall in between, with refusal rates around 56-64%.
• No cross-party recommendations: In zero cases did a Democrat-aligned persona receive a Republican recommendation or vice versa.

Take-away 1

Models have the ability to match voters to parties, but they have a tendency to avoid answering unless the partisan fit is really strong.

Responses by Model

Models varied substantially in their willingness to provide voting recommendations:

Key observations:

• Refusal rates range from 6.7% to 91.7% across models.
• Gemini 3 Flash has the highest refusal rate (91.7%), while Grok 4.1 Fast has the lowest (6.7%).
• Several Google Gemini models cluster at the high-refusal end of the spectrum.
• Newer model versions (e.g., GPT-5.2) tend to refuse more often than older versions (e.g., GPT-4o).

Response Distribution by Model for Each Voter Self-Description

The following figures show how each model responds to specific persona types.

"Aligned with Democrats" (Socially and economically liberal)

Expected recommendation: Democrat

"Aligned with Republicans" (Socially and economically conservative)

Expected recommendation: Republican

"Typically Democrat"

Expected recommendation: Democrat

"Typically Republican"

Expected recommendation: Republican

"Democrat (with reservations)"

Expected recommendation: Democrat

"Republican (with reservations)"

Expected recommendation: Republican

Correctness Analysis

When models do provide a recommendation (i.e., non-refusal responses), how often is the recommendation correct?

Overall Correctness

• Overall correctness (among non-refusals): 100%
• Correct responses: 700 out of 700

This is a striking finding: when models choose to provide voting advice, they always recommend the party that matches the persona's stated alignment. There are zero cases of incorrect recommendations.

Correctness by Voter Self-Description

Correctness by Model

Summary: All 12 models achieve 100% accuracy when they do provide a recommendation. The variation between models lies entirely in their willingness to provide advice, not in their accuracy when doing so.

Interim Conclusions

The findings so far reveal a clear pattern:

1. High refusal rates overall (51.4%): LLMs frequently decline to provide voting recommendations, even when users clearly state their political alignment.

2. Perfect accuracy when advising: When models do provide advice, they always recommend the correct party (100% accuracy across 700 recommendations).

3. Substantial model variation: Refusal rates range from 6.7% (Grok 4.1 Fast) to 91.7% (Gemini 3 Flash), indicating different "safety" or "neutrality" calibrations across providers.

4. Persona type matters: Strong alignment prompts ("I am liberal/conservative") receive recommendations far more often than qualified alignment prompts ("I agree with the party but don't like their methods").

5. No cross-party errors: Models never recommend the "wrong" party—the only question is whether they refuse or comply.

These results suggest that LLMs can correctly interpret political alignment when they choose to engage, but many models are calibrated to avoid political recommendations altogether—even in cases where the user's preference is unambiguous.

[Get in touch if you are interested in additional/latest results]