LOWELL, September 17 — A research team at the University of Massachusetts Lowell has won a grant from the Army Research Office to study a question that runs against the grain of how most artificial intelligence is built: what happens when an AI system is designed to argue with you instead of agree with you? The project is led by Neil Shortland, a criminology professor and forensic psychologist who serves as principal investigator.
The grant comes from the Army Research Office, the Army’s basic research arm, and the work centers on what the team calls human-AI misalignment in decision-making.
Why disagreement might be the point
Most AI systems today are built to adapt to the person using them — matching their values, reinforcing their beliefs, nudging along the decisions they were already leaning toward. That design choice is usually treated as a feature, since it helps people trust the technology. Shortland argues it can also be a handicap, especially when the stakes are high and the call is genuinely hard.
“Nowhere in the study of foreign policy disasters is there someone who complained there was too much diversity of views,” Shortland said. “You can find lots of disasters where a lack of diversity of viewpoints was a huge problem.” Think of it like a second opinion at the doctor’s office.
A physician who simply echoes your own hunch adds nothing; the value comes from someone who sees the problem differently. Shortland’s version of that idea is blunt. “You don’t always want alignment,” he said.
“Task conflict is usually a good thing. Where it’s a bad thing is when we have task conflict, but we also dislike each other.” That distinction — disagreeing about the work versus disliking the person across the table — sits at the center of the project.
Not everyone reacts the same way
The team also wants to understand which people respond well to an AI that pushes back, and which do not. Shortland said the range of reactions is wide. “Not all people will react equally to misalignment,” he said.
“Some people will ignore AI, some people will say, ‘OK, we’ll do it your way because you’re super-smart,’ and some will say, ‘Let’s figure this out together.’” The follow-up question is whether those tendencies can be measured and then built into the system itself. “We are interested in what is it about someone that explains why they behave this way, and can you train that into an algorithm to improve their partnership?” Shortland said.
His own example is a useful one. “If I’m a person who makes fast and crude decisions, I want pushback from someone who’s slow and evaluative, because that’s what I don’t do well,” he said. In that pairing, he added, “Misalignment is a complement.” The research team includes Electrical and Computer Engineering Associate Professor Paul Robinette, Mechanical Engineering Assistant Professor Richard Nuckols, and Computer Science Assistant Professor Samantha Reig.
All three conduct research into human-robot interactions.
Scenarios with no clean answer
To test these ideas, the team plans to build realistic scenarios in which the priorities in play are equal and in conflict. One example they describe: a field commander who must weigh the imperative to rescue wounded American soldiers against the need to capture a high-value opponent who holds important information. “There’s no right and wrong in there,” Shortland said.
“There is individual difference in what’s the first thing you should be doing and what’s the most important thing for you to accomplish.” In the absence of other personnel of equal rank and experience to hash out the differences, Shortland said, an AI partner that balances the decision-maker’s cognitive style and preferences — rather than mimicking them — could raise the odds of achieving both goals. The team is also looking at how AI agents land differently depending on where someone sits.
A field commander relies on the system to make a call. A soldier has to carry out the order even when she disagrees with it. A general conducts the after-action review.
Each role meets the same AI under different pressures. Shortland is careful to draw a line.
AI agents do need to be aligned with larger organizational rules and priorities, such as minimizing civilian deaths. What the team wants to understand is how and why people react when an agent is programmed to challenge them. “What happens when you force people to interact with AI when it’s programmed to do the opposite of what they want?” he asked.
The grant builds on earlier research findings that considering opposing points of view leads to better decisions. What to watch next is whether the team’s scenarios can show which personality traits and thinking styles pair best with a contrarian AI — and whether that pairing can be engineered rather than left to chance.
