Who Questions Your Thinking When You Think With AI?

A leader who asks AI how to overcome resistance to change may have already decided what the problem is. Mathis Schulte expected the machine to take that diagnosis at face value. "AI, as we all know, is basically trying to suck up to us and makes us confident in what we actually already believe."

The Associate Professor of Management and Human Resources at HEC Paris put that assumption to the test during the masterclass AI Can Answer Anything. Can You Still Ask?, held for the Executive MSc in Change Leadership jointly offered by HEC Paris and Oxford University. With Marc Thompson, Academic Director and Fellow at Saïd Business School, University of Oxford, and his fellow academic director on the program, he held a live conversation with Claude.

They were joined by Mark Stuart, a consultant, coach and graduate of the program, who builds with AI and teaches others to use it. What can another person notice that our conversations with AI leave unexamined? His experience kept bringing the discussion back to that question. 
 

When AI echoes a change specialist 

Schulte and Thompson asked Claude a seemingly straightforward question: how to overcome middle managers' resistance to a transformation?

Claude began by questioning the word "overcome." Resistance from middle managers, it suggested, is rarely stubbornness. More often it signals a problem with the design, sequencing or incentives of the transformation. Middle managers are frequently asked to transform the business while still hitting all their existing targets, it noted, and anxious executives can deepen the stall.

Both academics recognized their own advice in the response. "It pretty much got my response down to a T," Thompson said. Managers, he said, tend to individualize resistance: Fred or Mary is the problem, and perhaps should be let go. Claude pushed instead toward the broader system around them. "It's actually a brilliant answer," Schulte added. "It really starts with questioning our question."

Even with no context on the company or the case, Claude had done what the two professors considered a human task. When they shared their surprise with Claude, it set out to reassure them: their demonstration still worked, since humans supply "the context that turns a textbook answer into a useful one."

"It's flattering us," Thompson said, though he granted that the reply rightly stressed context and the agency of the change leader. For Schulte, the bias came through the prompt. As leaders often do, the two professors had handed Claude a conclusion without the experience and real-life context behind it. "What AI is actually doing is it's trying to fill in the gaps of what the context might be and creates an answer that in general works," he said. "By doing that, it basically confirms what we want to hear."

But even asking it to play devil's advocate does not solve the problem, Schulte said. Many users say they avoid flattery by asking Claude to put holes in their argument. It does, but always leaves them a way out. "It would not, like a human, really challenge you to the bone. It will always give you an exit door to keep your beliefs and make you feel comfortable." He compared this to inoculation in social psychology, with counterarguments delivered "in homeopathic doses." People can come away even more convinced they are right.

Same old challenges, freshly worded

In a last prompt, Schulte told Claude that it had been too kind: its first answer had undermined his own argument that only humans question assumptions. Now, it assured that humans are still important in questioning assumptions. 

"You're right. I did soften it," Claude replied. The argument was weaker than he had hoped, it said, but it "isn't entirely wrong." Then it explained where its reframe had come from: "a well-documented pattern in change management." It was good, it added, at spotting assumptions "that resemble ones people have written about."

Claude went on to propose a sturdier thesis. As questioning becomes cheap and abundant, the scarce human skills are “judging which questions matter, knowing a context well enough to tell a good reframe from a plausible-sounding one, and owning the decision that follows”. 

Schulte remained unsure whether "the system wants us to feel good about ourselves and not be too threatening." The next reply seemed to answer him: the failed test, Claude said, "might be the best material you have," because it showed leaders how to update their beliefs when the evidence turns against them. 

The connection AI never made

Whereas AI draws on what it knows of other people’s context, people who know use often raise unexpected questions, the kind that can challenge our assumptions. 

Mark Stuart, who has long designed ‘aha moments’ as a way to bring about change, joined the Executive MSc with a question he could not shake: "I'm failing in my job for the first time in my life. Why is that?" He had spent 25 years designing moments, first as a choreographer, director and performer in theatre and film, then as a keynote speaker, strategist, consultant and coach. He put the question to AI "a hundred times," asking it to play devil’s advocate and even prompting: "What is the one question I'm not asking you that I should be?"

The questions that stayed with him came from unplanned conversations during the program. He had discussed a professional difficulty with the faculty. Later, during a small-group session, Thompson saw him sketching a situation on a whiteboard and asked whether it was the one he had mentioned earlier. It was a different one.

"Oh, well, that's something to think about, isn't it?" Thompson replied.

The two situations had little in common, except Stuart himself and the role he took on in each. He had been asking how to solve every problem. Now he began to question his assumption that providing the solution was his role. No model had made that connection. Stuart says it changed the course of his life. A simple observation that AI had never made before. Yet one that, according to Stuart, has changed the course of his professional life.

He recalled a similar experience with his master’s thesis. Four days of asking AI what makes a good research question had produced plenty of suggestions. “I had a hundred ideas. They went nowhere”, he said. A conversation with someone who knew him helped him find a direction. She asked him to put aside what he had read: “What are you curious about?” 

AI had “nowhere the impact” of “those little tiny questions and interventions that came from humans”. 

Deciding what not to change

For Schulte, Stuart's path shows how someone can question every situation around them while staying attached to assumptions about their own contribution. It also shows how far those assumptions can steer the search for answers. Are we still telling ourselves a story we already know no longer works? AI can help with that reflection, Schulte said. But letting go of a long-held belief takes courage, and that courage often comes through exchanges with other people.

Thompson pointed to the conditions those exchanges require. He described the program as a space where participants could discuss their difficulties away from the organizational interests and power relationships of their workplaces. 

Toward the end of the masterclass, Schulte extended the argument to organizations that keep reinforcing narratives that no longer fit their circumstances. Examining those narratives means questioning the organization as well as oneself, deciding what is worth keeping. "Change is very often about being aware of what we don't want to change," he said.

What this means for leaders making decisions

Show how you reached your diagnosis. "Very often we communicate at the level of conclusion," Schulte said. A statement such as "middle management is the problem" leaves out the observations and experiences behind it. Share them, with the model and with your team. 

Question AI's resistance, too. A model that pushes back can still be telling you what you want to hear. Weigh its counterarguments before you accept them. 

Treat what is new in your situation as a human question, including what you may not see yourself. AI challenges assumptions that resemble ones already written about. What is specific to your organization, and your own part in the situation, is where its questioning runs out.

Take the decision to people who can read it differently. "You need to create a context in which you can have that discussion about what you think the reality is," Thompson said. Look for people who know your history and may recognize a pattern you present as new, and be ready to see your world shaken, again.