What Makes AI Projects Succeed? Lessons from Four Healthcare Implementations

By analysing four real AI implementations in Swedish healthcare and comparing them against reflections from leading AI researchers, this study identifies the patterns common to successful AI projects — and translates them into a practical checklist for assessing whether a task is suitable for AI automation.

How AI Projects Differ from Traditional Projects

AI projects are structurally different from conventional technology projects in ways that most organisations underestimate. The workflow is non-linear: data collection and exploration cycles are hard to predict and plan for. Success depends heavily on data quality and data characteristics — not just on technical capability. And the definition of "done" is genuinely different: an AI project rarely reaches a clean completion point in the way a traditional software build does.

There is also greater uncertainty about whether the project will deliver what was promised. In a traditional project, a skilled team applying the right methods can reliably produce a defined outcome. In an AI project, even a strong team with excellent data may discover mid-way that the task is not automatable in the way originally envisioned. This demands a different relationship with risk, scope, and stakeholder expectations.

What AI Research Experts Say

The study drew on reflections from two leading researchers at the frontier of applied AI in healthcare.

Thomas Schön (Uppsala University AI Research Centre) identifies large-scale data and clearly defined questions as the strongest combination for AI application. He emphasises that AI research is expected to produce clear, measurable results — which means the most suitable questions are specific, bounded, and testable, not broad or explorative.

Max Gordon (Clinical Artificial Intelligence Laboratory, Karolinska) notes that the strongest AI results in healthcare have come from image interpretation. He identifies meta-analyses — combining and reanalysing many smaller studies — as a major untapped opportunity for AI, since they have historically been delayed by the time required to conduct them manually.

Four Projects Analysed

The Common Success Patterns

Comparing the four projects against expert predictions, the study found strong alignment. The patterns that separated the successful projects from the disappointing ones were consistent:

• Data quality is foundational. The mammography project was validated on 55,581 images taken with the centre's own equipment under clinical conditions. This is not a detail — it is the reason the results were trustworthy.
• Task definition must be clear and limited. AI performs well on well-bounded tasks. The speech recognition project struggled in part because the task turned out to be broader than anticipated — transcription is routine, but verifying clinical accuracy is not.
• Stakeholder alignment on goals is critical. In the speech recognition project, doctors and administrators had fundamentally different views on what the project was for. This kind of misalignment predicts failure more reliably than technical problems.
• Tasks involving paired or group work are particularly suitable. When more than one person is involved in a task, AI can be introduced in a staged way — allowing the human to retain final responsibility while AI handles the routine component. The mammography project is a strong example of this.
• Implementation planning is not optional. The speech recognition project was introduced too quickly, with an implementation phase focused primarily on technical testing rather than on how the tool would change clinical workflows and responsibilities.
• Risk analysis — including ethical and legal dimensions — must precede deployment. The mammography project conducted this analysis before introduction. The speech recognition project did not do so adequately.

A Practical Checklist for AI Task Assessment

The study concludes with a checklist across three dimensions — task, organisation, and technology — that can be used to assess whether a given task is a suitable candidate for AI automation. Key questions include:

• Is the AI intended to assist in a task where more than one person is involved?
• Must results always be 100% correct, or is a high-accuracy approximation acceptable?
• Are there sufficient resources for a staged implementation that keeps humans responsible for final decisions?
• What are the human, economic, legal, and reputational risks of AI errors?
• Is the task routine, or does it require specialist expertise?
• Is the goal of the project clearly defined and agreed upon by all participants?
• Has it been assessed how organisational structure and work roles will change?

The full checklist, covering the task, organisation, and technology dimensions in detail, is included in the research paper.