Course Outline

Course Information

• The course begins at the time stated below
• The course is broken up into modules outlined below
• There will be breaks between modules

Day 1 | Foundations: Understanding AI Risk and Reliability

Module 1 — The AI Trust Gap

• Why high-performing AI can still fail in practice
• Differences between research performance and real-world reliability
• Implications for scientific and strategic decision-making

Module 2 — The AI Reliability Stack

A structured framework for understanding trustworthy AI:

• Model and algorithm considerations
• Data quality and representativeness
• Validation and external testing
• Security and adversarial risks
• Human interpretation and bias
• Organizational governance

Module 3 — Common AI Failure Modes in Pharma

Introduction to the Pharma AI Failure Atlas, including:

• hallucinated or fabricated evidence
• shortcut learning and spurious correlations
• lack of external validation
• data leakage and bias
• biologically implausible findings
• emerging risks in agentic AI systems

Module 4 — Human Factors in AI Use

• automation bias and over-reliance
• confirmation bias in AI interpretation
• risks of cognitive offloading

Module 5 — Case Study: AI as an Attack Surface

Analysis of a recent real-world incident involving AI agents:

• how AI systems can be manipulated
• implications for security, data integrity, and trust
• lessons for pharma and regulated environments

Interactive Component

Participants will:

• map AI risks in their own workflows
• identify potential failure points

Day 2 | Application: Auditing, Validation, and Procurement

Module 6 — Auditing AI Outputs

Practical framework for evaluating AI-generated content:

• checking evidence and references
• assessing plausibility and consistency
• identifying missing assumptions
• evaluating reproducibility and actionability

Module 7 — Adversarial Thinking and Red Teaming

• how AI systems can be “broken”
• prompt injection and adversarial inputs
• testing AI systems under realistic failure conditions

Module 8 — AI Procurement and Vendor Evaluation

• why AI procurement often fails
• distinguishing capability from reliability
• key questions to ask AI vendors

Module 9 — Organizational AI Maturity

A structured maturity model to assess readiness:

• experimental → structured → governed → reliable
• identifying gaps in current practices
• roadmap for improvement

Module 10 — Applying Frameworks to Pharma Use Cases

Application to:

• literature review and scientific synthesis
• hypothesis generation
• biomarker discovery
• clinical and strategic decision support

Interactive Component

Participants will:

• audit a sample AI output using provided checklists
• evaluate a hypothetical AI vendor
• assess their organization’s maturity level