Supporting Diagnostic Confidence Using Artificial Intelligence

Overview

Graduate counseling students are required to learn complex diagnostic frameworks (DSM-5-TR), yet many report low confidence and limited opportunities for applied practice. To address this gap, AI-supported simulations were introduced as a supplemental training tool to provide interactive, low-stakes diagnostic practice.

‍

Problem

Trainees struggle with:

• Low diagnostic self-efficacy
• Limited real-time practice opportunities
• Difficulty applying abstract diagnostic criteria to real cases

This creates hesitation, reduced confidence, and challenges in clinical decision-making.

‍

Research Goals

1. Evaluate whether AI-supported practice improves diagnostic confidence
2. Understand how trainees experience AI as a learning tool
3. Identify usability, trust, and accuracy concerns

‍

My Role

I contributed across the full research lifecycle:

‍Sample Size:
16 student participants completed pre-test surveys and 11 completed post-test surveys. Participants were graduate-level counseling students enrolled in clinical training coursework.

Recruitment Source:
Participants were recruited from multiple course sections at Palo Alto University, ensuring variation in experience levels and training exposure.

Demographics:

• Early-career clinicians / trainees
• Graduate students in counseling and psychology programs
• Individuals actively learning diagnostic frameworks (DSM-5-TR)

Selection Criteria:
Participants were selected based on:

• Enrollment in relevant clinical coursework
• Exposure to diagnostic training
• Willingness to engage with AI-supported practice tools

Context of Participation:
Participants completed the study within an academic setting, engaging with AI-based diagnostic exercises and completing pre- and post-assessment measures.

‍

Methodology

Approach: Convergent mixed-methods consisting of 42 diagnostic self-efficacy items, 22 items related to perceived competence in diagnosing DSM-5 areas, and six open-ended qualitative questions through a Qualtrics survey link virtually shared

‍

Quantitative:

• Pre/post diagnostic self-efficacy surveys
• Welch’s t-tests (unequal samples)

Qualitative:

• Open-ended survey responses
• Thematic analysis (Braun & Clarke)

Key Findings

Increased Diagnostic Confidence

Participants showed significant improvements in both diagnostic self-efficacy and perceived competence, with large effect sizes.

‍

AI as a Cognitive Guide

Users described AI as a tool that supported reasoning by:

• identifying blind spots
• suggesting alternative diagnoses
• encouraging reflection

‍

Repetition Drives Skill Development

AI enabled repeated, low-pressure practice, reinforcing learning more effectively than static case studies.

‍

UX Insights

Insight 1 : Users need iterative, low-stakes practice

AI created a safe environment for trial-and-error learning, increasing engagement and confidence.

‍

Insight 2 :Trust must be actively designed

Users questioned AI accuracy and required external validation throughDSM cross-checking

‍

Insight 3 : Clinical nuance is missing

AI struggled with:

• cultural context
• relational depth
• nuanced interpretation

‍

Insight 4 : Users benefit from “thinking support,” not answers

The value of AI was not in giving correct diagnoses, but in guiding reasoning processes.

‍

Impact

• Significant increase in diagnostic self-efficacy and improved confidence in applying DSM criteria, with large effect sizes indicating a meaningful impact on learning outcomes.

• Enhanced engagement through interactive practice

‍

Reflection

This project highlighted the growing role of AI as a supportive tool for students and clinicians, particularly in areas that require self-confidence and repeated practice. Rather than replacing human judgment, AI can serve as a resource that helps users think through problems, explore different possibilities, and build their skills over time. It creates opportunities for low-pressure, accessible practice that may not always be available in traditional learning environments. At the same time, this experience made it clear that AI should be approached as a complement to human expertise, not a substitute. Moving forward, its value lies in supporting learning, encouraging reflection, and helping users feel more prepared in complex, real-world situations.

‍

‍

‍

‍