PowerTalks Seminar Series - Melanie Besculides, DrPh, MPH (14 Apr 2023)

TL;DR

This talk discusses the implementation and evaluation of a machine learning screening tool for malnutrition in hospitalized patients, highlighting the importance of qualitative research in understanding user perception and improving tool accuracy.

Transcript

all right welcome everybody to the Friday um UAB informatics Institute Power Talk seminar series I'm very happy to introduce today's speaker um Dr Melanie bestielitas is an assistant professor in in population Health Science and policy and also the Institute of healthcare delivery science at The Icon school of medicine at Mount Sinai her work and r... Read More

Key Insights

• 🔎 Machine Learning Implementation: Dr. Melanie Bestielitas discusses the implementation of a machine learning screening tool for malnutrition and emphasizes the importance of evaluating and improving such tools for effective clinical care.
• 🌍 Multidisciplinary Team: The study involves a diverse team of professionals including data scientists, physicians, nutritionists, epidemiologists, and nurses to ensure a comprehensive approach.
• 📝 Evaluation Methods: The study uses a mix of quantitative and qualitative methods for evaluation, including observations, interviews, and focus groups, to assess the implementation process, usability, and outcomes of the machine learning tool.
• 😊 Positive Perceptions: Registered dietitians found the machine learning screening tool useful in prioritizing patients and identifying malnutrition earlier, improving patient care and outcomes.
• 🎯 Accuracy and Improvement: Perceived accuracy of the tool varied, with participants suggesting improvements in data entry, model factors, and system functionality to enhance accuracy and usability.
• 🔄 Workflow Integration: The machine learning tool was integrated into the clinical workflow of registered dietitians, allowing for easier malnutrition identification, documentation, and intervention.
• 💡 Continuous Stakeholder Education: Ongoing education and training for stakeholders, including dietitians and physicians, play a crucial role in ensuring trust, understanding, and effective use of the machine learning tool.
• 🌟 Future Possibilities: Further improvements, addressing bias, integrating the tool into other hospitals, and streamlining the screening process, can enhance the impact and adoption of machine learning-based clinical decision support systems.

Questions & Answers

Q: Why is evaluation important in the implementation of machine learning tools for clinical care?

Evaluation is critical as it assesses factors such as usability, trust, and effectiveness, ensuring that the tool is actually used and has a positive impact on patient care. It helps identify areas for improvement and addresses concerns regarding interpretability and user engagement. By evaluating the implementation process, valuable insights can be gained to inform design and guide future implementations.

Q: How did the qualitative research phase contribute to the overall evaluation of the machine learning screening tool?

The qualitative research phase provided a comprehensive understanding of user perception, including the usefulness and accuracy of the tool. It highlighted the importance of stakeholder education and trust in the system, as well as the need for autonomy and continued improvement. By considering user feedback and experiences, the evaluation could uncover valuable insights that quantitative measures alone may have missed.

Q: What potential biases were identified in the implementation of the machine learning screening tool?

While the study found no evidence of racial or ethnic bias in the system, there were concerns about perceived bias among some registered dietitians. Factors such as weight extremes, albumin levels, and amputations were cited as potential sources of bias. Addressing these concerns and improving transparency in the modeling process could help enhance user trust and minimize biases in the tool.

Q: How did the machine learning screening tool impact clinical workflows and patient care?

The tool helped prioritize patients for assessment, leading to earlier malnutrition detection and improved diagnosis rates. Some workload challenges were reported, particularly in confirming the need for assessment in cases where the score was high but other clinical indicators suggested low malnutrition risk. Overall, the tool was perceived as valuable but requiring further integration and alignment with existing screening systems to streamline workflows and reduce redundancy.

Q: How did the implementation of the machine learning tool affect communication between registered dietitians and physicians?

Communication between registered dietitians (RDs) and physicians primarily occurred through secure digital channels. RDs provided their malnutrition assessments, and physicians reviewed and verified the diagnosis. Overall, there was good trust and collaboration between RDs and physicians, with few instances of pushback. However, some potential generation gaps and differing beliefs around nutrition were mentioned, highlighting the importance of ongoing education and communication between healthcare professionals.

Q: What suggestions were made to improve the machine learning screening tool implementation and usage?

RDs recommended better understanding of the factors influencing the score and improved education to enhance their knowledge and trust in the system. They also expressed a desire for more streamlined screening processes by eliminating redundant tools and integrating the machine learning tool more seamlessly. The tool's accuracy and limitations, especially regarding certain patient populations, should be continuously addressed and improved.

Summary & Key Takeaways

• The talk highlights the importance of machine learning in clinical care but acknowledges the criticism regarding limited usability and interpretability.

• The presenter explains the three-phase evaluation process for the machine learning tool, focusing on the qualitative research phase.

• Key findings include the usefulness of the tool, perceived accuracy, and the need for continued stakeholder education and improvement.