The LLMonFHIR app leverage SpeziLLM, SpeziHealthKit, and SpeziFHIR modules to improve patient-interaction with health data and your health status.

Background

The ‣ application demonstrates the power of LLMs to explain and provide helpful context around patient data provided in the FHIR format. It demonstrates using the Spezi framework and builds on top of the Stanford Spezi Template Application. The application connects to the OpenAI GPT API to interpret FHIR resources using the GPT suite of large language models. LLMonFHIR supports multiple languages. The LLM is prompt-engineered to converse with users based on their system language. You can read more about LLMonFHIR:

Current State

You can check out the current state of the ‣ app. We are in the process of conducting a clinical trial to test the application with users.

Future Directions

We aim to further validate our LLM-based applications with real patients and real patient data to assess the applicability of these solutions to the day-to-day challenges of patients.

We are looking for support in extending the application, providing a RAG-based mechanism to add additional context to the LLM responses, facilitate the development of a fog and local-based execution of the LLM in LLMonFHIR, supporting the enrollment in the clinical trial, and data analysis from the data collected in the application. You can learn more about fog computing for LLMs:

Important Skills

1. Working on the application-side of the project: The LLMonFHIR applications is in Swift and SwiftUI. The application is built based on Stanford Spezi; you will be using different Spezi modules and components. Please check out the LLMonFHIR application; familiarity with the components and structure used in the application can be helpful.
2. If you are interested in supporting the clinical trial: Familiarity with submitting, iterating, and communicating with the institutional review board (IRB) to facilitate an ideal study design. Familiarity with collecting patient feedback, running clinical trials, and obtaining structured data based on usability metrics and assessing the effectiveness of the LLM-based research projects.

First Steps

<aside> 📧 Our Choose Your Research Project page provides a great overview of how to engage with your supervisors. The following first steps will help you to get familiar with the project and demonstrate your existing skills.

</aside>

• We encourage you to do 2-3 smaller pull requests in LLMonFHIR, SpeziLLM, SpeziHealthKit, SpeziFHIR, or Spezi Questionnaire to get familiar with the project and demonstrate your ability to work with Swift and the Spezi modules. The Spezi Project board provides a great overview of open issues in Spezi and great ways to get started with good first issues and issues. The LLMonFHIR repository contains a good overview of open issues in the LLMonFHIR project.
• If you are interested in supporting us in conducting a usability study and evaluation of these applications share your existing experience in clinical trials, IRB processes, and data analysis and identify a proposed study design to demonstrate your skillset in supporting the evaluation work for our LLM-based EHR-integrated research projects.

After completing these initial steps, please reach out to Paul Schmiedmayer to discuss how you can become more actively involved in the project and contribute to the research.