The LLMonFHIR and LLM Intake Forms leverage SpeziLLM, SpeziHealthKit, and SpeziFHIR modules to improve patient-interaction with health data and your health status.
The first example of our LLM & FHIR projects is the LLM on FHIR application. It demonstrates the power of LLMs to explain and provide helpful context around patient data provided in the FHIR format. It demonstrates using the Spezi framework and builds on top of the Stanford Spezi Template Application. The application connects to the OpenAI GPT API to interpret FHIR resources using the GPT suite of large language models. LLMonFHIR supports multiple languages. The LLM is prompt-engineered to converse with users based on their system language.
In addition, the LLM Intake Form project aims to replicate the traditional patient portal interaction and medical intake forms, ensuring up-to-date reconciliation of basic health record elements. The project will lead the path to an autofill-process for patient-facing questionnaires in Spezi Questionnaire. In a clinical setting, the intake form plays a pivotal role in capturing and updating essential patient information. Traditionally, every new or returning patient is handed a clipboard with a paper form to fill out, ensuring that their health records are current and accurate.
The LLM Intake Form project seeks to modernize this process by utilizing SpeziLLM, SpeziHealthKit, and SpeziFHIR modules and embed this functionality in the Spezi Questionnaire module. This digital adaptation not only streamlines the data collection process but also enhances the accuracy and efficiency of updating patient records. Starting with a simplified version, the module aims to gradually incorporate more features, making the transition from paper to digital seamless for both patients and healthcare providers.
| Chief Complaint | Ask the user about the chief complaint and possibly follow up with additional questions to clarify the issues or describe additional aspects that can then be used to generate a comprehensible and compact chief complaint. An LLM can then provide a one or two sentence summary of the “Chief Complaint” and ask the user if this sounds correct and/or if they would like to edit any details. | | --- | --- | | Extract Key Information | Extract key information using the Spezi HealthKit and Spezi FHIR module and further processing in the Spezi Standard:
We aim to validate our LLM-based applications with real patients and real patient data to assess the applicability of these solutions to the day-to-day challenges of patients.
For the LLM intake project, in the long term, the approach used in filling out the intake form should be generalized into an "autofill" feature for a wide variety of patient-facing questionnaires to alleviate the pain points of repeatedly filling out repetitive questions and typing information that is already available within the personal health record or other information. This future approach is similar to autofilling in web browser forms or other contexts on mobile devices.
An intake form is an essential use case of the possibilities of this autofill mechanism in daily medical practice: It is an excellent example of fields that can not be auto-filled (chief complaint) and fields that can be pre-populated (medications, previous history, patient information, and more ...). And like autofill in a browser, the user can change the answers, add context in text fields, or remove elements that are not true. In the long-term, this system could even be learning and, e.g., saving responses and then retrieving them if a question is very close to the original question that the answer is related to.
A more generalized workflow looks as follows:
In addition, it would be great to apply this process to paper forms or forms that currently exist in a PDF form by letting a multi-modal model interpret the image and create an FHIR Questionnaire that could be used as an input for the generalized flow described above: The tool that would generate the JSON (or ask for some input to then generate the JSON in code which might be less prone to errors) and would then serve as an alternative FHIR Questionnaire input source of the whole process and would not be a completely different setup.