Not much familiar wirh metrics for evaluating progression in medical fields, so asking in general sense.

  • AnyOldName3@lemmy.world
    link
    fedilink
    arrow-up
    4
    ·
    3 days ago

    You can’t make an LLM only reference the data it’s summarising. Everything an LLM outputs is a collage of text and patterns from its original training data, and it’s choosing whatever piece of that data seems most likely given the existing text in its context window. If there’s not a huge corpus of training data, it won’t have a model of English and won’t know how to summarise text, and even restricting the training data to medical notes will stop mean it’s potentially going to hallucinate something from someone else’s medical notes that’s commonly associated with things in the current patient’s notes, or it’s going to potentially leave out something from the current patient’s notes that’s very rare or totally absent from its training data.

      • cecinestpasunbot@lemmy.ml
        link
        fedilink
        English
        arrow-up
        2
        ·
        3 days ago

        If you end up integrating LLMs in a way where it could impact patient care that’s actually pretty dangerous considering their training data includes plenty of fictional and pseudo scientific sources. That said it might be okay for medical research applications where accuracy isn’t as critical.