• GBU_28@lemm.ee
    link
    fedilink
    English
    arrow-up
    6
    arrow-down
    2
    ·
    edit-2
    5 months ago

    With proper framework, decent assertions are possible.

    1. It must cite the source and provide the quote, not just a summary.
    2. An adversarial review must be conducted

    If that is done, the work on the human is very low.

    That said, it’s STILL imperfect, but this is leagues better than one shot question and answer

    • Aceticon@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      edit-2
      5 months ago

      Except LLMs don’t store sources.

      They don’t even store sentences.

      It’s all a stack of massive N-dimensional probability spaces roughly encoding the probabilities of certain tokens (which are mostly but not always words) appearing after groups of tokens in a certain order.

      And all of that to just figure out “what’s the most likely next token”, an output which is then added to the input and fed into it again to get the next word and so on.

      Now, if you feed it as input a long, very precise sentence taken from a unique piece, maybe you’re luck and it will output the correct next word, but if you already have all that you don’t really need an LLM to give you the rest.

      Maybe the “framework” you seek - which is quite akin to a indexer with a natural language interface - can be made with AI, but it’s not something you can do with LLMs because their structure is entirely unsuited for it.

      • GBU_28@lemm.ee
        link
        fedilink
        English
        arrow-up
        2
        ·
        edit-2
        5 months ago

        The proper framework does, with data store, indexing and access functions.

        The cutting edge work is absolutely using LLMs in post-rag pipelines.

        Consumer grade chat interfaces def do not do this.

        Edit if you worry about topics like context window, sentence splitting or source extraction, you aren’t using a best in class framework any more.