I have experience in running servers, but I would like to know if it’s possible to do it, I just need a GPT 3.5 like private LLM running.

  • [moved to hexbear]@lemmy.ml
    link
    fedilink
    English
    arrow-up
    8
    ·
    4 months ago

    They’re Ryzen processors with “AI” accelerators, so an LLM can definitely run on hardware on one of those. Other options are available, like lower powered ARM chipsets (RK3588-based boards) with accelerators that might have half the performance but are far cheaper to run, should be enough for a basic LLM.

    • exu@feditown.com
      link
      fedilink
      English
      arrow-up
      3
      ·
      4 months ago

      I don’t know of any project that already supports that AI processor. You’d still be using the CPU and GPU at the moment.

    • TheBigBrother@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      9
      ·
      edit-2
      4 months ago

      The K8 it’s Ryzen, the K9 Intel, money isn’t a problem and it’s not a spending it’s a investment I need it for business, which of these two models would you recommend for a reasonable good LLM and Stable Diffusion?

      I’m looking for the most cost-effective solution.