Interests: programming, video games, anime, music composition

I used to be on kbin as e0qdk@kbin.social before it broke down.

  • 0 Posts
  • 106 Comments
Joined 3 years ago
cake
Cake day: November 27th, 2023

help-circle
  • If you just pulled the default version of qwen3.5 from ollama’s repo you downloaded a mediocre one that only uses ~6GB.

    Check ollama show qwen3.5 and see if you get something like this in the result:

      Model
        architecture        qwen35    
        parameters          9.7B      
        context length      262144    
        embedding length    4096      
        quantization        Q4_K_M 
    

    This is the default version I got when I first tried using ollama without any experience. It worked, but it’s a heavily quantized, lower parameter version of the model – i.e. it’s pretty dumb – compared to what you can actually run on your hardware.


  • e0qdk@reddthat.comtoSelfhosted@lemmy.worldDo you host your own AI?
    link
    fedilink
    English
    arrow-up
    3
    arrow-down
    1
    ·
    3 months ago

    I started running LLMs a couple months ago on my own hardware. I have a Framework Desktop that I ordered last year and also recently picked up a refurbished 24GB AMD RX 7900 XTX which I’m doing some performance testing against. The dGPU is much better for dense models, and slightly faster for MoE if I’m willing to run them at a lower quant – but uses more power and has annoying coil whine. The Framework Desktop uses ~100W under load, is quieter, and for the MoE models already runs them fast enough for most of my needs – so most of my LLM use happens on that system still.

    For software: I’m using ollama on the Framework currently, but I want to replace it with just using llama.cpp directly eventually. I’ve been using llama-cli for testing the dGPU. I wrote my own chat client to interact with ollama as well as a few other programs for specific tasks.

    I’ve been using the LLMs for a mix of research (both personal and professional), entertainment, practical coding tasks (mostly debugging and brainstorming, plus a bit of UI prototyping, automatic generation of sequence diagrams for documentation, and light scripting), as well as automation of tedious tasks.

    As an example of the latter, people often send me requests to prepare data sets by email but don’t specify the sources they want precisely so I have to go match the name against the real name in our archives; LLMs are great for mapping the imperfect name – with typos, missing prefixes, incorrect addition of spaces, addition/removal of hyphens, etc. – to the exact name I actually need to pull the data off disk when given a lookup table to compare against.

    As far as models go, I’m mostly using various Qwen 3.6 and Gemma4 variants. I have multiple versions of each for different purposes. llmfan46’s uncensored Qwen 3.6 35B-A3B @ Q6_K (from Hugging Face) is my default model currently.



  • e0qdk@reddthat.comtoFediverse@lemmy.worldGhost of Lemm.ee?
    link
    fedilink
    English
    arrow-up
    56
    ·
    7 months ago

    Looking back through your history, that’s a post by a user local to your instance. You can see it because you’re on the same instance.

    If I understand how federation works correctly, posts don’t go directly to the instance a community is on when they are made. They are created locally on your own instance, and then federate out if/when they can. Since you’re both on the same instance, you can see the post and interact with it, but the post and your comments are (presumably) stuck in a queue trying to federate to the now defunct instance. Since lemm.ee is gone, it can’t federate out, so other people don’t see the post/comment on their instance.

    I think that’s what’s going on.



  • Should be trivial to set up something like that if you’ve got parts you want to work with. Any desktop with an automatic background switcher should be able to cycle through images in a directory you specify on a timer. Set up your favorite remote access software (SSH, Samba, NFS …) and you’re done. If you want more control over the behavior, you could script up something custom with a little more effort – but it’s still not particularly hard to implement something like that.

    Watch out for burn in on the screen if you’re leaving it on all the time.



  • By “legacy” they probably mean that they work with the older process technologies, not that the fab itself is old:

    The acquisition includes an existing 300 mm fab cleanroom of 300,000 square feet and will further position Micron to address growing global demand for memory solutions

    In its May 2024 ’Hooray, we’re open!’ announcement, PSMC said it invested more than NT$300 billion (US$9.5 billion) on the facility, and that it had capacity to produce 50,000 12-inch wafers per month under 55, 40 and 28 nanometer technology nodes.

    Those kinds of chips are still very useful for things like cars and washing machines and such where you don’t need bleeding edge chip tech.


  • I haven’t tried Nostr, so have no opinions on what the experience of actually using it is like, but cryptographic identity seems like it’d be a better way (technically speaking) of doing things than AP; tying everything to domain names has worked rather poorly – as we’ve seen repeatedly every time an instance goes offline…

    I ended up on AP after jumping ship from reddit. I was on kbin first (since it was readable w/o JS and I liked the UI), and then later using the mlmym interface for lemmy as kbin because more unstable and eventually went offline.









  • How is device-based age verification different?

    You put your device in child safety mode, and it tells sites “I’m a kid, treat me like a kid” – otherwise the site can assume you’re an adult with full rights. Done. No intrusive ID requirements. No face scanning. No third-party payment shakedowns. Parents, in theory, can still stop their five year olds from accidentally accessing PornHub or other content that would disturb them by just clicking a button when they set up an account on the device.

    It’s, frankly, the sane way to do this if we’re going to have age restrictions.




  • Assume an unlimited budget for now, I just want to know what’s out there.

    I mean, if you’re willing to pay the price of a car per SSD they go up to at least 122TB density per drive… (e.g. Solidigm SBFPF2BV0P12001 D5-P5336 – $16K~$20K depending on supplier from a quick search)

    I don’t actually recommend that for personal use, but since you were curious about what’s out there, there’s some absolutely crazy shit in enterprise server gear if you have deep enough pockets.