True, though there’s a big output difference between the 7B distil (or even 32B/70B) and the full model.
And Microsoft does host R1 already, heh. Again, this headline is a big nothingburger.
Also (random aside here), you should consider switching from ollama. They’re making some FOSS unfriendly moves, and depending on your hardware, better backends could host 14B models at longer context, and similar or better speeds.
One can get Deepseek R1 from many providers (including US hosts, or various other nationalities). Microsoft even has their own anti-CCP finetune, MIT licensed: https://huggingface.co/microsoft/MAI-DS-R1
…Banning the app is reasonable, and a tiny inconvenience for anyone who needs DS.
In other words, this is a big nothingburger because V3/R1 are open models. The story would be different if it was (say) an API-only model like Qwen Max or GPT4o, where ultimately one is beholden to the trainer’s servers.
And you are bringing up a red herring (salutes) that doesn’t matter, because, well, I don’t know. But that’s the whole point of the saying.
Tim Waltz does not have eugenics-adjacent beliefs.
Also, I think you may have misinterpreted what “red herring” means.
Nah its much simpler label than that.
“Troll”
The topic doesn’t matter, it more about fishing for provocation. In this case I couldn’t help myself, but… you know, don’t feed the trolls.
As a serious answer, Musk is into modern variants of eugenics and some scary techno-authoritarian stuff he shares with Peter Theil and other Silicon Valley royalty.
Maybe he’s a techno-neonazi? That’s just semantics.
His alleged Nazi salute is indeed kind of a red herring. But “Nazi” is an reasonable enough descriptor for his beliefs.
Heh, you shouldn’t be paying for LLMs. Gemini 2.5 Pro is free, and so are a bunch of great API models. ChatGPT kinda sucks these days (depending on the content).
I have technical reasons for running local models (instant cached responses, constrained grammar, logprob output, finetuning), and I can help you set that up if you want, but TBH I am not going into a long technical proof of why that’s advantageous unless you really want to try this all yourself.
At risk of going of topic, is this a Millennial meme?
Like, I really hope younger folks have seen this.
IIRC the LibreOffice fork happened years before the public had any awareness of OpenAI (and when OpenAI was, in fact, publishing open models).
The front end.
Some UIs (Like Open Web UI) have built in “agents” or extensions that can fetch and parse search results as part of the context, allowing LLMs to “research.” There are in fact some finetunes specializing in this, though these days you are probably best off with regular Qwen3.
This is sometimes called tool use.
I also (sometimes) use a custom python script (modified from another repo) for research, getting the LLM to search a bunch of stuff and work through it.
But fundamentally the LLM isn’t “searching” anything, you are just programmatically feeding it text (and maybe fetching its own requests for search terms).
The backend for all this is a TabbyAPI server, with 2-4 parallel slots for fast processing.
No amount of AI can replace professionalism.
This!
I don’t want to do that. Instead I let it suggest better phrasing, words, basically a better editor.
This!
Locally, I actually keep a long context llm that can fit all or most of the story, and sometimes use it as an “autocomplete.” For instance, when my brain freezes and I can’t finish a sentence, I see what it suggests. If I am thinking of the next word, I let it generate one token and look at the logprobs for all the words it considered, kind of like a thesaurus sorted by contextual relevance.
This is only doable locally, as prompts are cached so you get instant/free responses ingesting (say) a 80K word block of text.
That work well out of the box? Honestly, I’m not sure.
Back in the day, I’d turn to vapoursynth or (Or avisynth+) filters and a lot of hand editing, basically go through the trouble sections one-by-one and see which combination of VHS-specific correction and regeneration looks best.
These days, we have far more powerful tools. I’d probably start by training a LoRA for Wan 2B or something, then use it to straight up regenerate damaged test sections with video-2-video. Then I’d write a script to detect them, and mix in some “traditional” vapoursynth filters.
…But this is all very manual, like python dev level with some media/ml knowledge, unfortunately. I am much less familiar with, like, a GUI that could accomplish this. I’m sure paid services out there offer this, though.
Nvidia.
Back then I had a 980 TI RN I am lucky enough to have snagged a 3090 before they shot up.
I would buy a 7900, or a 395 APU, if they were even reasonably affordable for the VRAM, but AMD is not pricing their stuff well…
But FYI you can fit Qwen 32B on a 16GB card with the right backend/settings.
Well, if I am going to push this into the project I envision, privacy is going to be key, so everything will be done locally.
Reasonable! And yes languagetool has some online AI thing, probably a junky wrapper around an LLM API TBH.
One thing I’d be wary of is… well, showing you’re using AI?
As a random example, I helped clean up a TV show a long time ago, with mixed results. More recently, I brought the idea of making another attempt to the fandom, and got banned for even considering “AI,” with most of them clearly oblivious to how the original restoration was made… I have a little personal writing project going too, and wouldn’t dare bring it up to the fandom either.
I don’t know what’s involved in your project, but be aware that you may get some very hostile interaction if it’s apparent you use diffusion models and LLMs as helpers.
You don’t strictly need a huge GPU. These days, there are a lot of places for free generations (like the AI Horde), and a lot of quantization/optimization that gets things running on small VRAM pools if you know where to look. Renting GPUs on vast.ai is pretty cheap.
Also, I’d recommend languagetool as a locally runnable (and AI free/CPU only) grammar checker. It’s pretty good!
As for online services, honestly Gemini Pro via the AI Studio web app is way better and way more generous than ChatGPT, or pretty much anything else. It can ingest an entire story for context, and stay coherent. I don’t like using Google, but if I’m not paying them a dime…
Before it was hot, I used ESRGAN and some other stuff for restoring old TV. There was a niche community that finetuned models just to, say, restore classic SpongeBob or DBZ or whatever they were into.
These days, I am less into media, but keep Qwen3 32B loaded on my desktop… pretty much all the time? For brainstorming, basic questions, making scripts, an agent to search the internet for me, a ‘dumb’ writing editor, whatever. It’s a part of my “degoogling” effort, and I find myself using it way more often since it’s A: totally free/unlimited, B: private and offline on an open source stack, and C: doesn’t support Big Tech at all. It’s kinda amazing how “logical” a 14GB file can be these days, and I can bounce really personal/sensitive ideas off it that I would hardly trust anyone with.
…I’ve pondered getting back into video restoration, with all the shiny locally runnable tools we have now.
F me…
Does no one see the propaganda potential for this? Yeah, let’s give one guy more intimate, influential control over the psyche of hundreds of millions.
Yeah I mean it really depends on the app.
If it just occasionally peaks, maybe modest zram and low swappiness is all you need.
But sometimes i crack it up (and basically disable zram) because I’m paging like 4x my RAM pool size onto ssds with big files.
Try it here instead, set the temperature to like 0.1 or 0.2, and be sure to set 2.5 Pro:
It is indeed still awful for many things. It’s a text prediction tool, not a magic box, even though everyone advertises it kinda like the later.
Completely depends on your laptop hardware, but generally:
I use text-gen-web-ui at the moment only because TabbyAPI is a little broken with exllamav3 (which is utterly awesome for Qwen3), otherwise I’d almost always stick to TabbyAPI.
Tell me (vaguely) what your system has, and I can be more specific.