up to 150kW at full load.

That’s the last generation. They’re moving from Blackwell to Rubin chips now, and the 72-GPU Rubin servers use up to 230 kW.

The typical residential connection in the U.S. has a 24 to 48 kW electrical connection. A block of houses might not have enough power infrastructure to power just one of these server racks.

It’s that they are trying to use statistics to encode entire thought processes into hidden variables from conversation snippets. They want to use statistics to go from many individual interactions to a large model, and then use that model to predict individual interactions again.

Has it been shown that the human brain doesn’t model the world in a similar way, though? A huge portion of human knowledge is both stored and transmitted in the form of language. Lots of human knowledge also follows the garbage in, garbage out theory, where you can have entire areas of knowledge that aren’t actually true but might be internally consistent, at least within certain scopes: conspiracy theories, belief in the supernatural, entire academic disciplines built on a religion or theology that not everyone believes, etc. Or even world building in fiction, the words on a page can be enough to convey ideas such that it “tricks” human brains into filling in the gaps so that they internally see a rich, fleshed out world that is entirely fictional and where specific details might not find strong direct support in the underlying text.

it has no concept of correctness

But statistical weight on what is more or less likely to be correct still makes a difference to objective quality of the outputs. If the model weights are trained on the reality that high quality university texts describe something and reflect some sort of underlying model of what is described using language, then can’t the model itself learn as much as a human could from those words on a page?

All models are wrong, but some can be useful. And different models have different quality in different domains. So although I don’t believe LLMs will overtake the hump of getting ahead of human knowledge, I also don’t believe that any given LLM can be evaluated on quality, and that Facebook’s LLMs are significantly behind other LLMs we see.

And that maybe a huge part of it is its internal process of preparing the model to evaluate the quality of its inputs, such that the output it produces can also score high on quality.

Publication date April 2, based on reporting from Bloomberg on April 1. Anyone have an update 3 months later? I want to see what happened to the remaining half.

It’s like responding to your employee losing an arm, ripped off by your tiger, and saying “I’m never going to financially recover from this.”

Basically they’d need about as much in radiator fin surface area as they would have in solar panel area. The ISS has 8 solar array wings, 35m x 12m, that can produce about 30 kW each, or 240 kW total, in sunlight (which is only half the time). The ISS has a complex cooling system, but relies on 4 radiators about 3.1 m x 13.6 m to reject up to 14 kW of heat each (56 kW total) for cooling the solar arrays themselves. The main cooling system uses 6 radiators, each 23.3 m x 3.4 m, to reject 70 kW of heat (from this report it sounds like each radiator may be capable of rejecting more than 1/6 of the heat but that the system as a whole needs to be kept under 70 kW of heat rejection).

So that seems like about 650 square meters of radiators can provide about 120 kW of heat rejection.

Today, a 72-GPU Blackwell server is 130 kW in a single server rack. The next generation rolling out now has 72 Rubin GPUs in a 230 kW server, in a single rack. And that’s not even a “data center.” That’s just a single (albeit very powerful) server. How many can you string together, with networking equipment beaming data connections back down to the ground, before the ratio of solar panels and radiators to the actual ship size becomes unworkable?

That said, it’s technically possible, especially if you can radiate the heat at higher temperatures than the ISS does, as the Stefan-Boltzmann law shows that the hotter the radiator, the more heat it can reject. Just completely infeasible from an engineering and economical standpoint, for any data center that hopes to be relevant in an age of 100+ MW data centers.

No, the article is saying that it is why these robots were popular. Because unlike a human delivery person, there was no tip expected for the robots.

Trade secrets necessarily have to be analyzed under the protections of contract law.

Something can only be a trade secret if the purported owner of that proprietary information protects the confidentiality of that information, including through contractual restrictions. That’s why I’m talking about contracts when asking whether trade secret protections apply.

I just pulled up the ChatGPT terms of use

Who’s talking about ChatGPT or OpenAI?

I just pulled up the Anthropic commercial API terms, since that’s the situation covered by the original article (big corporation using Anthropic’s paid API):

Use Restrictions. Customer may not and must not attempt to (a) access the Services to build a competing product or service, including to train competing AI models except as expressly approved by Anthropic; (b) reverse engineer or duplicate the Services; or © support any third party’s attempt at any of the conduct restricted in this sentence.

Ok, so it’s a contract that purports to prohibit pretty much this kind of model weight extraction, and I’m saying that Anthropic probably considers the model weights to be trade secrets.

Are you under the impression that trade secret protection only happens when the contract says the words “trade secret”?

Or, analogously, consider customer lists. Having a contract that says “don’t copy my customer lists even if I sometimes disclose a single customer at a time when we partner together on projects” is probably enough to adequately maintain trade secret protection over those customer lists, even if individual customers are sometimes disclosed under a contract.

I’m just stating what I believe the law is, not what it should be, or even claiming that what the law is today is good. I’m just saying everyone should be aware that the law is quite protective of big corporations and their proprietary secrets. I still think this qualifies as a trade secret that they’ve protected with their own contracts.

Ok, do these countries also make a contract not to distill LLMs void, as well?

Can you name a country where signing up for a paid account to an online service, and using the service and paying the invoice that comes in, doesn’t form a legally binding contract between the customer and the vendor?

Sharing trade secrets under the terms of a contract that dictates how one can use the information still retains trade secret protections.

Without a contract: intentional disclosure to the person who receives it generally destroys the trade secret status of the information, because the “owner” of the information didn’t do a good job trying to protect it.

With a contract: intentional disclosure to a person under the terms of the contract makes the contract’s own protections of the information relevant, and misuse of the information by the recipient can get them sued under the contract. Plus, the information itself probably retains trade secret protection so that even if that person gives the information to a third party who can’t be sued under a contract they never agreed to, there are still rights to protect that trade secret as property.

I’d be shocked if any paid API use isn’t under a robust, enforceable contract. The only question is whether the contract language itself effectively prohibits distillation.

Yeah, it wouldn’t be copyright. It might be trade secrets, though. And trade secrets can be made out of public data, but arranged in a way that gives competitive advantage (for example, customer lists themselves might be trade secrets, even if each entry is a publicly available set of name/contact information/job title/company).

The actual process of creating semiconductors is basically:

1. Etch a stencil that has the pattern you want.
2. Place the stencil over a piece of silicon.
3. Bombard the silicon and stencil with radiation so that the chemical properties of the silicon change exactly under that stencil.
4. Repeat the process with multiple other stencils, so that the resulting silicon has basically shapes of wires and logic gates that can perform different functions with the electricity running through those shapes.

In recent years, step 3 has gotten so complicated, based on needing to create radiation of exactly a particular wavelength of extreme ultraviolet light focused exactly on the silicon (and the mask/stencil above it), because that wavelength allows for the smallest possible features on the silicon. So they take purified tin, melt the tin into molten liquid, and ejecting the molten tin in a liquid jet downward into a vacuum at exactly the right speed to where it forms into droplets of the exact size for the machine (about 50 μm), then blasts each droplet, mid-fall, with a 1.6kW laser that heats it up so hot that it vaporizes and ionizes into plasma at the exact position where a system of highly polished and precisely positioned mirrors focuses the UV radiation evenly onto the silicon surface.

Oh, and the machine makes one tin droplet every 1/50,000 of a second, so in any given second it ionizes 50,000 droplets in the stream.

The machine costs something like $300 million, and requires full time experts to make sure that it’s working correctly.

Everything else in the fabrication facility is similarly complicated, which is why a fab represents something like $30 billion in total costs over its lifetime.

The Arch Wiki describes the AUR in plain terms: it’s a user-submitted community repository of software, not warranted to be safe or even vetted by Arch maintainers, packaged to be friendly with pacman.

If you’re doing things the “Arch way” the differences between the AUR and officially supported packages should be obvious, and you should at the very least skim the PKGBUILD files to understand where things are coming from and how they work.

“I outsourced the copywriting to the lowest bidder, who happened to be in Poland”

At least they moved onto year-based versioning. That was probably the best part about the 26/Tahoe release.

It’s called decoding and encoding.

But the big data centers doing all the video processing for the big video services (including both permanent videos from a library and things like live streaming) are encoding the videos with settings that require less computational power to decode. The idea is to be able to let even old budget smartphones still be able to display the video with very low power requirements on the client device. There’s no universe where consumers decoding digital video will be a high-power computational task.

Restaurants have sharp knives in the kitchen, but generally serve food that requires only minimal cutting effort from the table knives set out with the rest of the table settings. Dining will always be easier than cooking, by a margin that makes the difficulty of dining not worth mentioning, so it would be bizarre to criticize a knife as being only good for cooking and eating food, when plenty of dining tableware knives out there would be insufficient for kitchen work.

You’ve made the mistake of lumping decoding and encoding together based on the algorithmic/mathematical similarity of those tasks, when everyone else is more inclined to discuss the very different end user use cases of those computing needs.

I don’t think government funding can actually offset the crash in consumer and business demand being insufficient to cover the cost of the most expensive models on the most expensive GPUs. But if you look through my comment history I’ve made the comparison to supersonic flight, because I genuinely believe there’s a possibility that governments fund the expensive branch of this technology for their own military or surveillance or law enforcement purposes without the benefits necessarily actually spilling out into normal commercial applications.

We’ve hit the point where training a model (both pre training and post training) isn’t the expensive part, and the expensive part is actual inference, which makes it hard to scale the most expensive models to where it’s useful for a lot of people. So it might be that the companies and governments that can afford to operate an expensive model might be the only ones to do it. And they’ll be able to, without necessarily the public being able to have access to the same tech.

Plenty of examples of companies spending more than they earn for decades. Before OpenAI and Anthropic, though, nobody has ever needed to raise more than $100 billion from investors before turning a profit, though. The scale is immense, enough to where it affects the liquidity of the investors that have funded their rise.