• 0 Posts
  • 1.66K Comments
Joined 2 年前
cake
Cake day: 2025年2月5日

help-circle
  • The main thing I find with the LLMs is that they’re very fast and capable and limited. You give them a task of low complexity, they can execute it with superhuman speed, and if there wasn’t hidden complexity in it, they do well.

    If the task is too complex (beyond their context window’s capacity), they start to “hallucinate” - make up things that sound reasonable to their training - to cover gaps in “what to do next” because they’ve forgotten some of the complex instructions. Putting the instructions “in writing” in a reference document that the agent doesn’t edit freely can help a lot, but eventually those documents get too complex as well and again it starts “winging it” rather than endlessly re-reading the instructions before making every next move.

    The models’ training is getting better, doing “the right thing” more often by default - without requiring as much explicit guardrailing. They’re also getting better at paying attention to requirements documents, but they’re far from perfect. As are humans, but humans don’t always move as fast as LLM agents, so LLM agents can make big messes faster…


  • Agentic AI has its place, in a tightly controlled sandbox.

    One of the anecdotes was about an agent which wiped out the production database, and all backups. While you can lay blame on the agent (AI or human) who made the mistakes, the real blame in that situation is with the system architect (likely there was no named system architect, but that doesn’t absolve whoever was acting as de-facto architect) who placed the backups in a hot, online accessible and erasable configuration. No agent, AI or human, should be able to wipe out all the backups with their normal access - ideally there are physical doors and keys involved - in multiple locations if the data is of any value.


















  • In my generation it was the nuclear industry (immediately post TMI & Chernobyl) which was giving away the huge incentive bonuses and premium salaries. I never filled out the Navy application - though I hear they paid the best overall. I did interview twice (nothing better to do at the time), got two offers, declined both, ultimately took lower paying jobs that I didn’t mind the idea of going to work at.