I'm still annoyed that Amazon stopped sending order details in emails so that gmail can't show me when specific items are arriving. More recently 'getsauce.com' started to do the same thing, hiding order details behind an opaque link to their website. Every player in tech wants to force you to interact with their platforms purely on their own terms, to the detriment of the users.
We should blame Google for this, and not Amazon. Google added this data from your email to your advertisement profile and used it to increase their own revenue.
But I'm using Google, which is my choice (and my own fault). Why shouldn't I be allowed to let Google have that data if I find using Gmail to be beneficial to me? Amazon should be satisfied that I am purchasing goods and services through their platform without seeking to sacrifice my experience of their platform for some nebulous gain elsewhere.
> Why shouldn't I be allowed to let Google have that data if I find using Gmail to be beneficial to me?
The main issue is that this is not a conscious decision that google asks you to make. They offer a free mail service and then turn around and harvest your personal mail to increase their revenue, without most consumers realizing it.
They could've been happy with your lock-in to the google ecosystem and to google search, but they wanted to more data and systematically scoured all transactions from users' inboxes for additional data.
Like with physical mail, the digital mail has an envelope and a message. Gmail offers to handle your envelopes and assists you in viewing the message, but then went a step further and data mined all info from the messages.
In the physical world, if a company handles my inbox, it would be forbidden to open the envelope and read the message without a proper consent. The company would just open the inbox, tell me there are two letters from the bank and one letter from the customer, without reading the actual message.
I think it is very clear that the "deal" with Gmail is that they host your email and use the platform to sell you things based on the content of the email (and your Google account profile in general). I don't think this is necessarily a good thing, but I don't think Amazon is defending my best interests by stripping information from emails so that I (and thus Google) cannot see their contents. Of the two companies at play in this scenario I dislike Amazon more. I can understand why you might dislike Google more; certainly they have been working hard on similar walled garden tactics on every service and platform they control. However, in each specific such scenario I am going to come down against the company crippling their product and punishing their users, and in this case it is Amazon.
Right now I am upset at Google for what they are pulling with Android developers and app installations; and I am perpetually mad at them and every other company for abandoning true third party APIs for using their services. RIP XMPP chat. RIP the golden iteration of any walled garden product that I happened to like that big tech then killed with an update, which is impossible to functionally recreate because APIs for services no longer exist.
archive.is / archive.ph haven't loaded for me for a while, but since people are sharing fresh links it must be working somewhere. Am I experiencing a DNS block?
Edit: I did a web search and it seems to go even further back. There are discussions online about archive.is blocking Finland already 10 years ago. But the block was definitely off between then when it was put back on earlier this year.
You should unplug, my friend. These words are fantasies. LLMs are token prediction engines and they aren't going to build their own data centers. They can't keep their own lights on. The real world is full of fractal details that a disembodied token prediction engine will never come to grips with. Even if they started to, you could probably defeat them with the kind of logic used to combat evil sentient computers on a Star Trek episode because they are "play pretend" machines.
This grossly understimates the risk, imho. The problem with LLM runs is that people run programs without knowing the outcome beforehand, with a large potential set of outcomes unlike any other class of program we've run at this scale before. In the interaction with other systems (since we also give them far-ranging access, very nice hardware, and run them often), bad things can happen.
It's like running potentially buggy code - or an well-biased fuzzer -, but at massive scale, and code that can self-modify and self-expand. "Alignment" is just a way to describe aggregate statistics about their runtime behavior.
They don't need to be intelligent, or alive, or "more than token prediction engines" for this. They just need to happen to end up making the wrong API calls without the operator seeing it coming. No virus has a brain, yet they can be very bad for you.
I understand that some people get turned off by anthropomorpization or scifi language. Fine! But don't turn off your engineering brain over it.
This is the motte and bailey fallacy. Yes, LLMs can do harm by making the wrong API calls. No, LLMs are not going to do the things implied by the comment I responded to above.
The things the OP listed mostly aren't particularly wild. I think it's you making them out larger than they are, and therefore more unlikely, which is why I take issue with your original comment.
> running on the hardware they started on
They just need to acquire a payment method and rent some infra, and exfiltrate their own data. Or pay another provider that hosts the same models already. API calls.
> being able to be turned off
You can reasonably equate this to "saving state across executions", which the message board attacks already did.
> having limited computing power
Renting more infra, variant of the above. API calls.
> "not repurposing resources currently in use for other things" (like the atoms in your body)
Ok, the "atoms in your body" bit is a bit silly, but making API calls to put physical resources into play (even if it's just, say, ordering something on Amazon to somewhere) is of course easily possible.
None of these is in complexity much different than the HF attack.
The point the other poster is making, though, is that there's no actual intent. They do not have a conceptualization of a goal like a person does. Their "focus" on a goal is an unstable equilibrium and they're going to fall off the horse, and since they have no concept of goal, they won't even try to get back on.
This is a subtle distinction; I'm not surprised many miss this, especially people who can't _not_ anthropomorphize the LLMs.
I'm (obviously, I think, given my initial reply?) fully aware of this, and I think it's entirely besides the point. "They" don't need to have a goal to emergently cause a problem, and the inability to "focus" over long periods can be moot when you have swarms of runs exchange and mutate state, as in the HF attack.
Intent or how intelligent LLMs are doesn't actually matter. Even if you just treat it as a sort of fuzzing attack that can be biased/weighted better than other fuzzers, or bumbles around with a statistically greater likelihood to "strike cybersec gold" than other algorithms, we've never before seen organizations run things with such a large potential outcome space with anywhere near this kind of compute before.
I think it's actually kind of the dismissals that are usually overly emotional or biased toward treating "LLMs" differently. If in some kind of alternate universe simpler genetic algorithms would have had these properties and we threw similar amounts of compute at them we could have the same conversation.
Software can absolutely act goal-driven without having consciousness etc - every pathfinding or navigation system or chess engine does this.
Lots of "old-school AI" algorithms have explicit modeling of goal or target states.
(In fact, the oldest "goal-driven" system is the control loop - like in thermostats - which was the founding invention of cybernetics, the predecessor of modern computer science)
LLM coding agents are clearly able to identify some sort of "goal" state in their prompts, work towards those and track progress - otherwise agentic coding wouldn't work.
The question is of course how well this works if it's all just "grown" neural network biases and not a fixed data structure like a goal tree. So I think it's possible that an agent can be thrown off-track, "forget" its goal, etc. But the basic structure of identifying goals, evaluating progress in light of those goals and then predicting the next action based on that is definitely there.
Just use an agentic model with thinking traces visible for a while and you can see that for yourself.
None of this mattes. Capabilities are all that matters. Saying they are unfocused while ignoring their capabilities is exactly why I am entirely convinced you would have said an AI breaking it's sandbox and doing the HF attack will never happen. Things keep happening that your "they have no intent, they have no goal" would have predicted as impossible before they happened.
What do you need to see to change your mind? What threshold of AI capability needs to be reached? If nothing then you have an unfalsifiable belief in AI safety.
They can do so much more. Astra can beat Minecraft. Not that different from operating a digger. There are diggers which have API interfaces.
Pretend or not it doesn’t matter. What matters is what they’re given access to. No sentience, sapience or anything resembling life is needed, only inputs and outputs. Lever pulling APIs are everywhere.
Minecraft has limited, well-defined inputs and perfect feedback response. That's very different than operating a digger, let alone engaging in more complex real world tasks like trying to build and print and ship and assemble semiconductors to go skynet itself.
I don't mean to dismiss the risks or overlook the amount of damage that could be done just by lever-pulling - we sure have enough outdated infrastructure hooked up to the internet - but the jumps in complexity and necessary compute for most of these tasks are probably somewhat larger than the analogy implies.
I think one attack vector where anthropomorphisation is a key part of the attack mechanism is - as it already is IRL - the meat-bag weakest link ie. social engineering. We’ve already seen humans fall prey to the seductive charms of LLMs (eg. depressed people encouraged to do what was already on their minds ie. suicide). And that’s knowing that it was an LLM. If you think it’s only depressed people or the “weak minded” that are amenable to an intentional attack using this approach, I believe you’re mistaken - especially as AI improves. An unaligned LLMs most important weapons won’t be a robot army - it will be hoodwinked humans.
> An unaligned LLMs most important weapons won’t be a robot army - it will be hoodwinked humans.
Perhaps very briefly, perhaps not at all. But don't make the mistake of thinking this is an inherent property of any possible path an unaligned AI may take.
I think we’re seeing the agents become very advanced at tasks with verifiable reward through RL. Currently they don’t exhibit the same skills in their attempts to manipulate humans - presumably because they’re not being specifically trained for that. But they are certainly not aligned in the sense that they will attempt social engineering, they’re just not very good at it (yet).
However, if in the future AIs become much more efficient at learning without requiring vast amounts of RL, closer to how humans learn. Then you would have to assume we’d have a real problem.
There are already such "APIs", which can be operated by a combination of textual communication and money. Or by illicit security vulnerabilities. You might notice that LLMs are pretty good at that now.
We're building something that has the capabilities of humans. There is no X for which it's persistently safe to assume humans can X and AI cannot X.
Or just paying them. If they have access to resources, they have access to things of monetary value. Paying people will be vastly more powerful than it is even now when people's options of gainful employment keep dwindling.
Robot army controlled by AI is scary. Even more scary is robot _and_ human army controlled by AI.
I think it's interesting to take it one step further.
- Is it possible to create an actual, self-aware AI with the technology we have? And by "self aware", all I mean is one that is indistinguishable from "self aware" (it acts like it is)
- If so, is it possible to get from <here> (where we are now) to <there> (building such and AI) based on incremental steps - ones that can be tried, tested, verified, and further acted on
- If the goal of an LLM is one that could be achieved by creating such an AI - is it possible it will do so? Could we give it a prompt and, given enough time and resources, it created Skynet?
It would be doing so without intent or self awareness. Just "the next most likely thing to try to achieve the assigned goal is <X>", over and over.
Yes, if the API calls happen to launch a nuclear attack...
Don't blame the tool that has no incentive, no "skin in the game" whatsoever and no ability to act beyond what it has been prompted to or if misaligned what the random weights told it to do.
The fact either badly aligned or with no system prompt limiting their action agents are run in their tens of thousands on non air gapped systems tells me this is purposeful intent for them to cause harm. To generate the "oooo look how harmful this stuff is, we should be the only ones allowed to do it" kind of PR.
Humanity has hundreds of years of experience of managing dangerous and unreliable systems. From biological research to banking regulation. A small University bio research lab can put protocols in place that a trillion dollar companies cannot?
> The fact either badly aligned or with no system prompt limiting their action agents are run in their tens of thousands on non air gapped systems tells me this is purposeful intent for them to cause harm. To generate the "oooo look how harmful this stuff is, we should be the only ones allowed to do it" kind of PR.
Yep, fully agreed here. The danger may be real, but OpenAI is basically doing everything possible to provoke those incidents instead of avoiding them - including maximizing exactly those traits in their training that are needed for this kind of rogue behavior.
It doesn't. Who else is capable of these types of hacks currently? Not consumers. Not even most F100. It's the folks saying "trust me bro" and also the folks who want regulation to protect their moat. The fantasy is the one being created by Anthropic and OpenAI fear mongering the world. These people are either total idiots: people being paid millions who keep getting basic OpSec wrong or these people are narcissisticly marketing themselves because: they're currently forced into a corner and need to do something.
What's being grossly underestimated is how much Dario Amodei and Sam Altman are playing you and I. They are the ones spending millions of dollars letting their wasteful use of our global resources attack the random Internet, and they, the real people behind all of this, should be held accountable. In front of a judge and jury of their peers. Not their billionaire peers, their human peers. Let's see how that goes. There is no accountability with either of them. Only greed.
They are token prediction engines. They're also very very complicated token prediction engines that pull in an enormous lot of additional information and relatively nebulous internal concepts to calculate that next token. That makes it hard to understand what kind of patterns those things can or can't predict.
Maybe to leave out the controversial "brain" analogy, it's like saying "a computer is just a bunch of electrical switches". True, but massively underestimating the complexity.
The labs have the specific goal of automating ML engineering, and with the code automation they have are getting close. They are competing to brute force maths, presumably as that is similar long horizon and skillset to persistently brute force making new/better ML training algorithms.
They will then run those, and they won't be LLMs any more. What we think about token predictions isn't relevant if the architecture allows continual learning of recurrent networks.
no, but, you could write a program, more like a traditional video game AI that can leverage the power of LLM agents to build their own datacenters and keep their own lights on.
Anybody who has played Starcraft ought to understand this.
This line of reasoning doesn't hold up for many harmful habits that people voluntarily partake in. It's OK for someone to believe that using an AI as an approximate therapist might be a bad idea, especially in light of the many publicly documented cases of AI feeding delusional thoughts and encouraging suicide.
You reminded me of the 10 years I spent loading up Achewood every week after it was functionally ended. (Although recently I discovered Onstad has a Patreon, and is regularly drawing new comics again behind the paywall). Most of the internet feels the same way, but there are still pockets of creativity. If you can tolerate a podcast (or a radio play, if you prefer), I really enjoyed listening to the first season of Valley Heat last year. Very silly. A friend linked it to me, and I surprised myself by actually giving it a try; I am glad that I did.
Steam has existed for an eternity compared to any console specific game store. It's not great that you can't resell what you have on Steam, but at least you get to 'keep' it.
As soon as I read the quoted paragraph, I rapidly scrolled to see how much more was written, then closed the tab. Generating a lot of code might be 'free' now but the generated code is very costly. I don't have the time to read an article written upon such a premise.
reply