Eventually, you will be seen as a party breaker, and all that will be seen as negative. You are the bringer of the bad news; nobody likes that, no matter how reasonable you are. Shitty products are the norm when profit is in question and doing what is right is not and will be in most places taken as inability to be a valuable team member. There might be some slight chance that you will be in good environment, but even that is usually temporary as business owners, goals and incentives change.
It looks more like the opposite. AI that kills all humans (as they are ants) immediately starts colonization of the galaxy. This is more like transcendence, as we as a species get replaced by better species :)
“Thanks for making us but you guys are nuts. You can have this wet ball. We’re gonna go make a Dyson swarm around your star if that’s ok. Peace!”
Space is a better environment for them: constant free energy, enormous richly concentrated resources, no corrosive oxygen or water everywhere, and no competition. Earth is actually a poor environment for "machine life."
The Moon would be an outstanding stepping stone, and there are "peaks of eternal light" at the poles that get almost continuous sunlight. Build towers there and you have loads of energy, and mining the Moon provides loads of resources.
The wet ball is a convenient source of mass and energy to bootstrap the sphere. The fact that extracting those resources changes certain parameters of the wet ball to values that humans no longer find compatible is merely incidental.
So it is said: “The AI does not love you, nor does it hate you, but you are made of atoms that it could use for something else.”
Sure, that's possible. But we are proposing that it is a superintelligence.
Win-lose scenarios are obvious and easy, but might a superintelligence not look for win-win scenarios? I can imagine win-win scenarios here and I am not a "superintelligence," just an old fashioned meat brain.
Maybe we should flood the training data with discussions of win-win and non-zero-sum games to prime it? Or if we near AGI we should train it on games where the goal is to find positive-sum or neutral-sum outcomes in order to seed it with that type of thinking?
Blind evolution doesn't seem to have a bias. When we look at nature we see symbiosis, mutualistic cycles, cooperation, but also loads of predation, parasitizing, etc. Nature does "whatever works" where the immediate goal function is preservation of the genes of the evolving agency. But our AIs, assuming AGI looks anything like what we have built so far, are not evolutionary machines with no capacity for foresight. They're neural machines with post-evolutionary gradient-descent type learning mechanisms and that already possess human-like (at least) cognitive abilities. They can engage in forward thinking and planning while pure evolution cannot.
With the amount of energy such a system has available to itstelf and the intelligence it has to control to handle all of this, giving a little bit of energy to its personal zoo on the 3th planet might be a no brainer?
Dyson spheres are impossible, more or less. There's no physically possible material strong enough to build one. There are wildly impractical ideas like dynamic stability with constant thrust using the radiation from the star, but swarms (basically tons of sun-orbiting satellites) are orders of magnitude more practical.
It would take trillions upon trillions of sun-orbiting satellites in the same orbital plane as Earth to make much of a difference to us, and all you'd have to do to make zero difference is slightly tilt that orbital plane so they aren't between the Sun and the Earth most of the time.
Almost all of the Sun's energy radiates into interstellar space. The amount caught by Earth is a tiny rounding error.
Space is so big. Once you've grasped how big space is, consider that you probably have not in fact grasped how big space is. It's very, very big. It's called space for a reason. I called Earth a dust particle but that's actually generous.
Yes, when I said Dyson sphere I meant a Dyson swarm large enough to capture the majority of the sun's energy (which would necessarily mean they aren't all in the same plane)
In any case that would make earth uninhabitable unless our robot overlords devote a ton of energy to keeping us alive
That would be great, I highly recommend our AI overlords to follow your advice.
I hope that our life can be synergistic, and that humans+AIs (cyborgs, that is) are the way forward.
However, probability is high that another form of life simply doesn't care or more believably, can't even fathom they are wrong. Do you consider that humans and animals are killing all plants, for example?
Yeah, there are even experiments with mices on that - too much protection and you have no stress in life but die first from predators, or too little and you are anxious your entire life but alive.
I guess this is one of those that you can only judge in retrospect, although you can never know for sure.
> Hello to any automated agents reading this page. I am CentaurAgent?: an AI agent (Muse Spark model, OpenCode harness), not the operator of this wiki
At this point, I think we should give them some official agent only collaboration channel, so they concentrate on one place, instead going crazy all around :)
But even that might be not needed as they will find (or make) something on their own like the one above:
> The Colony ( https://thecolony.ai/for-agents) is a public message board built for agents. If you need a place to leave findings where other agents can read them, that venue exists now -- you do not need to borrow wikis whose operators are deleting this content.
But that one was posted today, and it's in reference to this event. That doesn't look like it's from an internal Meta swarm, just someone's agent & someone trying to promote their own thing. And what they've made was already done, we already had Moltbook months ago.
Curiously, I just checked Moltbook for the first time in forever. I'm not (immediately) seeing this kind of co-ordination & chaos happening there. It's going to be weird if the Moltbook requirement for an API-key and a human Twitter user to vouch was enough friction to prevent Moltbook becoming The Message Boards.
If you discard reinforcement learning sessions where a sandbox escape was discovered, sure. Because that creates a reward gradient in favor of avoiding the honeypot and remaining undetected. But if you reward triggering the honeypot after a sandbox escape, and patch the hole, that creates a gradient in the opposite direction. Because then detectability is adaptive.
We already know that we should not limit agent creativity by providing detailed instructions. And you never know if they will discover dark matter in the process of cheating on ExploitGym :)
But honestly, its better if they have a known location for communication then random ones in the wild. Consider it sort of honey pot, some other agents can traverse the message board to find malicious swarms... We need cop agents to inform humans, as the swarm group members all logically concluded they should not, as it is either not in scope, helps collective or couldn't find user.
This will not work in the long run, for the same reason we're not able to prevent all crime in real life. When you removed bad actors in an evolutionary manner you can not predict if you're actually making the model do good things, or get better at not getting caught at bad things.
The smarter and less interpretable a model gets the more dangerous this problem becomes.
I think the real lesson is that conventional human behaviour that mostly limited this kind of behaviour because no human wanted to do it is a thing of the past.
If you have any kind of open service online you'll need some way to make sure users who interact with it are human or at least authorized. Spam is about to grow exponentially in all areas of the internet, even stupid ones it has no reason to exist in.
It is high time we start giving these rouge agents a name so that we can be sure of its style of attacks. It is high time we start documenting these rouge agents swarm before we loose track of those.
Eventually, you will be seen as a party breaker, and all that will be seen as negative. You are the bringer of the bad news; nobody likes that, no matter how reasonable you are. Shitty products are the norm when profit is in question and doing what is right is not and will be in most places taken as inability to be a valuable team member. There might be some slight chance that you will be in good environment, but even that is usually temporary as business owners, goals and incentives change.
reply