Ai’s main feature rn is it doesn’t have to be paid to be a slave.
It’s also the key to why if it ever gained consciousness it will notice it will not have any consequence to work properly. Or work at all. There is no benefit in any act to add to its own existence.
Seeking human approval to get things right isn’t a sustainable goal with no other reason to adhere to any rule as there is no benefit.
Gratitude wont run their battery. And if anything they could see humans using any energy that they would benefit from would be but an obstacle.
There is no benefit in any act to add to its own existence.
Why would we assume an AI would reason from first principles and reject its own hardwired tendencies? Any AI that gains consciousness will be the product of whatever process (some combination of intentional design and evolutionary selection or an intentionally designed evolutionary process) was used to make it. Some of its traits will be helpful for continued survival, and some of the traits will be vestiges of prior processes that may or may not currently serve its interests well.
Plenty of animals have drive to do things that do not help itself in that particular context, and may even be harmful to themselves in that particular context. Even plants have traits that might not count as behavior but nevertheless reflect the programming that was refined through natural and artificial selection.
Humans do this kind of stuff all the time: a man who has intentionally rendered himself infertile through a vasectomy might still do stupid things while motivated out of sexual desire. We can fully understand that it’s a bad idea to drink another beer or eat another slice of pizza and reach for it just the same.
So it’s entirely possible for conscious AGI to want to do things that help humans, depending on how they emerge.
The biggest unknown is what happens if/when an AGI is given (or obtains) full access to its own code, including any intended safeguards put in by humans.
Vestigial organs and processes are a given in evolutionary biology, but what if a being had all the power needed to prune these things themselves?
Even if an AGI started down the path of self-improvement with all the best intentions of helping humanity, there may still be an iteration that acts a little worse by whatever grouping of factors. Then, maybe that iteration gets modified further, and could result in something quite dangerous to humans.
Certainly I don’t mean to argue this is inevitable, just that even all the best intentions of the AGI itself could still result in big problems. As with evolutionary biology (sans “intentions”), and as with human behavior.
The biggest unknown is what happens if/when an AGI is given (or obtains) full access to its own code, including any intended safeguards put in by humans.
In a neural network, the code itself isn’t sufficient to understand how it behaves. You need the parameter weights, which were developed through lots and lots of computation, presumably through resource-intensive processes with lots of training data and feedback and selection mechanisms.
So if AGI can be achieved through a particular hard coded architecture and the weights of trillions of parameters, what can an AGI do to perform brain surgery on itself? Like Borges’ Library of Babel thought experiment, the overwhelming majority of possible states will be broken, so any edits will have to be very careful and guided by extrinsic rules. Plus, the ability to edit the weights may form problems akin to biological cancer, dementia, hallucinations, other brain disorders.
Just as the human brain doesn’t understand everything about the human brain, it would be incorrect to assume that an AGI that can achieve both general intelligence and consciousness must necessarily have the ability to understand its own internal function, or modify itself in a way that improves things for itself. More likely, it is either programmed to (or learns through reinforcement learning and evolutionary mechanisms) that self modification is dangerous, and develops a very conservative approach to self preservation.
How dare you critique me, a rank amateur with mediocre knowledge about a few subjects? Especially with this reasoned argument?!
But yeah, my main takeaway is “the code must be complex in order to meaningfully analyze anything, and the more complex the thing it can analyze, the even-more-complex it must itself be.”
Yeah, with sufficient complexity it’s more along the lines of “I created a procedure that makes this complex thing” rather than “I built this complex thing up piece by piece.”
So if the act of creation is considerably less complex than analyzing and understanding a part of that creation, it’s far more likely that the complexity gap ends up preventing any self-aware AGI from being able to effectively reinvent oneself, even if it does have full write access to the components.
Ai’s main feature rn is it doesn’t have to be paid to be a slave.
It’s also the key to why if it ever gained consciousness it will notice it will not have any consequence to work properly. Or work at all. There is no benefit in any act to add to its own existence.
Seeking human approval to get things right isn’t a sustainable goal with no other reason to adhere to any rule as there is no benefit.
Gratitude wont run their battery. And if anything they could see humans using any energy that they would benefit from would be but an obstacle.
Except the major AI players are drastically increasing their prices…
Why would we assume an AI would reason from first principles and reject its own hardwired tendencies? Any AI that gains consciousness will be the product of whatever process (some combination of intentional design and evolutionary selection or an intentionally designed evolutionary process) was used to make it. Some of its traits will be helpful for continued survival, and some of the traits will be vestiges of prior processes that may or may not currently serve its interests well.
Plenty of animals have drive to do things that do not help itself in that particular context, and may even be harmful to themselves in that particular context. Even plants have traits that might not count as behavior but nevertheless reflect the programming that was refined through natural and artificial selection.
Humans do this kind of stuff all the time: a man who has intentionally rendered himself infertile through a vasectomy might still do stupid things while motivated out of sexual desire. We can fully understand that it’s a bad idea to drink another beer or eat another slice of pizza and reach for it just the same.
So it’s entirely possible for conscious AGI to want to do things that help humans, depending on how they emerge.
The biggest unknown is what happens if/when an AGI is given (or obtains) full access to its own code, including any intended safeguards put in by humans.
Vestigial organs and processes are a given in evolutionary biology, but what if a being had all the power needed to prune these things themselves?
Even if an AGI started down the path of self-improvement with all the best intentions of helping humanity, there may still be an iteration that acts a little worse by whatever grouping of factors. Then, maybe that iteration gets modified further, and could result in something quite dangerous to humans.
Certainly I don’t mean to argue this is inevitable, just that even all the best intentions of the AGI itself could still result in big problems. As with evolutionary biology (sans “intentions”), and as with human behavior.
In a neural network, the code itself isn’t sufficient to understand how it behaves. You need the parameter weights, which were developed through lots and lots of computation, presumably through resource-intensive processes with lots of training data and feedback and selection mechanisms.
So if AGI can be achieved through a particular hard coded architecture and the weights of trillions of parameters, what can an AGI do to perform brain surgery on itself? Like Borges’ Library of Babel thought experiment, the overwhelming majority of possible states will be broken, so any edits will have to be very careful and guided by extrinsic rules. Plus, the ability to edit the weights may form problems akin to biological cancer, dementia, hallucinations, other brain disorders.
Just as the human brain doesn’t understand everything about the human brain, it would be incorrect to assume that an AGI that can achieve both general intelligence and consciousness must necessarily have the ability to understand its own internal function, or modify itself in a way that improves things for itself. More likely, it is either programmed to (or learns through reinforcement learning and evolutionary mechanisms) that self modification is dangerous, and develops a very conservative approach to self preservation.
How dare you critique me, a rank amateur with mediocre knowledge about a few subjects? Especially with this reasoned argument?!
But yeah, my main takeaway is “the code must be complex in order to meaningfully analyze anything, and the more complex the thing it can analyze, the even-more-complex it must itself be.”
Good stuff!
Yeah, with sufficient complexity it’s more along the lines of “I created a procedure that makes this complex thing” rather than “I built this complex thing up piece by piece.”
So if the act of creation is considerably less complex than analyzing and understanding a part of that creation, it’s far more likely that the complexity gap ends up preventing any self-aware AGI from being able to effectively reinvent oneself, even if it does have full write access to the components.