A thread for unboxing AI
The rapid progression of AI chatbots made me think that we need a thread devoted to a discussion of the impact that AI i
Sure, they are required to get their vehicle up to certain standards, but they won’t be expected to get their wheel bearing to the point where it will never fail ever.
They will be expected to get it to the point that it won't ever fail under normal usage and maintenance.
Well I think the concern is and the reason why it doesn’t work the way that you’re asking for it to work is that it seems like nothing could ever work if we had the standards that you’re talking about. Effectively you are placing a much higher standard on driverless cars than you are on cars driven by humans, even if the driverless cars would actually be faster, more efficient, and lead to less deaths.
In terms of products whose malfunction could lead to death or injury, all those types of products adhere to the bolded standard.
I will be curious to hear what you think if you read it. To be clear, the book isn't really about AI (although AI is a part of the book) or whether our machines should have legal rights if they become advanced enough. It is a first contact story that explores what it means to be alive, to be sentient, and to be conscious. As I mentioned before, the author does the reader no fav
I read the book over the weekend, I have not read the sequel / second book in the series but plan to.
I will preface by saying that I think it was a good and engaging book. I think you used the word "dense" earlier, which is a good description. With widespread use sentences like "The retros followed some preprogrammed algorithm that mated Newton with a roll of the dice", where you need to know 3-4 different technical concepts to figure out what it means. Each concept mentioned is not necessarily advanced reading on its own, but when packed tightly together it requires more of you. Especially when entire pages are crammed full of such sentences.
I also appreciate that the book is openly pretentious and intellectual, and that the author doesn't give a **** about holding your hand. There is no Andy Weir trying to teach you stuff in paragraph after paragraph, no Peter F Hamilton using broad concepts instead of getting bogged down in details, and no Asimov trying to be a science ambassador. Not that these are bad authors, but we live in times where even the dumbest stuff has to be explained in case a lazy person is on the receiving end, so this style of writing was refreshing. Even better is that there is little effort to make the characters in it especially likeable, but rather focus on making them interesting. The plague of least common denominator characters in modern media is real.
As a small side-note it was actually noticeable that the the author was a marine biologist, and some research showed that his credentials in that regard were pretty impressive.
I’ll put my thoughts on the book’s ideas on thinking and consciousness in a spoiler box since it such an integral part of the plot. This means the spoiler box contains actual spoilers. For those interested in the book, I strongly suggest not reading this spoiler box. The book and its concept is more interesting than my wall of text about it.
Spoiler
As we go through the book, what is revealed is the idea of consciousness as an observer, exemplified in particular with what the protagonist refers to as “blindsight”; that the brain can see without you being conscious of it seeing. The broader concept is known these days as “the illusion of consciousness” or “user illusion”. Basically, consciousness is viewed as little more than an observation of brain activity, it does not in itself contribute anything meaningful to thoughts or problem-solving.
In the plot, humankind as represented by a small team encounters an alien / aliens which instead of having evolved (presumably wasteful) consciousness, has instead continued to evolve increasingly sophisticated problem solving abilities. However, their lack of consciousness makes the team (or most of the team) encountering them continuously underestimate them. Basically, since they don’t see signs of consciousness, they constantly presume lack of sophistication.
Back around when the book was written, this “illusion of consciousness” was growing in popularity on the pop-science circuit, perhaps most famously touted by Daniel Dennet. Around the 2010s it also found some support in simple experiments and certain simulations. We can obviously see the appeal in using this description and apply it to LLMs and their abilities to take instructions and solve problems.
This view and others like it are appealing view for various reasons. They fit in nicely into an existing paradigms and simplified but functional scientific models. A neuroscientists measuring electricity can conclude consciousness is just the waste-product of electrical patterns, a biochemist can blame all actions on hormones and neurotransmitters, a biologist can blame brain activity on the sum of discrete brain-parts doing their own thing, a geneticist can point out what what we become is due to genes, a sociologist can say that how we are is all social learning, an information processing expert can make analogies how the brain is a computer and the “I” is just an OS’ desktop, and an evolutionary psychologist can say that the part that gave us consciousness was merely slapped onto existing brain structures that do their own thing regardless of what we think.
The problem is of course that once you start to add all these things together it goes from “simple and appealing” to “jumbled mess of interactions”. There is this joke in science and especially STEM, that every 101 introductory subject starts a semester by explaining why this particular science is the most essential of them all. While simple and a bit a crude, it is a piece of satire that does fit nicely into this discussion.
To reverse an analogy often used by proponents of the “user illusion”: Yes, a computer works just fine without user input. It even works fine without a lot its peripherals, you can even start ripping out a lot of its finer parts and will still function. However, here is the kicker: The computer will work very differently when you do these things. As a machine, what it can do and can’t do has been radically changed. And yes this is still an analogy, so this doesn’t tell us anything about the brain. However, it tells us that the people using the computer analogy are far more interested in scoring points than they are in conveying how information is processed and produced by a machine.
The best text on the nature of exploring and understanding consciousness is likely still Thomas Nagel’s 1974 essay “What is it like to be a bat?” Even the Daniel Dennets of the world are unable to sidestep the points he brings up. It is also a text I recommend people read for themselves as opposed to reading about it. It is brief, simple and concise to the point where it doesn’t really need simplification, explanation or exaggeration. It will also become obvious how a lot of the protests against his points are nitpicking rather than meaningful arguments.
Anyways, I am dragging on. Do I think the “user illusion” is wrong when applied to humans? Well, my credentials aren’t up to part with the big shots on this subject, but as a once programmer turned psychologist I do actually for once have some paperwork to go with an opinion. I think a lot of this discussion hinges on clever rhetoric more than clever facts. It’s like someone explaining the construction of a skyscraper using Stone Age masonry. Its relevance is often questionable, unless we just keep focusing on the specific details that can be explained using ancient stonemasonry. However, at the end of the day, people limited to the Stone Age can neither design functional skyscrapers, nor build them.
Our prefrontal cortex which is regarded as the part of the brain that gives rise to advanced thinking and which we have not observed consciousness without, continuously shapes and molds our bodies and our nervous system and vice versa. Like a computer’s central instructions, it doesn’t control everything in the machine, nor does it necessarily have to keep controlling things once it has told other parts of the machine what to do. In some cases there probably is “blindsight”, an illusion of control or control without conscious effort. In other cases it is probably vice versa, in that there is conscious control, but we can’t remember it, so after the fact we think of it as unconscious effort. The prefrontal cortex can also work in a less obvious manner by wiring the networks in our brain to respond differently than it otherwise would; It is in effect when we teach a soldier to close distance when fighting, teach a skydiver to jump from a plane, teach a kid to read etc.
Can I say that consciousness is needed for the prefrontal cortex to function like it does? I don't know, but at least in us, they seem to go hand in hand. At the end of the day, this is a discussion where we are likely stumped by the limits of a machine’s ability to understand itself. That is also why in this discussion, the nitpicker or person focused on details over totality will tend to sound the most convincing, because they have the luxury of ignoring the questions they can't answer.
It isn't obvious to me that it would be unconstitutional for a state to abolish its murder statutes.
that would actually be a very interesting constitutional law discussion i think. Legalizing murdeer, theft or kidnapping might be in violation of the due process clause of the 14a.
Maybe not in the originalist reading? but for sure it would even under originalism if it allows police (or any other state employee on his job) to kill, steal or imprison indiscriminately.
Maybe a statute that only prevents murder, theft and imprisonment for public employees could technically be constitutional under originalism? not sure.
that would actually be a very interesting constitutional law discussion i think. Legalizing murdeer, theft or kidnapping might be in violation of the due process clause of the 14a.
I would have to think, for example, about the case of a prison guard killing a prisoner while acting in his capacity as a prison guard. But I certainly don't think it would have any application if the prison guard went home after his shift and killed his wife in a drunken rage.
In any case, we obviously were talking about a broadly applicable murder statute. If the Illinois legislature wanted to make it legal (as a matter of state law, not federal law) for Illinois citizens acting in their capacity as ordinary citizens to shoot each other for sport in the street, I don't see why that would raise constitutional concerns.
In any case, we obviously were talking about a broadly applicable murder statute. If the Illinois legislature wanted to make it legal (as a matter of state law, not federal law) for Illinois citizens acting in their capacity as ordinary citizens to shoot each other for sport in the street, I don't see why that would raise constitutional concerns.
I don't think there's anything the state of Illinois can do to make it legal to play paintball with live ammo. I get murder isn't directly unconstitutional but there are enough SCOTUS precedent re a person not being able to opt out of their right to life and the state's duty to protect that right to make it virtually impossible at least not without major state revisions. It would be like trying to legalize dueling again.
I don't think there's anything the state of Illinois can do to make it legal to play paintball with live ammo. I get murder isn't directly unconstitutional but there are enough SCOTUS precedent re a person not being able to opt out of their right to life and the state's duty to protect that right to make it virtually impossible at least not without major state revisions. It wou
I don't agree with you, and I think you are misunderstanding the relevant Supreme Court decisions.
I don't agree with you, and I think you are misunderstanding the relevant Supreme Court decisions.
Okay. I can see that with my live ammo paintball example, or at least I can’t see why the government would have a duty to prevent it. But I don’t think that carries over to a duel proper. I would think that if a participant were acting under duress, say to avoid social ostracization, the government would have a duty to step in.
But even with the live ammo paintball, that could still apply if a reality show offered a million bucks to play. But I guess that would only render live ammo paintball illegal for the producers of the show, not the players. That got murky in a hurry.
We ain’t seen nothing yet. Wait for the ear buds and mics.
“Can I help you?”
In ear: GPS coordinates show you're at McDonald's. You typically order a #1 value meal, super-sized with a Coke.
“I'd like a num”
“Say no more. Here's your order. AI told us you were on the way and always order the same thing.”
In ear: I'm picking up some distant female chatter. One of them sounds interested in you. Would you like a few suggestions on what to say?
In her ear: Okay, per mutual data exchange I let him know you're interested.
“Heeeyyyy.... My data says we’d be a 98% match.”
In her ear: That’s a line from The AI Pickup Artist. According to your settings you should respond with.... And let me know if you'd like a psyche workup on him, income range, etc.?
Plenty of Fish tried to do that and nobody listened but I bet they would if it came from ChatGPT
I listened to Yudkowsky’s appearance on Ezra Klein - he’s obviously a smart guy (cool that he’s self taught) who spent a lot of time in the field and has what SOUNDS like the right idea? That we should be quite cautious with it if we want to preserve humanity…..and since we aren’t - humanity is likely doomed? What am I missing?
I listened to Yudkowsky’s appearance on Ezra Klein - he’s obviously a smart guy (cool that he’s self taught) who spent a lot of time in the field and has what SOUNDS like the right idea? That we should be quite cautious with it if we want to preserve humanity…..and since we aren’t - humanity is likely doomed? What am I missing?
When you talk about the apocalypse scenarios where AIs turn on humans or abandon humans, what is referred to is "General Intelligence Models" or "Artificial General Intelligence". These are AIs that function at a level of cognition that match or surpass humans in almost every way. They would also have the ability to expand on themselves in some manner that match or surpass our own.
However, we do not currently have such technology. So there is no doomsday scenario, because these machines do not yet exist. The large AI models that you see today are "Large Language Model" AIs, which are essentially extremely sophisticated word guessers trained on vast amount of human data that allow them to with exceptional accuracy guess a on a string of tokens in order.
Think of Artificial General Intelligence as a nuclear explosion. A principle with extreme potential for advancement and technology, but which also opens the door to potential untold destruction. The Large Language Models, at least currently, is more like windmall parks or hydro-dams. They can certainly create problems and disruption of technology, but you're not talking doomsday scenarios.
A far more sloppy (and incorrect) form to state this is that AI in its current form exists as technology that kills you by mistake or design, AI in a future form could exist as technology that wants to kill you. This manner of stating the problem is intended to be illustrative, it is not really formally correct. However, it puts it in human terms that are easier to grasp.
Now, the debates you see today is split in two parts: The first issue is how we should regulate Artificial General Intelligence if and when it it is developed, with the scary caveat that we might not even be able to regulate it - so perhaps we should strangle it in the crib.
The second issue is about whether Large Language Model AI could develop or evolve into Artificial General Intelligence. If you believe that, then you might also believe that we should start regulating even current models of AI as if they have the potential for untold destruction. This debate follows somewhat similar lines to debates on fossil fuels, with lines drawn in the sand alongside corporate interest, technology advancement and human preservation.
When you talk about the apocalypse scenarios where AIs turn on humans or abandon humans, what is referred to is "General Intelligence Models" or "Artificial General Intelligence". These are AIs that function at a level of cognition that match or surpass humans in almost every way. They would also have the ability to expand on themselves in some manner that match or surpass our
Hey really great answer, thanks. Fascinating and scary to think about
Think of Artificial General Intelligence as a nuclear explosion. A principle with extreme potential for advancement and technology, but which also opens the door to potential untold destruction. The Large Language Models, at least currently, is more like windmall parks or hydro-dams. They can certainly create problems and disruption of technology, but you're not talking doomsda
Good analogy. It brought to mind those first nuclear tests. But I think it’s the LLMs themselves that are driving most of the anxiety. A lot of people are still spooked by them.
When AI beat a chess grandmaster, nobody freaked out because we knew it was just crunching through massive decision trees. Same with Jeopardy; it was basically like watching Wikipedia with a buzzer. Even Siri fits that mold.
So the increased speed and data access are definitely factors, but what really unsettles people is how these newer AI agents are programmed to interact, to receive and present data as if they were human.
With AGI, as you said, we haven’t even conceived of what that truly means, let alone how to build one if we did. The human brain seems more like an analog, holistic processor, treating information as a kind of superposition that it can instantly collapse into an outcome.
Can’t recall who said it, but when asked to name one thing AI could never be better than humans at, the answer was lying.
When a person lies, there’s intention behind it. When an LLM “makes something up,” it isn’t trying to fool you. It’s just predicting what a correct or coherent answer might look like, and sometimes it guesses wrong.
Pretty hard to imagine where that kind of genuine teleological intent could come from without consciousness.
It's a fascinating topic. Not new as a topic but coming into focus for people.
I just want to add that 'wanting' may be a red herring. LLMs in a trivialised sense are word guessers but so are 'action' guessers which may be used in robotics/control systems etc. They do have goals. Whether they want their goals is probably besides the point except when we start talking about AI rights. This is no way implies that their goals are determined by humans either.
As for the 'guessing bit' The real issues is how accurate it is. Currently it's impressive but obviously weak compared to humans. If it start 'guessing' so well that it gets the right answer better than humans do then then argument that it's just guessing will be of academic interest only.
The 'want's argument bold down to whether or not 'guessing' (and other AI techniques) can surpass humanity without wanting anything in a human consciousness like sense. Personally I've been grappling with this question or 40 years or so. I guess not but even if we create AI that does surpass us we still wont know.
Plenty of Fish tried to do that and nobody listened but I bet they would if it came from ChatGPT
Makes me thankful not to be in the dating game.
It should be understood that “you” are patterns in spacetime. Humans became a more intricate pattern by a brain mutation that occured around 100,000 years ago. All of historical civ is only ~6000 of those 100000 years. We have reached a point where inevitably the pattern that makes humans humans will change, likely for the better.
It’s trivial to be concerned that “humans are doomed” by a variety of tech developments. A better way of thinking is how can we maximize the benefits to our descendants.
I think of what AI is doing as more cybernetic than teleological. It needs to be told the goal. But it can’t ever generate genuinely novel ideas or goals, because those depend on inference, while it operates purely through material implication. Then again, so does quantum mechanics, and it’s remarkably accurate given the right data. That level of accuracy with language is where I see it heading.
But again, we can’t get the genuinely novel out of what is, at its core, just correlations of material facts. For example, suppose computers and AI had existed in the 1950s. I don’t think they ever would have conceived of the internet or mobile phones. They could have optimized communication, maybe even predicted trends in miniaturization, but not imagined a global, wireless information network. That kind of conceptual leap comes from outside the data, from inference, not correlation.
What I mean by that is that right now people can’t live without their phones and everything that comes with them. But people weren’t inherently different in the 1950s in terms of their needs and desires. The difference is they didn’t know there was a better way to satisfy those needs or wants because they were latent needs. It took humans with skin in the game of improving conditions to infer those latent needs.
In other words, it wasn’t the higher-level math that created AI, or the predicate logic that created LLMs. It was human creativity and inference that used those new tools to create them to improve the human condition.
So despite the risks, freeing our brains from all that correlated data frees us to be creative. We can be like children again, with robotics and AI doing all the grunt work instead our parents.
You got it backwards: spacetime is a pattern in you, your mind. Math works just as well when you throw it reverse.
You misunderstand the realities of what im saying. The reverse is not true because “you” are not your mind or even inside your mind. That isn’t you. What is you is the pattern that plays out as a result of the indestructible information that makes up your body including the atoms that make up your mind.
I think of what AI is doing as more cybernetic than teleological. It needs to be told the goal. But it can’t ever generate genuinely novel ideas or goals, because those depend on inference, while it operates purely through material implication. Then again, so does quantum mechanics, and it’s remarkably accurate given the right data. That level of accuracy with language is where
AI's can do inference and in any meaningful sense can create goals. These appear as sub-goals but that's because we create the top goal. More generally AIs evolve and the top goal is survival of the fittest - we can create swarms of real/virtual agent with the ability to interact and then just let them evolve. They don't struggle to image things - they 'hallucinate' plausible (and less plausible) worlds very easily and those hallucinations can be tied/tested by us or by AI themselves.
It's early days. Whether there are any limitations down to what I shall call consciousness remains to be seen.
The most simplest part of my point is that there is going to basically be an infinite amount of conspiracy theories in regards to AI over the coming decades, as it is incredibly useful and Incredibly poorly understood.
In terms of things like the AGI moment of self awareness or terminator 2 style timelines, almost no chance it plays out that way. There should be regulatory agencies on a global scale to manage this transition so that bad actors do not use AI in horrible ways. Unlike Climate Change, AI is going to dramatically change the world over the next 100 years.
In terms of ending humanity, over time humans will get smarter via ai and ai will get smarter via smarter humans. In the long run given 1000’s of years this will lead to a new species one thats pattern is many standard deviations better than humans now. and thats a good thing.
AI's can do inference and in any meaningful sense can create goals. These appear as sub-goals but that's because we create the top goal. More generally AIs evolve and the top goal is survival of the fittest - we can create swarms of real/virtual agent with the ability to interact and then just let them evolve. They don't struggle to image things - they 'hallucinate' plausible (
I know LLMs hallucinate regularly, and I've gathered they are not really candidates to become AGI, or rather, they aren't real "artificial intelligence".
It's early days. Whether there are any limitations down to what I shall call consciousness remains to be seen.
I'm thinking you're probably right in theory. Given enough monkeys and keyboards along with the computational power to check for coherence, eventually the Pythagorean theorem, calculus, etc. would flag as a hit. Same with something like mouthwash if it hadn't been invented by combing words in the dictionary and seaching for correlates and need, like posts about bad breath. Now I have a headache.
