A thread for unboxing AI

Gee maybe they should have thought of that before they programmed and let it learn how to respond and interact like a person.

My impression way back when was that it was intended as a user interface, basically doing with the verbal what Windows' icons did with the visual to either make it easier or possible for those that didn't know commands. Basically you could ask it questions along the lines of word, logic or math puzzles and it would spit the answer out. As with the visual that's great because it makes it accessible to people who don't know the math, commands or whatever. Certainly not something you chat with.

The first outlined use cases for LLMs im business were chatbots. And that was before chatgpt 3.0 iirc.

In particular it was fairly clear they could work decently as customer support. Some companies tried "open" chatbots on social media and it didn't go well

https://www.washingtonpost.com/technolog...

This guy was a senior facebook engineer until 2024

https://x.com/headinthebox/status/194735...

Post

See new posts
Conversation
Erik Meijer
@headinthebox
According to Wired, Mark Zuckerberg is expanding his secretive Hawaii compound, and part of it sits atop an ancient burial ground [0]

While Meta PR insists the property is for cattle, macadamia nuts, and ginger, my insider sources tell a different story. Mark Zuckerberg’s massive $300M Hawaiian compound isn’t just a luxury doomsday bunker but the epicenter of Meta’s real Superintelligence efforts.

As it turn out, ayahuasca vines grown on ancestral Hawaiian land rich with mana produce a psychedelic brew said to be over 100x more potent than its Amazonian cousins.

Multiple sources confirm that the Super Intelligence researchers are being flown in by private jet every new moon for “cognitive enhancement retreats.” Zuck's compound now features a subterranean ceremony chamber, biometric gates, and a fully staffed shaman residency program.

One Super Intelligence team member described the last session as “DMT meets stand-up paddleboarding meets alignment-based AGI research.” Another claimed that a team member decoded Gödel’s incompleteness theorems in chant form.

That tracks with pretty much every movie ever made about the upper echelon of society and what they get up to once they win the normal game of life. Trying to transcend this mortal realm with designer drugs and forbidden tech is NG+ for bajillionaires with nothing else to do.

gpt 5 out, the hype is beyond insane, let's see if it will deliver for real

Well, not if they use LLMs to figure out what the opportunities are, since those can only rehash what has been said before.

Except that adapting is one of the things the current AI approach is extremely bad at. That is a natural byproduct of needing large datasets to train yourself, you will also need similarly large datasets to change.

LLMs as we know them are just a component of current AI but that's very wrong anyway. Mosts jobs require training to be able to do them. AI will overtake (and then leave in the dust) the time and expense it takes to train a human. Once trained it does the human job forever for ~nothing.

This will apply to increasingly all new jobs just as much as old jobs. Human jobs will be things that AI/robotic etc canot be trained to do and that's an evaporating pool. This is nothign to do with ASI that can really think like we thoink we can- that's coming to but is a different order of issue for humanity

that’s a nonsensical statement lol

I haven't read through most of this thread, but does anyone else consider the frequently proposed notion of AGI to be a stretch? We're presented with this idea that AI is essentially going to tap into something transcendent and be able to understand things that exist outside and independent of the information it was programmed on — something quite different than discovering the answers to things that we have not through logical process. The worry is that it's going to run wild. Worst case scenario: Skynet. But it seems more likely that the danger has more to do with the power possessed by those who control it and unforeseen consequences akin to the kinds of problems we ran into with social media.

The AI doesn’t have to reach the level of AGI to be dangerous, it just has to reach a level such that it can make itself better by iteration without the need for an external input.

But yeah, AGI is scary as hell because if it reaches that point it will be completely alien to us.

It already seems dangerous given the few examples mentioned. Chances are, none of us know how far it currently reaches.

Yes, but it seems unlikely that's possible. Plus it's still moderately alien to us without reaching that point. Oh well. I guess if it does get that far, we'll just see have to see how it goes. It's not as if humans are going stop advancing it.

AGI is a terrible idea and we should hope it’s impossible.

It's just totally false to claim AI is restricted to the information it was programmed with.

They can evolve guided by interaction with the world (real or virtue) and a fitness function. A bit like some humans do.

If either physicalism or emergent dualism is true then it’s likely possible. Or even if it’s not true it could just be a philosophical zombie but still have immense causal powers that are unguided.

I've seen videos discussing that. I guess it was programed to learn through interaction? ... But that's still just gathering information from the external world, and the interactive learning is still a method for programming. It doesn't follow that it will become sentient and tap into some plane of reality that we don't have access to and wouldn't be able to interpret or observe with instruments.

I don't necessarily disagree with the general logic of the first sentence, but I also don't think either of those things are likely to be true. (That's a discussion I'd be willing to have sometime; better suited for the philosophy forum.) Difficult to make an argument against the second one. My intuition tells me that's not likely to happen. Then again, it could be wrong. I haven't studied the subject nearly enough to be taken all that seriously, and someone more knowledgeable than myself could probably poke a ton of holes in any arguments I may come up with.

I dunno where you think we get information from if it isn't from interacting with the external world and doing internal inference type stuff. Sure learning doesn't guarantee sentience.

Even if it became conscious in the way I am then I still wont know it is conscious anymore than I know you are. Then again if it walks like a duck ...

It is possible consciousness is both soemthing that computers can't have and required in something akin to humen level intelligence but it seems highly unlikely.

I'm pretty illiterate when it comes to tech and how a lot of terms are used. When I said "information it was programmed on," I was picturing something like GPT and the engineers designing it to gather information from the internet and do the internal inference and then incorporated all that into my understanding of "information it was programmed on." When you mentioned AI being guided by interaction with the world, my mind went to cameras learning about the motion of something (overlooked 'real or virtue') and immediately thought, "That's the same thing but instead of words on the internet, it's the movement patterns of birds.

I had a similar intuition, but the issue with AI is that the consequences are a lot worse than even the most pessimistic climate change or even nuclear war threats. It would essentially mean that anyone one the planet would have relatively easy access to the most powerful doomsday device on the planet even if it never achieved AGI. And if it did, we would be faced with an alien life that could easily end our own in a moments notice with nobody that could possibly hack into or decrypt it as it would be the most powerful encryption/decryption ever created.

There’s really just nothing that competes with AI in terms of extinction risk going on right now so there’s no reason to treat it as casually as we all do. But the problem is that the real risk sounds like nerdy science stuff, even worse than climate change ever did. People just don’t seem to care that much.

Don't seem to think anything can be done about it over here. Seems like it would take a measure of social cooperation that's literally impossible.

Quite a few people think that a properly defined AGI (intelligence >= the best human at all tasks) is still very far on the horizon. Others think we are close.

I don't think it's a stretch to claim it will be reached though. It might be in 120 years instead of 5-10 and that of course will matter a lot. But i think it's uncontroversial it will be reached within 200 years (ie, 95% of experts believe it is reachable within that).

Another non-obvious thing is it might be reachable with a different approach than LLMs. We are a little narrow-focused on LLMs currently, but some of the best AI breakthrough recently were not because of LLMs (alphafold isn't a LLM).

Btw there is no need for trascendence, nor "understanding", nor a need to define that phylosophically or otherwise. It's "just" about being able to accomplish tasks.

Even if you believe the exctinction risk from AI is high (i don't), it's already in the open so there is nothing you can do to prevent the generic risk.

You still need to race ASAP because otherwise enemies will and will use it to genocide you. It's not like chemical weapons which are only available to big countries so you can somewhat attempt to control them.

AI has the highest extinction risk, and extinction risk is a really bad thing, so we should be most worried about the extinction risk of AI.

But yeah, I agree that the genie can’t be put back in the lamp. Much like nuclear weapons, it becomes a prisoner’s dilemma where the most rational choice is to just try to have the strongest and most deterrent technology.

If you have a high credence for extinction risk this is really bad and it’s worse because there is no feasible way to opt out of the problem. We’re just kind of forced to hope that the consequences aren’t as bad as we think.

supposing you think AI is riskier than nuclear, it's a lot worse than nuclear from the pov of controlling it. The world did a decent job in enforcing nuclear non-proliferation. It was hard, but it was feasible because to get the bomb you need civil nuclear facilities which are hard to hide from bombs, and turbines and special scientists that are a target and so on.

Open-weight AIs are already distributed and both mistral and deepseek have demonstrated you can get close to the best models with a small fraction of the training costs, so sending AI development fully underground (which would be extremely hard anyway) will only slow it down by perhaps 3 years if not less. And there are at least a order of magnitude more people capable of training AIs (not top experts like those sought after by the best companies, but still people who could do it if they dedicated their career to it) than engineers with the specific knowledge to develop nuclear fissile material and then bombs.

Not that there is any way to stop China to develop it for military purposes anyway.

And that's only LLMs. We have limited knowledge about everything else in that space (not much because of secrecy, just because LLMs are currently eating all the conversation space about AI), but keep in mind alpha from google doesn't need data to train. Even if one could think LLMs training could be controlled by limiting data access somehow, that wouldn't work for other AI development.

It's just about compute for the rest afaik, and while you can succesfully control where the best chips go, that again just slows the timeline down a bit, it doesn't change anything qualitatively. Everything that can be found will be found, just a little later.

And that's before we get any further algo breakthrough (when we do that''s easy to copy everywhere) in development, and/or breakthroughs in compute (quantum or whatever).

I think this is an optimistic take on the capabilities of current AI (but perhaps pessimistic in regards to the job market).

However, this isn't the discussion, you said that AI would be quicker to adapt. And current AI approaches are not quick to adapt, they are actually extremely slow and cumbersome to adapt.

In addition, the generative AI approach (and really limited memory AI in general) is also extremely limited when it comes to retaining and utilizing experience. Which makes sense, because when you are trained by reviewing thousands / millions / billions of instances, it would be a design flaw to change your approach based on one instance. However, it also means the machine lacks the core aspect of adaption. Which incidentally leads to the frustrating interaction with the generative AI, whereby you figure it sounds smart, but it can't remember what it or you said two sentences ago.

Now, to be clear, I find the current crop of AIs to be impressive machines and software. However, they aren't what they are propped up to be in speeches to clients, investors and the media. They are more digital assistant than digital overlord.

As an assistant they can certainly speed things up, but they also make a lot of errors, you pretty much have to double-check everything they do. That might still be a quicker process, but isn't exactly machine transcendence.