A thread for unboxing AI
The rapid progression of AI chatbots made me think that we need a thread devoted to a discussion of the impact that AI i
Chez, as an example where recursion cannot be unrolled into iteration, consider traversing a tree structure where every node has a parent node (except the root) and zero or more child nodes. This structure has to be traversed recursively, it cannot be done iteratively. And if the data structure is malformed and there is a cycle, for example A has children B and C, B has children D and E and E has child A, you will end up traversing the cycle and never exit.
I bring this up because I've actually had to deal with this in real life with trees stored as data in a table with foreign keys from each child to the parent. It's very easy to accidentally end up with cycles in such a structure and, with only TSQL recursive CTEs available for traversing it, cycle detection is.... not straightforward.
A more trivial example of where recursion might fail due to cycles would be a linked list where one of the pointers is to an element further back in the list. There are well known algorithms for detecting this, for example you traverse it with 2 cursors, one of which goes twice as fast as the other, and if they ever end up on the same element (i.e. the fast one "laps" the slow one), there is a cycle. https://en.wikipedia.org/wiki/Cycle_dete...
That's just a couple of examples of problems you can run into with recursion that you won't run into with iteration.
It can be done iteratively. I would definitely agree that in many cases recursion is far more natural and hence superior. Trees can be handled although I can't imagine why anyone would want to apart from maybe for optimisation.
Here's an intro from MIT
6.101 Fall 2022: Recursion and Iteration
Introduction
This reading examines recursion more closely by comparing and contrasting it with iteration. Both approaches create repeated patterns of computation. Recursion produces repeated computation by calling the same function recursively, on a simpler or smaller subproblem. Iteration produces repeated computation using for loops or while loops.
If we can come up with an iterative version, do we need recursion at all? In one sense, no, we don’t need recursion – any function we can write recursively could also be written iteratively. But some problems lend themselves naturally to a recursive solution. When we try to solve those kinds of problems iteratively instead, we find ourselves simulating recursion anyway, using an explicit agenda to keep track of what we need to do next. For those kinds of problems, it’s more straightforward to express the recursive algorithm directly, letting Python handle the agenda bookkeeping using its call stack.
The converse is also true: any function we can write iteratively, with a loop, could also be written recursively. So it’s important to be able to think in both ways and choose the one that is most natural and appropriate to the problem we are trying to solve.
Recursion, you see, produces repeated computation by calling functions recursively. Hope this helps.
Fair enough - if your thesis is that it's possible to simulate recursion with iteration, I won't argue, and I will read the full article you linked when I have a bit more time.
This is more academic than anything though - as you alluded, I can't think of why anyone would want to do such a thing out in the wild.
Wont disagree with that. Edit: apart from optimisation - I can image that being worth the effort in some situations.
Another academic way of thinking about it is turing machines. if it can be done then it can be done without anything fancy.
You can do it all with NOR gates aiui.
Blimey i can remember converting logic statements to NAND gates. lol me
https://web.mit.edu/6.102/www/sp23/class...
If I've understood this correctly, looks like to traverse a tree they're basically re-implementing the stack in the higher level language they're using. Logically it's still recursion, and would encounter exactly the same potential issues as a recursive implementation (possibly with the exception of a stack overflow). The "agenda" is the stack by any other name, and I think they say as much in the paper.
All I know is AI is amazing for the computer illiterates. I'm doing stuff weekly in my job that would be outside of normal IT department stuff but never would justify a developer hire.
All this talk makes my head spin. The important information is maybe chez and d2 sort of agreed on something?
All this talk makes my head spin. The important information is maybe chez and d2 sort of agreed on something?
I think the bigger takeaway is that chez managed to post something which was both coherent and technically correct. This is not a particularly frequent occurrence. Might even be the first time.
Its not easy for it to be simple enough for you to understand
What can possibly be alarming about the only AI juggernaut pretending to have morality sharing this
It's not wrong but I expect the leading companies will happily nod along to the idea of a brake pedal because it sounds like doing something while being utterly meaningless.
It's marketing. They just hired Karpathy to accelerate their model self-improvement.
Had just started playing with the fabled fable. Now more myth than mythos in the uk
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees. The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance. Access to all other Anthropic models will not be affected.
We received the directive from the government today at 5:21pm (ET). The letter did not provide specific details of its national security concern. Our understanding is that the government believes it has become aware of a method of bypassing, or “jailbreaking” Fable 5. We reviewed a demonstration of this specific technique being used to identify a small number of previously known, minor vulnerabilities. These vulnerabilities all appear relatively simple, and we have found that other publicly-available models are able to discover them as well without requiring a bypass.
Fable is the one that won't let you use it for cybersecurity anyway so there is no point to restricting it afaik. I have access to it but I don't know that it's worth the double tokens for my non-swe use cases.
people were sharing jailbreaking methods within hours of it getting released
in my limited usage i found it wasn't significantly better at what i was doing and cost 5x more so stuck with opus
If the Mythos family costs more to train than is economically viable to sell, there must be an intervention to cause scarcity and increase demand or the bubble will pop. Distillers are only 6 months behind the latest model release. This story could potentially help Anthropic and other US frontier labs avoid commodotization by creating artificial scarcity. Unclear how they profit, but something is required to make the math work. As it is now, if every single person on earth signed up for Claude, Anthropic would instantly go bankrupt. If a more sensible number subscribed like 500 million and used the service, they would also go bankrupt. They lose money on each subscriber. There's no economic model that produces profit and avoids their tools becoming a commodity without external forces. This (or something) action is required in order to avoid the AI bubble bursting.
Based on my observation, this is likely true. It's likely that Fable is close to as good as it gets for what's currently viable to train and attempt to pretend to sell for profit or whatever they do. I guess they don't even do that. Attempt to give away at unsustainable prices to acquire future users who their bosses at work will pay 10x api rates and maybe they can almost break even on training.
