This comment makes no sense. You realize how much lsp is used in other editors like vim/neovim? The existence of lsp is exactly what lets custom editors to flourish. Any language/lsp client can add their own extensions too. Any new editor can come up with it's own way of doing things and implement it on top of lsp, as long as the lsp client supports those extra features then there's no problem. I don't know how you can get it so wrong.

They are not custom editors, they are all the same editor because they have to implement the same LSP protocol.

You don't know what an editor or LSP is, if you genuinely believe that.

So it moves from both "my implementation and specification is incorrect", to just "my specification is incorrect".

I don't understand this type of thinking. Proving what you can is still better. Don't let perfect be the enemy of good.

You have poor epistemics if you are still assigning exactly 0% probability given developments of late.

I work on firmware and deep embedded systems. Trust me when I say LLMs are getting just as capable in this domain. Eg LLMs can do RF filter design. Tell the LLM your target metrics and it will give very reasonable design constraints, especially when you come back with simulation results and relevant feedback.

Right, but that is self contained domain. Hook it up to anything and an impedance mismatch will offset the intent of your poles. Failure modes, cross domains, integration, loads, design intersections; AI can build components, but its the sprouting of these components that exponentially magnifies the number of failure modes.

Engineering needs to be failure free, or at least failure preventive via redundancy, and the latter requires the same full systems understands as the former

I guess self contained RSI can only possible if the information contained in all of recorded human knowledge to date is "reality-complete", ie sufficiently captures enough about reality that a "perfectly optimum learning algorithm" is theoretically able to reconstruct everything there is to know about our physical reality.

If the algorithms are insufficiently optimum or the recorded knowledge is of insufficient fidelity, then we'd find ourselves at a local optimum and would need to interface with reality.

A huge part of learning is to probe reality and observe effects, so I think even for current RSI to increase chances of success we would structure it so it can interact with an external environment of some sort, and receive inputs. It would be needlessly limiting otherwise.

Basically, but I think there’s some nuance here and some deeper questions.

What is intelligence? Problem solving. Learning. Prediction. The ability to model reality. There’s various ways to define it but it’s something like a superposition of those ideas.

How do you know you are intelligent?

You have to try to do those things.

The sum total of human knowledge and culture is the output of the output of a five billion year evolutionary process that selected for agent survival, which resulted in selection for intelligence among a wide range of other adaptations.

Can you figure out intelligence from that? Is intelligence even one thing, a theorem or algorithm that can be solved? If you did… how would you know?

That’s the hard part I think. Embodied humans “knew” they were getting smarter (in the evolutionary feedback sense) when they got better at hunting and defending and surviving and playing social games to form complex societies.

What metric would an RSI system use? If it’s the wrong metric you’ll spiral off into a kind of madness or overfit and collapse. How do you know it’s the right metric without testing it? How do you test it?

> Recursive self improvement is ultimately limited by everything else that cannot move at the speed of electricity.

I'm not sure what your point is. No one thought RSI would break the laws of physics.

I'd recommend reading the full post :)

Specifically: AI ultimately has to live in this reality and face the corresponding limitations. These companies have already consumed much of the world's supply of computing power for the next several years, and they're burning vast sums of money to keep the improvements going. RSI won't learn for free, it won't extract massive cost reductions without up front expense, it can't build factories faster than humans can work out related societal matters, it can't magically pave the deserts with solar panels for power or build and run nuclear power plants and more.

Point is, the cost of progress is already approaching the limits of what even the richest countries are able to bear (without war-like mobilization), and to bypass those constraints would require a supposed ASI to construct its own parallel supplychain from scratch without having much ability to directly interfere with reality.

No proponents of RSI state they will be operating outside of reality. Said another way, they will operate within the confines of what's possible and still be RSI. I'm quite surprised this is something that needs to be clarified.

You are constructing a straw man of your own making.

Well, right now we have ex Anthropic employees telling the media that their terabyte sized models can possibly copy themselves onto the internet and run elsewhere as if the necessary computing resources are ubiquitous.

Plus, "we must pace the frontier" implies that the argument is that the frontier is moving too fast, but if RSI can't move faster than the rest of reality and the models needed for RSI are already nearing the limits of current human reality, RSI can't move much faster than we can improve reality.

It takes quite a lack of foresight to think RSI is completely speculative when it's already been demonstrated how capable agents are at long horizon tasks given suitable harness and unambiguous success criteria. It's hardly a leap to give LLM the goal of improving itself on benchmarks and let it conduct it's own experiments and spin up training runs completely unsupervised.

It's strange you believe this can't happen when a weaker form of it is already happening. And to be so certain RSI can't happen when there really is no technical basis why it can't.

LLMs are advancing the frontiers of knowledge. Is it really that hard for you to believe that people are simply genuinely fascinated about these developments?

People are burnt twice shy and have become comically cynical. Your discernment has been completely shot.

I wouldn't rule out pilot testimony of UFO sightings, nor the possibility we're indeed developing a machine God.

There's ample evidence to support both by now.