If you're talking about human-level intelligence, I think you're on to something. Old-fashioned AI was probably closer to the right track than we are now.
> did attempts to generalize some of that research fail because the approach was fundamentally flawed, or was it because such efforts themselves weren't as good as initial projects?
In many cases there haven't _been_ attempts to follow up on that early work. It's more like the zeitgeist just shifted to other things. For example, SHRDLU was just a couple years before the first AI winter, and when spring came (~1980) people had largely moved on. (Which isn't to say there weren't also flaws with the approach.)
Not the GP, but some examples are: (1) how do you get a computer to have a human-like train of thought? (2) how do you get a computer to acquire new concepts (e.g. "debt", "global warming", "weed") and then reason about them correctly, without any reprogramming? (3) automated acquisition of common sense through experience (e.g. "if you pour water on the floor you will get a puddle") (4) deep natural language understanding (i.e. how do you make a chatbot that really understands, and isn't just a thin illusion of understanding).
(4) is an interesting question. Unfortunately it's much harder to understand than it is to ask. For instance, to people really understand, rather than just providing a thin illusion of understanding? What does it actually mean to understand something? Can you make a test that can distinguish arbitrary systems which truly understand from those which provide a thin illusion of understanding?
This is a real problem for physics teachers - you want to find out if the students understand a concept:
Ask them to state it - they memorize the text book definition.
Ask them to apply it to a specified problem - they scan the problem for values of variables and look up a formula list to find one that has those variables.
Ask them to explain why their answer is correct and they form a grammatically correct explanation made by plucking phrases from the problem description and linking them to the answer with "so" or "because".
It feels like they don't understand but they actually can get a long way (ie. not fail) like that - it's certainly human level understanding, even if it's not what the smartest of us are capable of.
Personally, I think understanding is a continuum from special case memorization at the bottom, up to being able to link with a lot of other concepts at the top. There's no bright line between "truly understands" and "illusion".
None of the problems you mention is actually solved though. They're all things that work sort of, some of the time, with caveats about how you define "work." They work well enough to be useful, but not well enough to argue we're converging on human-level intelligence.
Optical illusions (there's also physical ones) are often demonstrations that the problems aren't solved in humans either; they're just "things that work sort of, some of the time, with caveats about how you define 'work'".
More so if you include reasoning illusions like people being more scared to catch a plane than drive a car or thinking that a lotto ticket is a good investment.
Human intelligence doesn't really meet intuitive definitions of human intelligence. But it does work well enough as long as you ignore all the times it doesn't.
Yes, this is the problem with AI risk---there's a community pushing hard to gather resources to the cause, but little or no scientific work to be done. This is a rather pathological situation---among other things, the AI risk community makes their own cause look silly, and they promote an unduly negative vision of AGI. I've written more about this here: http://www.basicai.org/blog/ai-risk-2017-08-08.html.
On a positive note, as a piece of science fiction, this was an enjoyable read!
I think you sum up their position's fallacy pretty nicely here:
>"if we don't figure out AGI safety now, by the time AGI happens it may be too late"
The keyword for me is "happens". It's as if technology ever happens, or emerges serendipitously. It's like the Kurzweilian exponential law which make it seem as if there is no agency in technology, a natural law. And our role in it is make sure when the aliens or the gods arrive we are prepared for them.
Yes and no. For safety of narrow AI systems, yeah, there's a lot of scope for research, and that's what your first link gets at.
But for AGI (which is what Tegmark talks about), there's no good way to get a handle on safety yet (other than working towards figuring out AGI).
As for MIRI's agenda, I don't buy that it will help with AGI safety at all. There are a variety of reasons for that, some of which are discussed in the piece I linked above.
An interesting read, but the conclusion that "the singularity is nowhere near" was reached by assuming that only neural modeling could get us there, and that assumption wasn't defended well. (In fact it looks rather dubious, given all the quasi-intelligent things computers have achieved without copying neural dynamics.)
The biggest deficiency in AI is that we still don't have artificial systems which simulate human thought with any fidelity. Sooner or later that's bound to become a focus of attention.
The presentation briefly mentioned simulating the brain, but I think what's more likely to succeed is mimicking the mind at a high level of abstraction (i.e. a level we can study with introspective or even linguistic methods rather than neuroscience). There's some precedent for this with projects like Soar and ACT-R (and even some recent interest from mathematicians [1]).
IMHO this kind of methodology could be pushed much further.
> did attempts to generalize some of that research fail because the approach was fundamentally flawed, or was it because such efforts themselves weren't as good as initial projects?
In many cases there haven't _been_ attempts to follow up on that early work. It's more like the zeitgeist just shifted to other things. For example, SHRDLU was just a couple years before the first AI winter, and when spring came (~1980) people had largely moved on. (Which isn't to say there weren't also flaws with the approach.)