YouKnowWhoTheFuckIAM

While I agree with you about the economics, I’m trying to point out that physical reality also has constraints other than economic, many of them unknown, some of them discovered in the process of development.

Bird’s flight isn’t magic, or unknowable, or non reproduceable.

No. But it is unreproducible if you already have arms with shoulders, elbows, hands, and five stubby fingers. Human and bird bodies are sufficiently different that there are no close approximations for humans which will reproduce flight for humans as it is found in birds.

If it was, we’d have no sense of awe at learning about it, studying it. Imagine if human like behavior of intelligence was completely unknowable. How would we go about teaching things? Communicating at all? Sharing our experiences?

To me, this is a series of non-sequiturs. It’s obvious that you can have awe for something without having a genuine understanding of it, but that’s beside the point. Similarly, the kind of knowledge required for humans to communicate with one another isn’t relevant - what we want to know is the kind of knowledge which goes into the physical task of making artificial humans. And you ride roughshod of one of the most interesting aspects of the human experience: human communication and mutual understanding is possible across vast gulfs of the unknown, which is itself rather beautiful.

But again I can’t work out what makes that particularly relevant. I think there’s a clue here though:

…but I also take care not to put humanity, or intelligence in a broad sense, in some special magical untouchable place, either.

Right, but this would be a common (and mistaken) move some people make which I’m not making, and which I have no desire to make. You’re replying here to people who affirm either an implicit or explicit dualism about human consciousness, and say that the answers to some questions are just out of reach forever. I’m not one of those people, and I’m referring specifically to the words I used to make the point that I made, namely that there exist real physical constraints repeatedly approached and arrived at in the history of technology which demonstrate that not every problem has an ideal solution (and I refer you back to my earlier point about aircraft to show how that cashes out in practice).

There are no known problems that can’t theoritically be solved, in a sort of pedantic “in a closed system information always converges” sort of way

Perhaps. The problem of human flight was “solved” by the development of large, unwieldy machines driven by (relatively speaking, cf. pigeons) highly inefficient propulsion systems which are very good at covering long distances, oceans, and rough terrain quickly - the aim was Daedalus and Icarus, but aerospace companies are fortunate that the flying machine turned out to have advantages in strictly commercial and military use. It’s completely undecided physically whether there is a solution to the problem of building human-like intelligence which does a comparable job to having sex, even with complete information about the workings of humans.

The Sequences are inherently short, there are just massively many of them - the fact that each one is woefully inadequate to its own aims is eclipsed by the size of the overall task.

The longer stuff, Siskind included, is precisely what you get from people with short attention spans who find it takes longer than that to justify the point that they want to make themselves. There’s no structure, no overarching thematic or compositional coherence to each piece, just the unfolding discovery that more points still need to be made. This makes it well-suited for limited readers who think their community’s style longform writing is special, but don’t trust it in authors who have worked on technique (literary technique is suspicious - splurging a first draft onto the internet marks the writer out as honest: rationalism is a 21st century romantic movement, not a scholastic one).

Besides which, the number of people who “read all of” any of these pieces is significantly lower than the number of people who did so.

The only thing any of us can do is choose how we are going to get dumber every day

I’ve just dipped in and out of it all day - I can’t look away! It’s better than a car crash: you can slow down multiple times

All i hear is that “polycule drama” is back on the table

There are too many comments in here going for the stringy lean detail and not pointing out magnificent conceptual errors like this

I’ve been saying this more often lately, but LessWrong gets its readers in, by and large, at the absolute bottom rung of intellectual thought, they don’t know anything else

You have to interpret somebody getting into LessWrong as just graduating from Cracked or Newgrounds in the mid-2000s

It’s a technique he uses to get you, the reader, to understand that you aren’t the person who thinks in terms of elan vital.

In one of his essays on quantum phenomena and personal identity he does it with time. He explains something like if you think time in the universe works in the sense of clock time, then you just don’t have a clue about physical reality, so when he gets to his next point it stands in contrast to the straw layman. But his readers are obviously already the sort of people who do know that, because they’re nominally smart, education-enthusiastic western(ised) nerds, even if they understand next to nothing about how this works out in real physical theory.

So the strawman doesn’t just create a favourable contrast for Yudkowsky’s argument, it constructs them as smart and different from lay people - it isn’t a one-shot effect, it builds as he starts small and piles on increasingly esoteric speculations (even if this is the first “mind = blown” blog post they’ve ever read from this weird guy).

I reply: Because the strength of the material is determined by its weakest link, not its strongest link. A structure of steel beams held together at the vertices by Scotch tape (and lacking other clever arrangements of mechanical advantage) has the strength of Scotch tape rather than the strength of steel.

This is sub-childishly false and he opens with it. Unbelievable.

“…trying to head off an argument by bringing their estimates down as low as possible” - you’ve got it. We’re done. You can stop now.

and seeing people talk about how they’ve never tried rice before having it at the restaurant

And this is the fundamental, absolute, no-further-down essential appeal of rationalism. Fans have never encountered ideas before, or not in any way that they’ve been able to digest them (“what the author really meant: the curtains are fucking blue”). And it bubbles up throughout, in every environ of the culture. If you are ever as stupid and masochistic as I am and find yourself on TheMotte.org, it’s the key to everything they say: they literally have never encountered facts from beyond their own rabid whirlpool of hatred, and everything that they learned in the whirlpool was laced with contrarian ressentiment towards an enemy they literally haven’t seen.