ifsidew.gif NYC jaron1.gif (5k) Jaron Lanier
March, 1994

A pioneer in the field of virtual reality, Jaron Lanier thinks deeply on subjects related to computers and people. He is also a musician. This talk and performance took place at NYU's Interactive Telecommunications Program.

Musical instruments are, for me, a critically important human artifact. I think of them as the most eloquent machines that have ever been created. And I think of them as giving us a unique record of cultures, in that they are devices that are molded by the somatic aspect of cultures -- they are devices that mold against the bodies of people who are no longer with us, from thousands of years ago and continents away from ours.

And so when we play musical instruments, I think we can enter this sort of somatic gestalt of another people.... Musical instruments are, for me, much more like masks than masks are, even.... Working with masks is very interesting because, as people we sort of have an illusion of a coherent, singular personality, with a continuous stream of memory and so forth, and consciousness. And actually, that's not how we are.

That sense of unity is something that's reinforced by our social fabric and our culture, but in fact we have the capacity to be many personalities at once, and the masks bring that out. Which is fascinating. And by using a mask, it's done in a safe way. It's like a little, personal proscenium arch around your head. But even moreso, I think musical instruments turn you into a difference creature. When you play a musical instrument, you essentially have an aural version of a mask, and you take on a different voice, and become something very different -- something of a different consciousness, a different experience...

This, of course, brings up a very important connection to virtual reality, because VR is a masked world. In all the most important ways, VR is exactly like everyday reality here. It's us, the same souls here, communicating. Everything that's changed in VR is the superficial stuff -- just the particular forms we are. But by making the superficial things fluid, you discover what is constant, what is deeper....

The experience of improvising with musical instruments is the closest experience that I have been able to have access to, to what I hope that post-symbolic communication would be like -- the spontaneous creation of a shared world with other people, as fast as you can think and feel....

Now, of course, the difference between improvised music and post-symbolic communication is that music is all form, no content -- it's pure form, there aren't any particular things in it, recognizable things....

The hardest problem to be solved in VR, both from a technical point of view, and a cultural point of view, is the question of how we'll make up the worlds of the future -- how we'll make up the actual things that are there in virtual reality. And the approach of standard programming is definitely not tenable. The complexity of virtual worlds becomes too great too quickly. So normal programming is not possible What I desperately want -- what I believe is the reason VR is worth doing -- can only happen if we get this thing, which is the ability to make up the content of virtual worlds very quickly, at the rate you make up English....

This sort of improvisatory experience of just making up the content of a shared world as you go, as you're there, so that you can have conversations with other people that consist of creating the world directly, instead of talking about it. Right? That's what post-symbolic communication is -- cutting out the middleman of words, and just directly making the world, as a form of communication -- what we all wish we could do.

So, the question is, what will programming be like?... What I believe is that anything that uses formulas and secret codes and syntaxes of various sorts, is just too detail-minded to really function, in the way that programming has to be in the future, for this kind of improvisation. It's going to speed programming.

I believe the body is able to know much more than the mind, in terms of instantaneous action. And there will be some very somatic form of programming in the future. And I look to musical instruments as an inspiration as to what that will be like....

So, essentially, I think the way we look at in musical instruments today is what programming languages will look like tomorrow. Not the music, but the instruments....

So what I'm going to talk about tonight -- I'm going to play an instruments, just because I love playing instruments. And I'm going to go into a number of specifics about building instruments in VR, looking at real instruments -- just considering them as artifacts, as designs, as possible tools for computers in the future. All of these things....

Let me start by playing a little music.... This is one of my favorite instruments... let me get it going, then I'll play it, then I'll tell you what it is.

khen.gif (11k)

Let's consider this. Well, it's from Laos, it's called a khaen. I'll tell you a little bit about this instrument. This particular one has been picked apart a little bit, right there, by the customs police of the San Francisco International Airport. They were convinced that it was a vehicle for hash or some other substance that we're not able to manage ourselves. It's an instrument of very elusive physical principles. It's pretty easy to understand sort of how it works, but to understand exactly how it works is very difficult indeed, and in fact, I once spent an afternoon with Richard Feynman, the physicist, trying to figure it out, and he couldn't quite, which means that nobody really can.

It's a very, very simple device. When you breathe in and out there's no sound, and you put a finger on a hole there is a sound. There's a little free-floating reed inside each tube. The reeds are made according to an extraordinary recipe which consists of old, old Chinese coins that are arranged in a certain order, then pounded down, so you have a multi-layered structure of metals that vibrates assymetrically.

These things go way, way back. The oldest ones are the Chinese ones, which are 3,000 to 4,000 years old at the least, we're not quite sure, but there are records going back at least 3,000, and these are probably as old as that, these from Southeast Asia.

In contemplating this, there is a great deal to say. The first thing is that, for me, the experience of sensitivity and sensuality and meaning of playing this instrument far exceeds any experience I've had with a computer - far exceeds it, at this stage. I have many speculations on why that might be.

I have a feeling that part of the problem is that when we interact with computers, we're not really interacting with chunks of silicon or anything, what we're interacting with is the ideas we program the computers with. And so, what the real difference is between this and a computer is not so much that one is made of Chinese coins and teak and cane, and the other one is made of silicon, but rather, that one is made of the material earth and the other is made of ideas.

If you consider the relationship of ideas to practice in music, we've seen a real shift in recent years, with the advent of computers. I'm using music here because that's what this is about, but obviously, you can translate this to visual arts.

There's a very curious thing about Western music, where, starting a little bit before the Renaissance, there was a process in which Western music seemed to be preparing itself for computers. We started to have a notion of notation not being just a little helper device for performance, but to really become a primary representation of music. And the music changed in order to adapt to being notatable. And this is a very, very curious thing, this sort of anticipation of the computer by hundreds of years.

Most musical traditions in the world have some form of notation, but only the West treats the notation as being as important as we do. For instance, in the Indian musical tradition, there is notation and has been for a long time, but the saying is that the music exists between the notes, not in the notes themselves. Which is to say that all the little inflections, all the musicality, all the stuff that is not notatable, is really the music, and the other stuff is sort of a carrier for it -- the bread slices for the sandwich.

In Western music, therefore, we've created a bizarre historical illusion of the notation being the music, and we think we know the music of our past, whereas other cultures of the world simply know that they don't know the music of the past.

This is a big difference, of course, between music and art, in that music vanishes with the wind, before recording, and paintings sort of stick around if they're preserved.

This has led to some really strange anomalies. For instance, our ideas about what our music was like, say in the Medieval period, or the Renaissance, are just bizarre. I mean, we have the Notre Dame Mass, a famous piece by Jean de Machaut, comes off sounding in performance today as a very weird, complicated, angular, bizarre, rhythmic thing. And very clearly, it's almost unperformable. It couldn't have been that way, and obviously the notation was just striving for something that we can't imagine. It must have been something about inflections of the voice of some sort. But it couldn't be the literal reading that we have.

Another thing I'll share with you is that if you look up ear-witness accounts of Bach from his day, the music is not described as being an abstract, sort of stair-steppy music that we think of Bach, but it's described as a music that's filled with the sounds of cries and laughter. It's described as incredibly fluid, almost luridly fluid, very, very emotional. And he would play to audiences - it sounds more like Italian opera, or Bob Dylan or something, than the way we think of Bach's music now being some abstract thing. So we have this excess of abstraction in retrospect, because of our belief in notation.

What's reallly interesting is that with the advent of computer tools for making music, now instead of these abstractions coming after the fact of making music, they come before it. In other words, when you make music, or art, with computers, you have build the thing out of editors, and the editors have to have some concept of what you think music is made of, such as notes, built into them. Sequencers are built on notes, MIDI is built on notes, therefore notes actually exist. Note's aren't merely interpretations after the fact, they are the building blocks. So for the first time in history, computers have made notes real. They never were before.

That's not necessarily horrible. I mean, it's not like, evil or anything. But, it's tricky, it's dangerous. The danger is to lose track of the fundamental mysteriousness that surrounds us, by working within our own concepts as starting material. You see, that's the danger. It's forgetting the mystery. I believe that this is one of the reasons that there is this sort of blandness and nerdy feeling that's associated with computers in general. It's because of this priority given to abstraction, by necessity. People tend to forget the mystery.

Let me talk about mystery a little bit here. In one of my lives, I'm a scientist, and in the discipline of science, mystery is the first premise. The notion is that we're surrounded by a sea of mystery. We don't know for sure anything about it. Any theory that isn't falsifiable, that you aren't sort of waiting to be disproved is not a scientific theory. Everything in science is temporary. It might last for a while, but it's ultimately temporary. In fact, all the science we know now is based on theories that we know are wrong -- there isn't a unified physical theory. So we know that our ideas about electricity and gravity and so forth are wrong, because they don't connect together. So we have contingent theories. Science respects mystery.

Now, in working with physical materials, you're immediately reminded of the mystery, because these things are so subtle, there's always more. And I don't know how to describe what that "more" is. Sometimes when I say this, people say, "well, tell me, what did you discover this time?" I can't put it into words, and that's the point. These are things beyond analysis. There's always more - there's always more sensitivity, it's a bottomless depth, this little piece of the physical world here [the khaen]. It reminds me of the mystery every time I play it. Because the physical world is utterly mysterious.

The thing that amazes me is that people look to, I don't know, flying saucers, whatever, to find bizarre, mysterious things. But just this stuff right in front of us - the fact that we're all sharing this weird physical place, that it is the way it is, that we can communicate through it to eachother. I mean, this is so weird to me. The more you think about it, the weirder it gets. It's just extremely incomprehensible.

I'm going to relate an ancecdote from a recent dinner I had with a well-known computer industry figure. During the dinner, I was having an argument with this fellow, around the topics such as I have just been describing. This sort of talk does not go over well in Silicon Valley. The fellow said, "Well, take a violin, for instance. I mean, a violin has a crummy user interface. It's so hard to play. With computers we can build a better user interface for the violin, and then kids will be able to play it, everyone will be happy..."

What's wrong with this picture? It's subtle to say what's wrong, actually. It's very hard to give a slam-dunk criticism of it, because there's a coherent worldview which shows it being perfectly rational. I mean, if you like violin music, then make it easier to play the violin. If what you can do is only 99.9% a correct simulation of a violin, if there's a tiny bit of mystery missing, so what? That's a coherent worldview, and it makes sense on its own terms. The thing it leaves out is a very scary thing indeed, which is essentially mysticism. That's what's so difficult about these arguments, because you have to become a full-blown, wild-eyed, nutty mystic to really be rational about these things.

In order to consider why you wouldn't just want to design a better user interface for a violin, you have to consider why you play the violin. This is a very interesting exercise. Whenever you're doing anything, really, not just playing a violin, but any old thing, the question "Why do you do this?" inevitably leads to another question, "Well, because I'm doing this other thing, because I like the sound." Well, why do you care if you like sound? Why do you want to be happy? Why, why, why? And when you ultimately get to the end of your chain of justifications upon justifications, you end up, where? What is the ultimate thing? That's when you hit mysticism, in my opinion. It's the reason that has no other reason. It's just this energy that keeps us going through this nutty adventure of ours.

That's where it becomes so tricky. Playing the violin is actually that mystery, it's the end of the line, it's not for any other reason. And that makes it rather sacred. Things with reasons are dispensible, because there's always some other way to achieve the same goal. Things without reasons are indispensible and sacred. Those are the things that really constitute the reason for living. They're the only thing that's worthwhile.

I was once on a tour of some science facility, I think it might have been an accelerator, this was a long time ago. There were some senators coming through it, and one of them said, "What does this thing contribute to the national defense?" That's the usual way you scam money in science. They said, "Well, it makes life worth living, so that we care about having a nation." In terms of the violin, it's very much the same thing.

I find it so hard to talk about this, because I also find language, in a sense, to be antithetical to the ideas I care most about. Of course, that's not surprising for me, since I'm very much involved in this dream of post-symbolic communication. With language, I'm forced to create a little world of false reasons just to form the next sentence. For instance, if I say something like, "How can we make computers more like this?" A sentence like that has, already built into it, so many false premises about human motivation and all. It's funny. Language forces this sort of constant illusion that things have more reason than they do, because you have to fill in all the references in your sentence, and it takes a little bit of spontaneity and direct sense of joy out of things.

One of the things I'm convinced of it that the interface to computers is far too abstract, and not nearly physical enough. The interface to computers must be more like musical instruments, or like dancing, or like actually smearing paint on a canvas for real. When it becomes more that way, we're going to discover a kind of body knowledge of dealing with these abstractions, that turns them into something entirely different....That's the magic thing that you don't get to experience with computers very much. Computers, in their current generation, require us to know what we're doing, and we're not capable of knowing very much so it limits what we can do. People are actually not very good at understanding themselves, and it's certainly impossible for anyone to explain how they play a musical instrument. But it is possible to explain what you did in Photoshop, say, which is pathetic.

Let me tell you about some of the other projects I have going now. I'm working toward the notion of playing a musical instrument being a form of programming. And I'm still many layers from achieving that goal, I'm just approaching it step by step, getting closer each time. The current project I'm working on is a way to play a piano -- this is an acoustic piano - and have cathedrals come out, that are designed according to how I improvised, that I can then go inside, and play more music, and have it all happen spontaneously, and seamlessly.

It's worth talking about the piano a little bit. I'm quite a piano obsessive, and there are a number of reasons for that. The piano fits into a unique niche in the sort of mysterious world of acoustic instruments, versus the idea-driven world of computers. It fits right in between them. I don't know if there's anything else that fits quite in between like that, in the visual world.

Now the reason for that, if you talk to engineers who have studied a piano, what they will tell you is that, "Hey, we've studied pianos, put sensors all over them, we've put listeners through double-blind tests in which they listen to piano sounds, and what we can tell you is that, all a piano can do is control the volume of the note, and the time it happens, and when it goes off. And that's all a piano is."

Well, I'm sorry, I'm a pianist, that's not all a piano is. But I can't see what the difference is exactly. I mean, I can see it from both sides. But the piano somehow mysteriously transcends button box-hood. It transcends its digitalness, and it becomes something else. And it's not any piano. Some pianos do and some pianos don't, which really makes it strange, but pianos do have a sense of tone, and you really can change the tone. You can't do it on a single note, but somehow, when you play music on it, you can.

This is mysterious, something beyond the reach of science at this time. So it's absolutely fascinating. It gives the piano a very romantic quality, because there's this sense that somehow you have to reach through it, through this initial barrier of these digital switches, and then you reach this other thing, which is this living piece of the earth, this piece of the world. And that distance gives it the romantic feeling, I think, that it's known for.

I'm fascinated with the piano because whatever the piano does is exactly the thing that computers need to do. I don't know what that thing is, but there we have a living example of a button box that has transcended it.

So I'm working on these algorithms. I don't have them quite working yet, but they're beginning to work. In general, the approach I'm taking is using some of the new mathematics of complexity, where you can deal with large patterns as a whole with simplified explanations, this whole world of fractals and so forth called complexity theory, that you might be familiar with, which is really quite interesting.

Mathematics just took a big leap in the last ten years or so, where it gave up its determination to have a literal control of every point in its domain, and instead it's looking at patterns as a whole. And it turns out to have some powerful ways of looking at these patterns. I'm using that sort of thing to try to look at these piano patterns, and relate them to architectural patterns.

In virtual reality, you can become a gazelle or a lobster, or other creatures. One of the most surprising results in the early development of virtual reality was discovering how readily almost all subjects can do that - that you can give somebody a completely different body, and they just seem to pick up on it.

So what happens when you play cathedrals out of a piano is essentially that your body is turning into cathedrals. You're having a sensor motor loop in which architecture is the output instead of, say, hand motions. When you can create forms directly, improvisationally, the things you can do with your body, the forms are essentially the part of your body that your eye sees, instead of hands. And you start to develop a body knowledge of the system that's creating those forms. When you achieve that, you can start to achieve a body intuition, and really get good at something. I'm hoping to eventually, after I've been able to do cathedrals, to have programs come out of that. That's the development path, as we say in the trade.

That's a natural stopping point, so let's take some questions.

Q: You see this incredible elegance in musical instruments, and I wholeheartedly agree with you, I see that too - there's something mystical and magical about them. But if we already have something that's mystical and magical, what is it that a computer cando that musical instruments can't?

A: Why bother with all this if we already have cool instruments? There are two reasons. First of all, the only justification for such things is cultural -- there's no objective need for it. But just because there's no objective need for something is no reason not to do it. I consider our particular culture to be technology-obsessed, on the whole, and at this point that obsession is rather irrational, since most of the original goals of technology have been met. With the exception of medicine and natural disasters, all the problems that we attempt to solve are actually created by our own behavior. So any technology related to anything other than those two things is done out of love, or for art, or for aesthetics, or out of momentum, or something. But it's not done for any rational reason.

I think that technology is sort of a talisman for our particular civilization. In another culture and another time, there might be very little potency to it, and there would be very little energy behind this sort of idea at all. In our particular culture, I think this is our talisman, and there's a sort of a cultural duty, or a kind of adventure that has to run its course. Or I might say a kind of karma, maybe, that we have to turn this stuff around, and turn it into a spiritual and meaningful thing. That's sort of the apparent task before us, that what there's energy for. There's a big hole in that the thing we're in love with is making us into bland nerds, so the obvious completion of the task that there's immediate motivation for in the situation at hand is to overcome that.

But then there's something else. There is something that technology can do that nothing else can do, which is to create post-symbolic communication. That is one thing that you simply cannot do any other way. There are arguments that you can take the funny Yaqui brew from the Amazon valley and have shared hallucinations with other people under conscious control. Perhaps, but in terms of the world that our general cultural gestalt accepts as being real, virtual reality is the only way I'm aware of to have that experience, of creating a world with other people, as a way of communicating. Which resolves the misery of childhood into something tolerable. So that's the reason we have to do it.

But why do computers have to come in between it? Why can't we just make this great and fabulous network, where you can play music with people all over the world?

Well, OK. Turn into a giant squid before my eyes. See, the moment you can do that, you don't need virtual reality.

You're saying that a programming language should be like a musical instrument. Yet, I feel that, to play a musical instrument well takes as much dedication and practice and discipline as programming. So in a way, you're making [programming] harder.

Absolutely. That's why I say that, to deal with this stuff, you have to become sort of a crazy mystic. Because most human activity, in fact all human activity, is basically irrational. Religions invent little games to explain what we're doing and why we keep going, but, come on, it's irrational. That's the mystical viewpoint, that there's this meaningfulness and joy in and of itself, that it all comes together. That this irrational sense of life and having these adventures is in itself the bedrock of reality, on every level. That's essentially the mystical perspective, and I think you have to have that perspective to make sense of any of this.

And as I say, there is nothing but cultural relativity involved here. On the one hand, we have tremendous flexibility, I think, to be many, many different things, in many different ways. We were talking about mass drama before. Or you can play with hypnosis. You can play with many things, explore many personalities that are unusual, and all of that. But at the same time, there is a core reality to what we are, which is our culture, and our bodies. And you can't just suddenly imagine that those things aren't real. They're so real that they are everything to you. And I just think that technology does play a particular talisman-like role in our particular culture. And that's why there's so much energy around it, and that's why it's interesting to me to try to find a more interesting thing to do with it than the usual thing, which is to try to make us more powerful versus nature, which is getting really old. And really stupid. I think that pretending to some sort of objective justification just confuses you, ultimately. It's all done for love.

You were saying earlier that music is all form and no content, and I don't necessarily agree with that.

Yeah, well, actually, I don't quite agree with that either, but they're the best words I can find. And this is a really problem with -- well, let me try to describe what I mean, but some of the things I want to say are genuinely hard to say with words -- it's very hard to get it just right. But sometimes getting a quick approximation at least communicates something.

One of the amazing things about words is how much they let you lie. You can communicate a little in the course of a lot of confusion and at least a little bit happens, its sort of -- it's a remarkable property, really.

So in this case, what I'm saying is that when you improvise music with other people, first of all, you're not making houses and fish, and battles and polo games, and you're not making anything replicable. There's nothing you make in music that can be considered equivalent to anything else. There are some abstractions you can make in music, such as playing a fragment of a tune, or playing a scale, or a chord or something, that can be classified with other things, but the actual product that you make cannot be considered equivalent to anything else. Now that's maybe a slightly more precise thing to say, but it still doesn't quite get it.

What I'm trying to get at is exactly the difference between improvised music and post-symbolic communication. And there's a big difference, because with post-symbolic communication, you can create the entire content of a shared, objective world with someone else, at the speed you think and feel so that you can become a squid and so forth.

Now, what's the real difference between that and improvised music? It's not only that one has images and the other one only has sound. What it really is, is that you can create equivalent forms at will, whereas, in the other case, you can only create forms that are not equivalent to anything, I think. But it's very hard. It's subtle. There is an important distinction there -- there's a critically important, fantastically important distinction there, but to articulate it is very, very hard.

Are you trying to say that in post symbolic communication that you actually build something that has duration beyond the act of creating it? Because I think that you're legitimately creating something, especially if you're explaining to people, you know, musically who understand, that you can communicate at some level.

Mmmmm.... No.... Having the thing have duration is not at all what I'm talking about. In post-symbolic communication, you're actually making up the whole world, which is, you know, yuppie water, and cabs, and not just objects, but events - traffic as well as cabs. You're making up the world, and the world that you make up is made of specific replicable objects, things that people can recognize. Things that have meaning for people, and things that have meaning when placed together.

The building blocks of music have no meaning by themselves. That's one of the interesting things about music, actually, because a note doesn't have meaning, and a group of notes doesn't necessarily have meaning, but a piece of music has meaning in some form, but when did that happen? When did that meaning come into place? That's the magic of music. There's just this meaning that happens. But that meaning is not a result of any parts that are meaningful or recognizable, it's something that emerges out of completely simple and meaningless events.

[I think] that the deepest parts of music, which are the breadth and the musicality of things - these are things which really have no sense of reference at all; they're just there, or they're just forms. You can make theories about how there's some kind of reference there; for instance, you can make a theory that when you listen to somebody playing music, your body experiences vicariously what it would be like to play that music ... therefore you kind of take on the body-music of the other person, and therefore you're kind of in their groove and you can experience the world thru their eyes. I think there's some kind of truth to that.

And yet... see, what post-symbolic communication does is that it has that experience of improvising music, plus all of the sorts of meaning that language attempts to refer to, such as referring to specific objects, ideas, and processes as something else that can also be created.

Any of you who have heard me speak when I'm in a large audience know that I have snappy phrases that sort of approximate these, but I'm sort of being more honest with you now. I'm saying that I can't quite articulate these things, they're genuinely hard ones, and that's the reason why. But there's something there. I mean, you can help me with it. Think about it.

When you're talking about the building cathedrals out of the music, it almost sounded like watching Jackson Pollock, live. You're Jackson Pollock, and we're watching you throw the paint all over the room - the entire room exists and we're all able to watch through virtual reality. And then the next step would be to give us all paint, and we all get to throw paint.

Well, except that you want to throw planets, houses, and tarantulas, and so forth, I mean the point of it is to have unlimited ability to create all recognizable things.

But we're talking about more] simple stuff...

No, don't give me that old crap. No we're not. Let me just say something about this, and then I'll demolish that comment.

jaron2.gif (11k) Seventeen years ago, or something like that, when I was living here in my teens, one of the things I was obsessed with was taking timelapse of paintings being painted and setting that to music. I do think that there's this kind of sense in which I think that we're always denied all these wonderful flows that we could be apart of, all around us.

I talked a lot last time I was here about what I feel is the frustration that children feel in early childhood of having to accept a world that is much more stubborn than the world of their imagination in order to reach other people and to reach their food, and all that. I think that we forget just how frustrated we are, that there are all these hidden grooves, these hidden flows, that we could be apart of. And these grooves and these flows are the ultimate things that could be meaning, I believe.

This sound like a way to have a cross between art and entertainment at the same time, where we're watching you build cathedrals, and houses, and planets and all that, and perhaps even, like you said, [we can throw up ourselves?]....

Oh yeah, I think It's critical, I mean, I think that's the one thing that I think is most important question of the future of virtual reality development, which is: Will people make their own worlds or not? And one of my projects now, that I'm hoping will pan out, is a virtual reality theater that will be community driven, where people can drop in and make parts of the world, rather than having a design team create it. I think that's just absolutely the most important thing.

Essentially, I've used symbols as a trick that we use where you can use a small part of the world that you're able to control as about as fast as you can think and feel -- which is your mouth and your hands -- to refer to all the things that you can't create as quickly. For instance, you can't suddenly become a squid. To become a squid would take centuries of genetic engineering and trillions of dollars. But you can say you're a squid, and you're a squid.

Symbols are this kind of quick hack that you can use to refer to all the contingencies that you can't actually realize because you're not powerful enough. The interesting thing about post-symbolic communication is that you can actually realize them in the first new objective thing since the physical world, which is virtual reality. You can actually make them, if these interfaces are good enough for design, which, I think, might look a lot like musical instruments. And in that case you're actually directly creating the world rather than referring to it and therefore cutting out the middle man of the symbol, so these things are emphatically not symbols. Simply because you can recognize something doesn't make it a symbol.

Symbols are, after the fact, middle men, little abstract entrepreneurs who suck our blood. They're like these leeches -- I mean, they're not real life. They're given this primacy by thinkers because thinkers use symbols, and so you have someone like Umberto Eco who says that all we have is words, but that's not true at all. Words came later.

So if we experienced being the squid itself, rather than the symbol of the squid...

Well, being the squid of course is far beyond the symbol of the squid, but when you see a squid, you see a squid. The basic existence proof that symbols are not primary is to learn lucid dreaming. You can go out and buy a book called Lucid Dreaming [by Stephen Laberge]; if you study it, you'll learn a technique where you can be awake inside a dream and control the content of that dream.

Now, the only difference between that and post-symbolic communication is that there isn't anyone else there, so it's solipsistic, it doesn't really do anything. But then you'll discover that you can have the direct experience of creating anything that you want without having to name it. And then you'll discover that symbols are not primary.

But you will have to name it. I mean, it may not be...

You only have to name it to tell us about it. Believe me, I've made things in dreams that you could not name. Trust me. Right now, you have to trust me, but when we have post-symbolic communication you won't have to, because then I'll do it in front of your eyes.

You may be interested in Carlos Castaneda's latest book. It also involves - so did his past books -- but his latest, The Art of Dreaming, also involves the exact same lucid dreaming.

Right. Yeah, there's actually a lot of good lucid dreaming books around. You might want to pick up Amazon Dreaming too. It's about the Amazonian dreaming tradition, where they use a - it's a long story. But anyway, I would really recommend Steven Laberge's book, Lucid Dreaming as a starting point because it's got a practical sort of self-help approach so that you can do it yourself and experience it, which is the most important thing.

I'm wondering what you think about the act of ritual, the function of ritual, which I think, could be a kind of bridge between symbolic and post symbolic... and I guess more traditional attempts of a group of people to act in this realm the role that has been played by ritual - if that this, in your opinion, has been replaced by interactions with new technology?

Well, what I think ritual is, is that there's a distinction between the mundane and the inspired and there's no objective, definite baseline for inspiration - everything's relative because nothing really matters, we make all this up. Right? So this is all just done out of love; it's out of a vacuum.

There's a necessary distinction between inspiration and mundanity, so I think everything is ritual, everything is hypnosis. We tend to go thru life thinking that this is all sort of normal, like just being in this room, but it's really not. I mean, on many levels this is just a socially reinforced, very curious illusion. But it just happens to be - perhaps if my lecture is good, then there's just a little bit of the inspirational side, but in general, it's just on the mundane side of things, where we're all just sort of here.

I think that what happens, in ritual and in many other experiences is that there's a different, collectively created gestalt. These are very mysterious phenomena, I mean, what I can say from my scientist personality, all I can say is that science can't touch this stuff. I don't know how to talk about this as a scientist, all I can do is talk about it as somebody who has experienced it. But people collectively create the world that they live in somehow and there are these punctuated moments where things are more intense, more inspired, more magical. There have to be, because if things were like that all the time, then it would never be.

I think ritual is a time when people collectively create this alternate experience gestalt that is more magical. That's an almost too mundane way to put it, but it's sort of precise, too. Because, as I say, if you were doing ritual all the time, then you never would be; you have to have the mundanity to have the special times, because that's the only thing that creates them.

How do you see the act of ritual with respect to your work?

Well, I have a feeling that what will happen in the future, if virtual reality is used in anything like the way that I imagine it will be is that it would be used mostly in a mundane mode, and occasionally in a ritual mode, just like the physical world. It would sort of show up both ways. The mundane world will just guarantee it, in fact.

I think a lot of us are familiar with the visual picture of fractals now. Do you think that music is a sort of aural fractal? Is possibly the pleasure we take out of it a subconscious recognition of the patterns that cause its fractility, or fractality, or whatever.

Fractality... that sounds like some sort of alien race on Star Trek, or something. Well, once again, this gets back to something I was saying before, that music is just music, and so that using fractals to understand it is just an abstract interpretation, just like notes. It's very very important to understand that. I don't think that you can understand music as information. I don't think you can look at a piece and analyze it and know anything about it . You have to understand it as a system with listeners, with a history and all. It's a human phenomenon -- human's listen to it. If you play our music for rocks, it doesn't mean anything to them, right?

There are some things that you can study in isolation. For instance, if you want to study the corona of the sun, you can build a physical model of it, and predict things about it. That makes sense by itself. But you can't do that with music. Music as a document just doesn't stand on its own. I just don't think that any analysis of music as an information record makes any sense at all.

I think you have to have a very social or historical perspective to understand any music. Those are the approaches that would have some sort of meaning, and therefore they're sort of vague and require social references that cannot be completely defined.

I think that a mathematical technique similar to fractals can be very useful in making music, and, once again, there's the danger of working with abstraction as your primary material and becoming a nerd as a result. I think it's a great kind of math, and it's a lot of fun.

One of the things about fractals is that you can fit them on anything better than you can fit other curves on other things in general. In other words, if you take a square and try to fit it on things, it tends to fit on other square things. If you take a fractal, you can figure out a way to fit it on almost anything because fractals intrinsically have a rich variety of forms in them. So there's sort of an illusion created. There's a whole slew of papers that you can read, where somebody says, "Wow, we stuck fractals on this and it kind of worked." And like, well, whoopee - that's what a fractal is. Fractals are even used to compress pictures because you can find a fractal that matches a picture and use it to represent that part of the picture in a much more condensed way. I'm sure you could fit fractals on all kinds of music effectively, but... I don't know what you've done; I mean, you could get a doctorate that way, but who needs one of those?

It seems like you're after some sort of universal communication, yet to me, as an instrumentalist, I feel like the majority of people - the physical manipulation of the interface - might be your biggest challenge.

Well, first of all, let me say that I'm not seeking a sort of universal communication, necessarily, at all. In fact, there's a profound question there about how universal communication might be there in the future.

What we're seeing at the United States now -- there's been something very interesting. Mass media were critically important to creating a unified sense of nation. I mean, in the 20th century, there's just been an enormous influx of immigrants speaking different languages and coming from these different cultures. They've all grown up initially with radio, and then television, and there's this sort of simulation of a shared culture that becomes real as people grow up with it. In the new media that won't be true anymore. And considering the tendency towards vulcanization of everything, as Bill Gibson says, the tendency of the world to be breaking into little cultural and ideological enclaves. That, combined with new media, has some positive and some negative elements.

It certainly is possible that in the next century, in the "interactive century," we'll see much less in the way of mass culture shared. Already the networks have supported a certain amount of enclave dialects of English. Or, like the FurryMUCK enclave, or the MUD enclaves, these are people with separate vocabularies, who are really separated at great distances.

In California, there's a town called Boonesville, that's kind of remote, and has its own language, a variant of English that's not comprehensible. It's a weird dialect, and there are still some old timers that still speak it. That can happen geographically, but now it can happen on the network. I'm not necessarily talking about something universal. I think it's actually wonderful to have little cultural enclaves, as long as they don't kill eachother.

There's this culture in computer design assuming that the purpose of technology is to make everything easier to do, and that's a good summary of the way a lot of people think of computers - they make life easier. It's easier to write your music, it's easier to write your book, it's easier to do this or that. But the problem you run into is this mystical problem, which is, well, why are you doing this thing? What's the ultimate purpose? And the problem is that some of the hardness that you might have been getting rid of might have been the purpose in the first place.

Especially when you think about orchestral players, when you think about, say, a violinist. Oftentimes, what stands in the way of their inspiration is the mastery of physical technique. And a lot of times you find people who don't have the mastery of the physical technique, who are brimming with music. Other times you have people who are brimming with technique because it is by rote, because they have mastered their instrument, as opposed to having a mastery of music. I would think that having to master a physical input of any sort lends itself to these same sorts of problems.

I think there's a bit of an illusion at work here, of this sort of disembodied creativity - this notion that you're just brimming with music, but you just haven't realized it. Let me compare this for a second to dreaming. How many of you have had the experience of hearing new music in a dream that you've never heard in real life? That's a great experience, right? Now, the question is, can you wake up and write it down or make it? The answer is, it's pretty hard, right?

What happens inside your brain is that, and I can go into this in some technical detail, because I think that I can make some pretty good arguments for this based on experimental work. I think your brain is capable of creating an experience without having constructed the stimuli for that experience, because it's got both the beginning and the end of perception, in this one little mushy thing up here. There's a certain illusion of creation that can take place in dreams, and why not? How would you if you've actually constructed the thing that you're hearing or not?

I believe that music, for one example, is not sort of a pristine, abstract thing that one simply composes, and that it has meaning in itself, in some sort of vacant platonic space, where it just rises like a cathedral. But rather, I think that music happens through the graceful pushing against a boundary. I think that, in fact, most good music is written by composers who are really, at that moment, stretching to learn a new technique, or to overcome something. Most good music happens in learning. It's very rare that good music happens after mastery.

Let me support that a little bit. I'll use an example from popular music, like rock music. You have this phenomenon where there was a period of intense creativity, where all this stuff was new. When the Beatles were doing their music, for instance. All of these sound effects, and backwards sounds, and reversed voices and all this stuff, was brand new, had never been done before. And in that context they did some superlative stuff.

And once it's been done, even they themselves can't quite do it anymore. The music actually happens in the pushing, it doesn't happen in retrospect, after you've pushed. So I think that the difficulty of learning to play instruments is actually an asset, in general, even though it's very frustrating. I don't think that real creativity is easy, nor do I think a worthwhile life is easy.

What I think is sort of interesting about VR is the possibility of perhaps eliminating the performer as a middleman, where even though you have an improvisation, you have a composer who composes a space, a world, things in which an observer, an explorer, can go through that and experience that form of communication that the composer is trying to put out, whereas what we have in music, ofttimes, is a composer putting notes on a sheet of paper, the performer exploring the possibilities as put forth by the composer, and a passive audience sitting there and applauding at the end, instead of actively exploring it themselves.

A: I've noticed that problem too. And a lot of times the audience isn't even sincere in their applause. It's just pathetic. But what I want to stress is that, I love the idea of more interactivity, and people being involved. What I think is specious is this notion of easiness being positive, because easiness is just another way of saying meaninglessness.

You're pushing a lot against language this evening. You acknowledge a kind of constriction or problem in -

I'm anti-language, yes. No I'm not really. You can see, I'm using it at this moment. Language and I have a sort of love-hate relationship.

But I think we all push against language, we all push against symbols, because they're insufficient, so we're in this constant push and pull to try and get somewhere... You're post-symbolic language environment, one in which you can say, "Name a squid, make a squid," seems to have an ease to it which you're in fact against, in terms of wanting to be pushing against some sort of a discipline, and envelope, a problem.

It would seem to me that, with notation, if you have the note first and then the music, it's a problem. If you have the word "squid" and then the squid, then it would be a problem, it would seem to be the other way around. I mean, you'd want to be pushing in that sense, against some sort of a discipline, a tradition, a problem.

Well, I'm mostly talking about music, but when I'm talking about post-symbolic communication, what I say is that this is pretty far in the future, because what I see having to happen is not only the creation - I mean, have you ever programmed a computer? It's really hard to make large programs that are good, it's actually very, very hard work.

What I usually say is that I see this happening pretty far in the future, because not only do we have to come up with user interfaces that make it easy to do, and by easy I don't mean easy easy, like we were talking about. I just mean possible, I just mean tenable. Not only that, but a generation of kids has to grow up doing it, with eachother, so that a culture can be born. The real development is all cultural, not technological.

I actually completely agree with everything you just said, I think it's exactly right. In the particular case of being able to spontaneously program to content of a virtual world, right now it's got to get a lot easier than it is, because it's not even remotely tenable. So in that particular case, easiness is just a priority.

As I say, music happens when you push against a boundary, and some boundaries are better than other boundaries. There are boundaries of great beauty, and there are boundaries that are silly, or just too difficult. So, for instance, let's take the piano, one of my favorite boundaries to push against. It went through a centuries-long development period in which it was being refined, and we have this magical thing that these amazing experiences can come out of. We don't know why; it's a collective product of many people. Something like that would have to happen to these tools, these instruments for programming, as well. I think the thing that has to happen is the design of these instruments has to evolve so it becomes a boundary to push against. Exactly how hard or easy it might be is hard to predict. But I think it has to at least be tenable in order to be worthwhile. But I think the notion of making it ever easier, as a principle itself is also wrong.

But the concept of what you seem to be saying is a dreamworld. The metaphor, the place that the inspiration comes from, the sort of un-verbal part of ourselves, the dream part, is the model for it. You refer a lot to lucid dreaming, to the sense that people can create things in their dreams that can't be named, and there's an interest in creating things that can't be named, that there's no verbal attachment to it, no symbols, no previous reference point. The public world we've already named, we already have reference points we have in common, and so on. Obviously there's a long tradition of leaving dreams individualized, leaving them private -

It's not just a tradition, there's no choice. That's like saying there's a long tradition of not flying to Jupiter.

I mean the Freudian tradition that the dream is important, that it should be shared. There's the Jungian tradition that this dream should get out there in one form or another -

But only the public part of it can be shared. That's what's sort of humorous to me about the psychoanalytic tradition is that the aspects of dreams that are shared are in many ways the least interesting to me.

But your assumption is that anything you put words to is public, and anything you can't is private.

No, that's not true. That's just the way I used the word "public" in the last sentence because it seemed like the best way to communicate quickly, at that moment. But that's just a problem with words, not my fault. I'm sorry, I'm giving you a hard time, please go ahead, finish your point, I won't interrupt you again.

It seems that things you can name, things that you can put words to -- you began the talk by saying you find it very constricting. Freudian psychoanalysis, for example, is all about using words and associations, people's feelings and visions and ideas and imagery and iconography. Since traditionally it's been done by naming it, as opposed to by drawing it, or by some other kind of imagining it, it's come out as a verbal tradition. But it does refer back to the private, not the public domain. And it seems like you have a sort of multimedia approach, in which you can have images or music, and not just words.

No, what I'm after is something a little bit different. The type of shared dreams I'm talking about are reallly anything like -- see, what Freud, and those who came after, are interested in are dreams that come out of the unconscious. So, for instance, a lucid dream would probably be of lesser interest to them. What I'm interested in is a method of communication that's very much as conscious an act as language is. I'm only using the word "dream" because a dream is the only thing we've experienced commonly in which imagination is not fettered by the constraints of the physical world we happen to have landed in here, in which anything is possible that we can perceive. And so, therefore, it's the only world really available to describe that state of affairs, without having to repeat the same sentence I just said over and over again.

So, with that, I'm going to conclude.

Kevin Walker
--> 'use strict';