Design Talk

beyond desktop. create a Utopia world with new technologies and touching stories

多元新宇宙。

f( )_future virtual world + touching story =

Podcast. f( )_utopia。

Technology will change so rapidly and impact everything and everyone. Human-created technology is growing at an exponential pace. Gary Kasparov was scorning the computer in 1992, and just five years later, a computer program defeats him. It is a sure answer that machines will encompass us. The human brain has flaws. That is why technology has been become and will be more tremendously be our extended mind. With the increasing intelligence of future technology, technology can be embedded anywhere at any time.

This is where everything started, let me tell you a little history/fairytale. This is not a standard-textbook history - this is only my way seeing it. I always call beautiful history as a ‘fairytale’. Let’s get into it. 1968 – Douglas Engelbart demonstrated NLS, a system which uses a mouse, pointers, hypertext, and multiple windows. 1979 – Steve Jobs and other Apple engineers visit Xerox PARC. 1984 – Apple Macintosh popularizes the GUI. Super Bowl commercial shown twice, was the most expensive commercial ever made at that time. In the 1990s, Steve Mann developed a number of user-interface strategies using natural interaction with the real world as an alternative to a command-line interface (CLI) or graphical user interface (GUI). Mann referred to this work as "natural user interfaces", "Direct User Interfaces", and "metaphor-free computing". Examples of interfaces commonly referred to as NUI include: MS PixelSense, MS Kinect, 3D immersive touch.

Technolgy grows, always, faster than you think. Ray Kurzweil has mentioned an exponential growth trend in his book Sigularity is Near. He said that the technology capacity will grow faster than we can foresee in the next 100 years - that is a period of time that in our generation.

This is a famous quote from Blake J in 2011 that I alwways love to mention. Mordern human life is surrounded by all sorts of screens/user interfaces. We designers always adovate for natural/fluent UI. As Blake said: A natural user interface is a user interface designed to use natural human behavior for interacting directly with content. Another quote from a reseracher I love - Mark Weiser - the Father of Ubiquitous Computing & Calm Technology. Technologies is all around us, but as he said: The most profound technologies are those that can disappear.

Today we heard a fancy word coming up - metaverse. let’s do not talk about crypto currecies/bitcoins etc, here yet. It becomes a financial tool at some point for sure. I agree with fact that - like a lot of others saying - "It‘s a fascinating vision, but at the moment the “metaverse” is mainly used as a buzzword and right now we are seeing a struggle to fill it with meaning." So true fact is - if you can mentally change that name to something else - you will understand it better. My way of calling it is Future Vitrual World - that is on top of our current world. But where does this idea coming from? Of course every future is driven by either new technologies or gret artists.

This concept is not new at all. It is originally mnetioned as an embedded or embodied technoloy by many reserachers. The main intention is employing and leveraging our natural, bodily interaction abilities and embeding computing within the social and physical contexts in which we live and work. As I just mnetioned, there are two kinds: embedded or embodied technoloy. The embedded ambiance technolgy includes Ubiquitous Computing, Augmented Reality, and Location Based System. The embodied ambiance technolgy includes Vitrual Reality, Tangible Computing and Affective Computing.

We are already familiar with VR and AR devices that apply gesture control using depth cameras and related gesture-capture devices. Now we see a trend of gesture recognition systems that leverage wireless signals (Wi-Fi ) to enable whole-home sensing and recognition of human gestures. This type of hands-free gesture control system, together with Holo-Projection or Hologram, can be helpful to create a much more engaging and immersive experience. Hologram and AR enable us to go into the new Spatial User Interface world with new “reality”. Scientists have figured out how to manipulate tiny nearly unseen specks in the air and use them to produce images more realistic than most holograms. There is this technlogy called Projection Mapping - also called video mapping and spatial augmented reality - has been used in light Art design for years. What if we can bring that technology into everyone’s daily life? With projection mapping, we can make that dynamic light happen more vividly in our workspace, home, and anywhere else. Not only that, with the development of TUI - Tangible User Interface - that light effect can be changed while humans interact with it. And with special AR (augmented reality), we can make the computer-generated light directly to the human's environment.

Normally a gesture recognition unit including at least three dimensional gesture recognition sensors, a range finder, a depth camera, and a rear projection system. So all those extra stuff - VR headsets and cameras, etc - are very clunky, and most people experience motion sickness or physical pain if they wear them for too long. However, in 2013, Pu and others found a new way. The project WiSee, a novel gesture recognition system that leverages wireless signals (e.g., Wi-Fi) to enable whole-home sensing and recognition of human gestures. This enables gesture control without requiring instrumentation of human body with sensing devices. This picture shows the University of Washington visiting research assistant Qifan demonstrates WiSee, a technology that recognizes gestures such as a punch because of the slight disturbance it causes to the frequency of Wi-Fi signals. The gesture could be used to control another device connected to the Wi-Fi network. We always talk about empathy as UX designers. Empathy is defined as a combination of perspective-taking and shared emotion. Only by combining human emotion with technology can we meet the real needs of people instead of just following orders. Otherwise, we may generate potential harm to human-being. As designers for the future, how can we leverage new technologies to create an emotional experience for humans?

Technology/life will be a black mirror when no humanity inside. As designers for the future, how can we leverage new technologies to create an emotional experience for humans? Good emotional experience should cover as much human sensing system as possible. Typical sensing includes sight, hearing, touch, smell, and taste. Accordingly, we should apply visual, auditory, tactile, olfactory, and gustatory effects in those immersive experiences. Well - let’s bring back an interesting discussion point - Augemented Reality and Vitrual Reality, what is Reality?

Here is an example, for presenting a human, we should at least have 3 elements, model, lighting and texture, then we can see a proper 3D image like this. And there are more of course, motion, action and reaction, etc. Vitrual world should mimic real life physics: gravity, speed, etc. to make people feel they are “real”, however, let user feel not real as well. Let them feel like they are playing magic - and that is called ART. We already know that good emotional experience should cover as much human sensing system as possible. More than that, emotional stories are still the most efficient way to generate emotion, and it is the crutial part. This brings up another topic called “poetics” written by Aristotle. That is the way of writing a proper touching story. No matter tragdies or comedies, the story delivers emotions like pity, fear and passion - those things teach us how to be empathetic, how to think about others, not just oneself, and ultimately, how to be a better person.

I call this story-based ambient computing. And there are three levels or three milestones: 1st, single person ambience + single or limited endings. 2nd, multi person interactive ambience + multiple endings. 3rd, together with AI interactive ambience + multiple endings. To reach level 3, we need to work on human-character-based AI algorithm, which is something super challenging. I call this type of AI algorithm Emotional Plots, in which we really to have to design a person, not just shape, look and feel, but characteristics as a ‘human’ - like writer do. I like to use a formula to describe this: future vitrual world plus touching story equals to Interactive Film Experience. It is a bit like a game experience, but it is a game for everyone.

I am not the only one proposing this. One company that is often mentioned in the discourse is roblox. A game-as-a-platform that allows its users to create their own games and worlds. But we are still at a very early/immature stage at this point. I call this formula Creation of Utopia. A utopia (/juːˈtoʊpiə/ yoo-TOH-pee-ə) typically describes an imaginary community or society that possesses highly desirable or nearly perfect qualities for its members.

As I said, the new world should have humanity inside, that is why it should be always better than our current world - it should teach us to be a better human being. I am proposing the idea of Story-centered Utopia, which is a new type of universe that goes way beyond our current universe. In the future, everyone will have their own Utopia. And we are together, creating a multi-universe utopias future. There are supporting technolgoies for those. But we definitely need more. There are supporting technolgoies for those. But we definitely need more. Adn we all need to get prepared for the future on the way getting there. As designers, we should be responsible to put human humanity as the top priority.

*2022-2

To Be Continued.

Speaker  Ruixin J. Zhang  Contact  Email

©2022 Ruixinj.com All Rights Reserved.