VR TERROIR

[WIP Dec’24-Jan’26] : A project from the S+T+ARTS EC[H]O residency

  • challenge: Virtual Representations of Users / Spatial Computing

  • context: Virtual Reality, Scientific Visualisations, Collaboration, Avatar, Design

  • keywords: Biases & Challenges / scientific datasets / creative potential / realism and abstraction / nonverbal communication / perceptions.


VR TERROIR explores the virtual representations of users. in collaboration with the High Performance Computing Center in Stuttgart (HLRS), who have a solution to explore scientific data in Virtual or Augmented Reality settings, allowing for remote and multiple users to be present.

It is an amazing tool, but there is a missing piece of the puzzle: how do we represent users there, in a non uncanny valley way, supporting non-verbal communication and with a right degree of realism and abstraction?

My proposal, in a nutshell is: building from the existing rich and cultural background of hyperlocal particularities found in popular culture, heritage, nature and space, physical and digital

We could call it the Avatar Terroir. same as in wine, where the terroir defines the complete natural environment in which a particular wine is produced, including factors such as the soil, topography, and climate.

  • Going from: a fake, generic, loud, bias, profiling, sterile and simplistic representations of self.

  • To: Using treats of the local culture, materials and place.

On the left, we place the current state of avatars, which are mainly an expression of Corporate Optimism, hyper functional, median, politically correct digital beings. From the cuteness of apple’s memojis, to Meta’s cartoon clones.

There are also places of extreme niche self expression, as virtual communities like VRCHAT, where a plethora creatures roam around polygons. Those avatars are wild and creative and fantastical, but are mainly rooted in digital culture, lacking a sense of place or context.

And here on the right, where I want to go, is the existing representations of beings and humans by popular culture. We do have a rich, tactile, physical heritage of creatures in europe.

From these down there from the basque country in northern spain, who dress with sheep wool and giant bells, to this guy from hungary. I do not aim to make a direct translation of all of this, but this is to illustrate the physicality I aim to achieve, also taking from identifiers of the localities, from the Kelp of Greece or the moss in some Alpine forests.

I also want to explore how the digital context within a digital environment can drive the representation: since it is a scientific visualisation environment, how avatars should look behave or be affected if we are exploring nano microbial data or outer space simulations?


It turns out that this have a name :)

I’ll tap into the Critical Regionalism movement in Architecture, that reject placelessness of postmodernism and reclaims a contextualized practice, using local materials, knowledge, culture while using modern construction technologies.

For insance, the image in the background is a map of the Bioregions in europe, geographical areas with a similar biodiversity and common geographies. This could be a starting point, to break away from country boundaries that tend to be simplistic and profiling. For exampleinstance, I feel more mediterranean than spanish; in some aspects I have less in common with someone in the woods of Galicia, and more in common with someone from the coast of Lebanon.

Could the type of trees, the style of the cities or the food grown and eaten on a certain place drive the look and feel of avatars?

If this sounds political, is because it is,

digital spaces are colonized with globalized corporative capitalist extractive practices, Google Facebook Amazon and Apple dominate the language, the platforms and the narratives.

And I believe we need to build tools to reclaim identity and agency.

And how to do all this?

With my skillset, Digital Craftsmanship, coming from an industrial design background, I’m familiar with the world of making things, and over the years I’ve been intentionally de-materializing my practice and designing for the digital space, because I don’t see a reason strong enough to add make more stuff to the world. So I have experience in 3D environments and digitization.

For this project I envision using 3D scanning, computer vision, generative textures, parametric design, computational photography, generative AI, motion capture.

As a hint, what I could see myself doing in this project is climbing a mountain deep in the carpats, 3D scanning a rock and feed it to a generative system that blends it with a scientific dataset and proposes avatars.

 

VR TERROIR : contextual placefullness


FRAMEWORK

Technical scope

Research

 

Technical context

From the Host: HLRS

VR tools and facilities

Software developed at HLRS transforms data into dynamic projections. Wearing 3D glasses or head-mounted displays, users of our facilities and other tools can meet in groups to discuss and analyze results collaboratively.

The CAVE

Wearing 3D glasses, users can step inside simulations in this 3x3 meter room. Using a wand, it is possible to move through the virtual space and magnify small details.

Collaboration in virtual reality

When face-to-face meetings in the CAVE are not possible, software developed at HLRS enables persons in different physical locations to meet and discuss simulations in virtual reality from their workplaces or home offices.

Software

The Visualization Department at HLRS offers powerful virtual reality and augmented reality tools for transforming abstract data sets into immersive digital environments that bring data to life. These interactive visualizations support scientists and engineers across many disciplines, as well as professionals in nonscientific fields including architecture, city planning, media, and the arts.

COVISE (Collaborative Visualization and Simulation Environment) is an extendable distributed software environment to integrate simulations, postprocessing, and visualization functionalities in a seamless manner. From the beginning, COVISE was designed for collaborative work, allowing engineers and scientists to spread across a network infrastructure.

VISTLE (Visualization Testing Laboratory for Exascale Computing) is an extensible software environment that integrates simulations on supercomputers, post-processing, and parallel interactive visualization.

 

Other tools and technical solutions for this project:

  • Visual Language Models: for understanding images

  • Generative 3D models: for giving shape to concepts

  • Parametric Design: For computationally define boundaries and shapes

  • Gaussian splatting: For details and visuals beyond meshes and rendering

  • Computer vision: for movements and pose estimation

    • Depth estimation / video-to-pose

  • 3D scanning: for real world sampling

    • photogrammetry / dome systems / lidar / optical scans

  • Image generators: for texture synthesis

 

Societal context

Understanding the role of avatars in culture

Bibliographic research at the library collection from the Museu Etnològic i de Cultures del Món looking for folklore representations of beings.

[non-human / human] representations in folk culture

Pagan / material-rich costumes / masks.

A good repository of material qualities of costumes is found on the work Wilder Mann by Charles Fréger:

https://www.charlesfreger.com/portfolio/wilder-mann-fr/

photos: Wilder Mann by Charles Fréger

Other sources consulted include:

  • The photographies by Ivo Danchev, specially the Goat Dance collection

  • Masques du monde - L'univers du masque dans les collections du musée international du Carnaval et du Masque de Binche

  • Mostruari Fantastic (barcelona)

  • El rostro y sus mascaras (mario satz)

Analogies and parallels

  • Expressive avatars at edge digital cultures (Vrchat)

  • VR spaces

  • Costuming Cosplay Dressing The Imagination (Therèsa M. Winge)

Webcam backgrounds as a shy attempt to self expression

Relationship between the human body, space, and geometry [ Bauhaus ]

Triadisches Ballett (Triadic Ballet) is a ballet developed by Oskar Schlemmer. The ballet became the most widely performed avant-garde artistic dance and while Schlemmer was at the Bauhaus from 1921 to 1929, the ballet toured, helping to spread the ethos of the Bauhaus.

The Triadisches Ballett was conceived in 1912 in Stuttgart in a cooperation between the dance troupe of Albert Burger (1884–1970) and his wife Elsa Hötzel (1886–1966) and Oskar Schlemmer. Parts of the ballet were performed in 1915, but though Burger asked Arnold Schoenberg to write the score ("Your music, which I know from the local concert, seemed to me to be the only one suitable for my ideas"[1]) he had no success so Schlemmer and the Burgers performed to music by Enrico Bossi. It premiered in the Stuttgart landestheater on 30 September 1922, with music composed by Paul Hindemith. It was also performed in 1923 during the Bauhaus Week at the Nationaltheater, Weimar, and at the Annual Exhibition of German Crafts, Dresden.[2][3]



DEV / WIP / EXPERIMENTS / PROOF OF CONCEPT

Proof of concept 01

Characters informed by the materiality of their context.

In this experiment, I combine the aesthetics, material and texture characteristics of a setting [microscopic imagery, fluid dynamics, space simulations…] and translate them to visual cues shaping and texturing a character.

Visit at the HLRS and Media Solutions Center - January ‘25

Demo of equipment, projects, use cases and vision

With the aim of including humanistic thought and sociological context in this project, a conversation was arranged with the Head, Department of Philosophy of Computational Sciences at HLRS: Nico Formánek

Some of the concepts discussed were:

  • How do we “represent”?

  • Representation is a choice

    • When something is defined/described (i.e in thermodynamics) there is a whole set of things that are undefined.

    • Every choice leaves something out. (as per Jean-Paul Sartre’s concept of "choice and loss" —that every decision involves a trade-off, meaning something is always left behind. Sartre’s idea of "radical freedom" suggests that we are condemned to choose, and in choosing, we necessarily exclude other possibilities.)

    • Oportunity for this project to make explicit what is left in its choices

  • The concept of positioning: in a social context

    • How conventions apply in digital spaces / VR?

  • Standing - relationships between two things

    • How context guides standing. (i.e music in opera)

  • References: Claus Beisbart

    • Virtual Realism: Really Realism or only Virtually so?

      • This paper critically examines David Chalmers's "virtual realism," arguing that his claim that virtual objects in computer simulations are real entities leads to an unreasonable proliferation of objects. The author uses a comparison between virtual reality environments and scientific computer simulations to illustrate this point, suggesting that if Chalmers's view is sound, it should apply equally to simulated galaxies and other entities, a conclusion deemed implausible. An alternative perspective is proposed, framing simulated objects as parts of fictional models, with the computer providing model descriptions rather than creating real entities. The paper further analyzes Chalmers's arguments regarding virtual objects, properties, and epistemological access, ultimately concluding that Chalmers's virtual realism is not a robust form of realism.

    • new book: Was heißt hier noch real? (What's real today?)


Previous
Previous

POLAR BUBBLES

Next
Next

UTOPTICAL