VR TERROIR
[WIP Dec’24-Jan’26] : A project from the S+T+ARTS EC[H]O residency
challenge: Virtual Representations of Users / Spatial Computing
context: Virtual Reality, Scientific Visualisations, Collaboration, Avatar, Design
keywords: Biases & Challenges / scientific datasets / creative potential / realism and abstraction / nonverbal communication / perceptions.
VR TERROIR explores the virtual representations of users. in collaboration with the High Performance Computing Center in Stuttgart (HLRS), who have a solution to explore scientific data in Virtual or Augmented Reality settings, allowing for remote and multiple users to be present.
It is an amazing tool, but there is a missing piece of the puzzle: how do we represent users there, in a non uncanny valley way, supporting non-verbal communication and with a right degree of realism and abstraction?
My proposal, in a nutshell is: building from the existing rich and cultural background of hyperlocal particularities found in popular culture, heritage, nature and space, physical and digital
We could call it the Avatar Terroir. same as in wine, where the terroir defines the complete natural environment in which a particular wine is produced, including factors such as the soil, topography, and climate.
Going from: a fake, generic, loud, bias, profiling, sterile and simplistic representations of self.
To: Using treats of the local culture, materials and place.
On the left, we place the current state of avatars, which are mainly an expression of Corporate Optimism, hyper functional, median, politically correct digital beings. From the cuteness of apple’s memojis, to Meta’s cartoon clones.
There are also places of extreme niche self expression, as virtual communities like VRCHAT, where a plethora creatures roam around polygons. Those avatars are wild and creative and fantastical, but are mainly rooted in digital culture, lacking a sense of place or context.
And here on the right, where I want to go, is the existing representations of beings and humans by popular culture. We do have a rich, tactile, physical heritage of creatures in europe.
From these down there from the basque country in northern spain, who dress with sheep wool and giant bells, to this guy from hungary. I do not aim to make a direct translation of all of this, but this is to illustrate the physicality I aim to achieve, also taking from identifiers of the localities, from the Kelp of Greece or the moss in some Alpine forests.
I also want to explore how the digital context within a digital environment can drive the representation: since it is a scientific visualisation environment, how avatars should look behave or be affected if we are exploring nano microbial data or outer space simulations?
It turns out that this have a name :)
I’ll tap into the Critical Regionalism movement in Architecture, that reject placelessness of postmodernism and reclaims a contextualized practice, using local materials, knowledge, culture while using modern construction technologies.
For insance, the image in the background is a map of the Bioregions in europe, geographical areas with a similar biodiversity and common geographies. This could be a starting point, to break away from country boundaries that tend to be simplistic and profiling. For exampleinstance, I feel more mediterranean than spanish; in some aspects I have less in common with someone in the woods of Galicia, and more in common with someone from the coast of Lebanon.
Could the type of trees, the style of the cities or the food grown and eaten on a certain place drive the look and feel of avatars?
If this sounds political, is because it is,
digital spaces are colonized with globalized corporative capitalist extractive practices, Google Facebook Amazon and Apple dominate the language, the platforms and the narratives.
And I believe we need to build tools to reclaim identity and agency.
And how to do all this?
With my skillset, Digital Craftsmanship, coming from an industrial design background, I’m familiar with the world of making things, and over the years I’ve been intentionally de-materializing my practice and designing for the digital space, because I don’t see a reason strong enough to add make more stuff to the world. So I have experience in 3D environments and digitization.
For this project I envision using 3D scanning, computer vision, generative textures, parametric design, computational photography, generative AI, motion capture.
As a hint, what I could see myself doing in this project is climbing a mountain deep in the carpats, 3D scanning a rock and feed it to a generative system that blends it with a scientific dataset and proposes avatars.
VR TERROIR : contextual placefullness
FRAMEWORK
Technical scope
Research
references / inspiration / links
links database → https://cunicode.notion.site/129454e7e114817e9ac2e1fc298a9241?v=129454e7e1148128ae00000cd13f0c98
Technical context
From the Host: HLRS
VR tools and facilities
Software developed at HLRS transforms data into dynamic projections. Wearing 3D glasses or head-mounted displays, users of our facilities and other tools can meet in groups to discuss and analyze results collaboratively.
Wearing 3D glasses, users can step inside simulations in this 3x3 meter room. Using a wand, it is possible to move through the virtual space and magnify small details.
Collaboration in virtual reality
When face-to-face meetings in the CAVE are not possible, software developed at HLRS enables persons in different physical locations to meet and discuss simulations in virtual reality from their workplaces or home offices.
Software
The Visualization Department at HLRS offers powerful virtual reality and augmented reality tools for transforming abstract data sets into immersive digital environments that bring data to life. These interactive visualizations support scientists and engineers across many disciplines, as well as professionals in nonscientific fields including architecture, city planning, media, and the arts.
COVISE (Collaborative Visualization and Simulation Environment) is an extendable distributed software environment to integrate simulations, postprocessing, and visualization functionalities in a seamless manner. From the beginning, COVISE was designed for collaborative work, allowing engineers and scientists to spread across a network infrastructure.
VISTLE (Visualization Testing Laboratory for Exascale Computing) is an extensible software environment that integrates simulations on supercomputers, post-processing, and parallel interactive visualization.
Other tools and technical solutions for this project:
Visual Language Models: for understanding images
Generative 3D models: for giving shape to concepts
Parametric Design: For computationally define boundaries and shapes
Gaussian splatting: For details and visuals beyond meshes and rendering
Computer vision: for movements and pose estimation
Depth estimation / video-to-pose
3D scanning: for real world sampling
photogrammetry / dome systems / lidar / optical scans
Image generators: for texture synthesis
Societal context
Understanding the role of avatars in culture
Bibliographic research at the library collection from the Museu Etnològic i de Cultures del Món looking for folklore representations of beings.
[non-human / human] representations in folk culture
Pagan / material-rich costumes / masks.
A good repository of material qualities of costumes is found on the work Wilder Mann by Charles Fréger:
photos: Wilder Mann by Charles Fréger
Other sources consulted include:
The photographies by Ivo Danchev, specially the Goat Dance collection
Masques du monde - L'univers du masque dans les collections du musée international du Carnaval et du Masque de Binche
Mostruari Fantastic (barcelona)
El rostro y sus mascaras (mario satz)
Analogies and parallels
Expressive avatars at edge digital cultures (Vrchat)
VR spaces
Costuming Cosplay Dressing The Imagination (Therèsa M. Winge)
Webcam backgrounds as a shy attempt to self expression
Relationship between the human body, space, and geometry [ Bauhaus ]
DEV / WIP / EXPERIMENTS / PROOF OF CONCEPT
Proof of concept 01
Characters informed by the materiality of their context.
In this experiment, I combine the aesthetics, material and texture characteristics of a setting [microscopic imagery, fluid dynamics, space simulations…] and translate them to visual cues shaping and texturing a character.
Visit at the HLRS and Media Solutions Center - January ‘25
Demo of equipment, projects, use cases and vision
With the aim of including humanistic thought and sociological context in this project, a conversation was arranged with the Head, Department of Philosophy of Computational Sciences at HLRS: Nico Formánek
Some of the concepts discussed were:
How do we “represent”?
Representation is a choice
When something is defined/described (i.e in thermodynamics) there is a whole set of things that are undefined.
Every choice leaves something out. (as per Jean-Paul Sartre’s concept of "choice and loss" —that every decision involves a trade-off, meaning something is always left behind. Sartre’s idea of "radical freedom" suggests that we are condemned to choose, and in choosing, we necessarily exclude other possibilities.)
Oportunity for this project to make explicit what is left in its choices
The concept of positioning: in a social context
How conventions apply in digital spaces / VR?
Standing - relationships between two things
How context guides standing. (i.e music in opera)
References: Claus Beisbart
Virtual Realism: Really Realism or only Virtually so?
This paper critically examines David Chalmers's "virtual realism," arguing that his claim that virtual objects in computer simulations are real entities leads to an unreasonable proliferation of objects. The author uses a comparison between virtual reality environments and scientific computer simulations to illustrate this point, suggesting that if Chalmers's view is sound, it should apply equally to simulated galaxies and other entities, a conclusion deemed implausible. An alternative perspective is proposed, framing simulated objects as parts of fictional models, with the computer providing model descriptions rather than creating real entities. The paper further analyzes Chalmers's arguments regarding virtual objects, properties, and epistemological access, ultimately concluding that Chalmers's virtual realism is not a robust form of realism.
new book: Was heißt hier noch real? (What's real today?)
Proof of concept #3
Using gaussian splatting to represent detailed material features
-> scan of the Triadisches Ballett at the Staatsgalerie Stuttgart
Text 2 3D models
challenge:
need to find: if a system can output a turntable of the creation before the meshing step, it could be possible to nerf it, and create a neural representation of the volume, without going through triangles…
HLRS collab
Meetings with the HLRS team to discuss the project. Some of the things we discussed are:
VR continuum and role of the CAVE systems
XR expos and industry events
Remote Access to a GPU cluster
Tech Sessions
Meetings with Carlos Carbonell to explore the technological soundess of the idea and preceding , existing and future solutions.
Session: History of identity in digital spaces and vr platforms
Session: Hardware setup.
Session: visit at Event-Lab at the Psychology department from the Universitat de Barcelona - where they carry out technical research on virtual environments, with applications to research questions in cognitive neuroscience and psychology. We were hosted by Esen K. Tütüncü who showed us their research projects, equipment, pipelines and vision.
Session (upcoming): we’ll meet in VR to have a safari-tour in VRCHAT
Next Sessions
Meetings with Dr. Claudia Schnugg to discuss and explore how to shape this project for continuation beyond the residency
