Shaping the Unknown: Technology as a Design Material
Traditionally, materials have been the primary teachers of artists and designers.
Think of a ceramicist at the wheel. Their hands respond to clay, how it shifts with water, how it resists pressure, how it remembers touch. Push too far and it collapses. Add too much water and it slips. Every adjustment becomes a small experiment. The material talks back.
Architects understand this instinctively. Wood carries warmth and grain and moves with the seasons. Glass is fragile, yet transforms light into structure. Steel resists. Each material encodes physical properties and cultural meaning.
As artist Richard Tuttle once said: “It’s the material that teaches you.”
And yet, when we move into computing, something changes.
The Problem With Invisible Materials
As digital designers, our “materials” are harder to grasp. Pixels, algorithms, sensors, data. These things don’t sit in your hand. They often live behind screens, or worse, disappear entirely.
With AI, this abstraction deepens. Even people building these systems will admit they don’t fully understand how they work. As Dario Amodei, CEO of Anthropic, recently wrote, people outside the field are often “surprised and alarmed to learn that we do not understand how our own AI creations work.”
This opacity has consequences.
One of them is interface homogenization. Nearly every AI product now looks the same: a large text box, a blinking cursor, a chat window. When we don’t understand the material we’re working with, we default to the safest, most familiar forms.
Designers are often brought in after technologies are built, asked to “color within the lines.” Technology becomes a constraint.
But what if we flipped that assumption?
What If Technology Were a Material?
What if we treated technology not as a black box, but as something to shape?
What if designers were involved earlier, not just in polishing interfaces, but in discovering what new technologies want to be?
This idea isn’t new. In fact, it has deep roots.
History of Material Thinking
At the Bauhaus in the 1920s, education began not with tools or outcomes, but with materials themselves.
In the preliminary course developed by Johannes Itten, students explored contrast, texture, weight, and perception through direct, hands-on experiments. Later, under László Moholy-Nagy and Josef Albers, this approach became deeply intertwined with modern industry and technology.
One of Albers’ most famous exercises was deceptively simple: he gave each student a single sheet of paper and asked them to transform it into a three-dimensional form. No glue. No cutting. The goal wasn’t usefulness but mainly to have a dialogue with the material. To feel resistance. To discover thresholds. To learn the material’s language.
The Bauhaus described this approach as “contact with materials.” Form, in this view, does not originate from ideology or intent alone, but emerges through attentive engagement and listening.
While industrial design and architecture have long treated materials as central to form-making, digital technologies are often approached as immaterial or purely functional. Algorithms, data, and computation are taken as given—black-box constraints rather than materials to be explored.
Design theorist Johan Redström has pointed out that interaction design has struggled to treat technology as a material in its own right (see Redström, On Technology as Material in Design). Computational materials do not reveal themselves through direct sensory contact like wood or steel; instead, their properties emerge through execution, timing, feedback, and behavior across layers of abstraction. As a result, designers often engage primarily with surface-level interfaces, disconnected from the deeper material dynamics that shape experience.
Yet it is precisely through engaging with these computational materials (via experiments, probes, and material studies) that new forms of interaction, expression, and meaning can emerge. From this perspective, design practice becomes a continuous negotiation between people, culture, and material possibilities. Materials are no longer passive constraints to work around, but active participants in the invention process.
When Designers Make the Invisible Visible
We see this approach resurface whenever designers encounter new, opaque technologies.
In 2009, the design studio BERG began experimenting with RFID, an invisible, proximity-based technology used in transit cards and payment systems. At the time, RFID was still relatively new in consumer contexts. It was mostly used behind the scenes in logistics, inventory tracking, access badges, and transit systems. The technology was inexpensive, small, and increasingly embedded in everyday objects, but almost entirely invisible. Instead of asking how to build a product with RFID, they asked: What does RFID feel like as a material?
Their project, Immaterials, used light to reveal the invisible sensing fields around RFID readers. Suddenly, range, sensitivity, and irregular shape became perceptible. Designers could play with the material and design from understanding rather than abstraction. These experiments brought an informed perspective of how RFID technology should be integrated into products.
Within human–computer interaction, scholars such as Mikael Wiberg and Erik Stolterman have articulated a material-centered approach to design. This perspective is commonly framed around three interrelated dimensions: material, referring to the properties, constraints, and affordances of the technology itself; making, the craft and practice of shaping those materials into form; and meaning, the integration of those forms into cultural, social, and lived contexts where they acquire significance.
Material Studies at Google ATAP
We spent several years at Google ATAP, a research and development group inside Google that operated more like a lab than a product team. ATAP worked on technologies that were often five to ten years away from being viable for consumer products, which meant that outcomes were uncertain and success was rarely defined upfront. The group brought together scientists, engineers, designers, artists, and technologists, and design was not treated as a downstream function. Instead, designers were involved from the beginning, working alongside researchers to explore what new technologies could become before they had clear applications.
One of those bets was Project Soli, a miniature radar sensor and sensing platform capable of detecting motion from sub-millimeter finger movements to full-body gestures (Learn more about Project Soli).
Radar possesses unusual properties: it is low-power, works through materials, and is robust to light, temperature, and occlusion. Despite these advantages, early on, it was deeply unintuitive. What does radar “see,” exactly?
To understand its nature, the team didn’t start with use cases. Instead, the approach was to comprehend radar as a material. In collaboration with our AI research team, we developed algorithms to translate the raw radar signal into intelligible visualizations. These early visualizations were crucial, allowing us to develop an intuitive understanding of radar’s qualities.
By isolating and testing different variables, we began to see consistent patterns emerge. We built an environment that let us visualize the output of radar detections overlaid on a camera feed. This allowed us to run several experiments, observing in real-time how radar reacted to different body movements. We were able to map out the interaction space the radar created and discover the patterns to which radar was most sensitive: Presence, Orientation, and Pathways. Small cues like glancing toward a device or turning away from it—everyday movements that already carry meaning and intent—became discernible.
From these material studies, we established four main interaction primitives (Approach/Leave, Pass, Turn, Glance) and an interaction framework, FIELDS, which uses a continuous value of interest between people and devices to determine their intent and level of engagement.
These concepts were demonstrated through a series of speculative prototypes that reimagined the future of Google Home Products. This research was well received at Google for introducing interaction techniques aligned with ambient computing, a paradigm rooted in the work of Mark Weiser, who argued that the most effective technologies “disappear” into everyday life. By prioritizing subtle, context-aware, and minimally intrusive interactions, the work advanced systems that remain in the background, anticipate user needs, and surface only when contextually or functionally necessary. (Learn more about the project)
A Material Mindset
If technology is a material, the question becomes how we learn to work with it. At ATAP, we approached this not as a linear design process, but as a form of material inquiry. We often began with core sensing algorithms, treating them as unfamiliar materials whose properties had to be discovered. Rather than starting with users or applications, we focused on what the technology could perceive, how it behaved over time, and where it broke down.
Material studies were how we built a “legible map” of how radar technology works. Through direct experimentation, patterns emerged and we translated those insights into interaction frameworks by defining new primitives and metaphors grounded in the system’s intrinsic behavior. Sharing these frameworks, often paired with speculative videos and scenario prototypes, helped make the technology legible and align teams around new possibilities.
This way of working sat outside traditional user-centered design. It combined speculative design, technical feasibility, and interaction theory. By deeply understanding the material properties of the technology, we established the foundation for interaction languages that felt native to the system and capable of supporting new forms of engagement.
Right now, with AI still unclear and changing quickly, we have some agency. We can treat it as a fixed constraint, or we can approach it the way designers have always approached new materials: by exploring it, questioning it, and shaping it carefully (more about AI material studies in a future post). The choices we make here matter, because they influence how these technologies show up in everyday life.
 | A guest post by
|
Making the Invisible Actionable: A Pragmatic Approach to Digital Materiality
The design methodology proposed by Bedal and Giusti in Shaping the Unknown can be interpreted as a practical validation of the philosophical theories of William James, Jakob von Uexküll, and James Gibson. While Bedal and Giusti argue for treating invisible code as a "material," the theoretical text clarifies why this approach works: because we cannot perceive the "essence" of a technology without actively engaging with it for a specific purpose.
1. The "Material Dialogue" as an Action-Perception Loop.
Bedal and Giusti observe that traditional materials "talk back" to the craftsman—clay resists pressure, and wood moves with the seasons. This feedback loop allows the maker to learn the material’s language. However, they note that digital materials like AI are often treated as "black boxes," leading to a lack of understanding and "interface homogenization".
The enactive view of the mind explains this failure: perception is not a passive reception of data but an "active process of exploring the world for its action possibilities". By failing to "handle" algorithms like clay, designers break the loop of perception and action. Bedal and Giusti’s solution—"material studies"—restores this loop by forcing designers to actively experiment (action) to reveal the technology's properties (perception).
2. Determining Essence Through "Thinking for Doing"
The article describes how the Google ATAP team struggled to understand radar technology, which was "deeply unintuitive" and invisible. They did not attempt to define the radar by its technical specifications alone; instead, they built visualizations and moved their bodies in front of sensors to see how the signal reacted.
This approach directly mirrors William James’s pragmatism. James argued that the "essence" of an object is not absolute but determined by our "doing". Just as paper becomes "combustible material" only when one intends to start a fire, the radar signal only became an interface for "presence" and "orientation" when the designers engaged it with specific physical actions. The technology’s identity was "inextricably linked to practical goals and actions".
3. "Interaction Primitives" as Functional Tones and Affordances
Finally, the "interaction primitives" discovered by Bedal and Giusti (such as Approach, Turn, and Glance) can be understood as the discovery of affordances. Gibson defined affordances as the action possibilities an environment offers to an actor. Similarly, Uexküll described "functional tones," where an object takes on meaning based on an organism's biological capacities.
In Shaping the Unknown, the designers had to find the "functional tone" of the radar. By mapping "micromovements" to the sensor's capabilities, they established a "continuous negotiation" between human capability and material possibility. This confirms the theoretical assertion that perception and action are "mutually defining". The designers could not perceive the radar as a "glance-able" interface until they acted upon it, proving that "cognition's primary role is practical engagement with the world".