Latent Structures: Between Generative Models and Contemporary Design
by Fabio Gnassi
The meteoric rise of generative models capable of creating text and images has revealed the immense potential of using artificial intelligence in the creative process of designers and architects. Research is advancing relentlessly, laying the foundations for a new generation of models capable of transforming textual or graphic input into three-dimensional objects, ready to revolutionise existing design paradigms.
We talk about it with George Guida, researcher at the Harvard Laboratory for Design Technologies, is co-founder of ArchiTAG and xFigura. An expert in AI for architecture, he integrates technology and design with global and academic projects.
3D generative models are based on a process known as “ML Representation,” a particular form of representation that can be achieved through various techniques. Can you explain how it works?
3D generative models use different approaches to create three-dimensional objects with AI. Unlike text and images, which benefit from abundant and well-defined datasets, training for 3D generation presents greater challenges. In architecture, 3D datasets are fragmented across various formats (mesh, NURBS, surfaces, etc.) and are difficult to centralise due to intellectual property issues. Moreover, processing them requires high computational power.
To tackle these challenges, new approaches have emerged, which I have explored as a researcher at the Harvard Graduate School of Design. These include techniques based on language models (LLMs) or “multiview” processes that utilise 2D images to generate 3D objects. One example is “Gaussian Splatting,” popular in 2023. Similar to photogrammetry, it leverages AI to produce high-quality results, now applicable in architecture for digital reconstructions or innovative concepts.
We explore the impact of real-time AI applications, enriching traditional methods. We believe in using AI to give citizens a voice, helping them imagine better urban spaces.
What is a Gaussian Splat, and how does its training process work?
A Gaussian Splat is an innovative technique for representing 3D objects through Gaussian functions, which overlap to recreate complex surfaces. This methodology has evolved from Neural Radiance Fields (NeRF), offering greater efficiency and more refined visual results, and it is particularly useful in architecture and design.
The training process involves optimising the position, size, and intensity of the Gaussians to fit the reference data, such as images or point clouds. At my architecture firm, ArchiTAG, we have integrated this technology into an advanced workflow combining Unity and Google Maps’ API. This allows us to digitally scan a site, modify it with text prompts and image processors, and visualise the results in real-time.
Beyond research, you also pursue a professional career with the firm ArchiTAG. What role does artificial intelligence play in your design process?
At my firm, ArchiTAG, artificial intelligence (AI) is integrated into all projects, especially during the conceptual phases. This approach reflects my academic work at Harvard University and Wentworth Institute of Technology. One example is a project for a climbing gym in Uganda, where we used AI-driven workflows to transform initial sketches into conceptual images and ML models for environmental analysis.
We explore the real-time impact of AI applications, enriching traditional methods. We believe in using AI to give citizens a voice, helping them imagine better urban spaces. We also provide AI consulting for architecture firms, supporting the adoption of innovative tools to enhance competitiveness and awareness. Recently, we helped a firm develop a RAG model to improve RFP creation, securely integrating an LLM with company data.
Technological progress is leading to the proliferation of numerous applications that could play a significant role in the compositional process. The implementation and refinement of these products outline a system in which the architect seems to play an increasingly marginal role. What is your opinion on this?
I always describe artificial intelligence as “augmented imagination.” AI can enrich creative workflows if used as support rather than a shortcut. This creates space to curate generative results, optimising them for informed, design-based decisions. It is crucial to maintain an active and critical role in the use of these tools. As Marshall McLuhan said, “we must shape the tools that in turn shape us.” Architects must be involved in the development of these technologies, especially now that code is more accessible.
This principle also guides xFigura, a generative ideation startup we will launch in January 2025. It will combine the best AI models in a collaborative space, helping architects and designers harness these technologies without losing their creative identity.
You are also one of the first architects addressing the topic of regulating AI use in architecture. What is your opinion on this?
With the growing use of AI in architecture, it is essential to maintain our decision-making role and the responsibility to ensure the quality of projects for the public. There needs to be a clear definition of what “responsible AI” means for our profession.
The European Union has taken decisive steps with the AI Act and the Product Liability Directive. Italy, ranking third globally for supercomputing infrastructure, has the opportunity to lead this discussion rather than follow it. I collaborate with NCARB in the United States and RIBA in the United Kingdom on this topic. I do not believe that AI in architecture should be regulated immediately, but it is crucial to establish a general framework for informed and ethical decisions. AI, currently viewed as a tool to automate tasks like checks and reviews, could evolve into a collaborative agent. We must ensure safeguards for its safe and responsible use.
