{"id":450,"date":"2023-07-31T12:01:01","date_gmt":"2023-07-31T12:01:01","guid":{"rendered":"http:\/\/smalldivine.com\/?p=450"},"modified":"2023-10-04T11:31:26","modified_gmt":"2023-10-04T11:31:26","slug":"plot-twist-generative-ai-may-actually-encourage-architects-to-draw-more-2","status":"publish","type":"post","link":"http:\/\/smalldivine.com\/index.php\/2023\/07\/31\/plot-twist-generative-ai-may-actually-encourage-architects-to-draw-more-2\/","title":{"rendered":"Plot Twist\u200a \u2014 \u200aGenerative AI May Actually Encourage Architects to Draw MORE"},"content":{"rendered":"
Keir is an AEC Domain Expert<\/a> operating at the intersection of architecture practice, sustainable development and software design. With over 15 years in practice, he has crafted high-quality projects across various sectors, including education, health, housing, and workplaces. He helps Architects, Clients and Startups thrive in an ever-changing industry.<\/em>\u00a0Connect on LinkedIn<\/a>.<\/em><\/p>\n A new week, a new way for Generative AI to blow our minds.<\/p>\n Images generated from text prompts have now filled my news feed; they have swiftly ascended the viral ladder, caused heated debates and gained \u2018meme status.\u2019 When these arresting visuals started appearing, it felt like each novel experimentation demanded our close attention: \u201cWhat has this inexplicable new tool done now?!\u201d<\/p>\n Yet modern attention spans are increasingly short, and bold imagery can quickly become ubiquitous. Our sense of wonder is easily replaced by boredom and ambivalence. Most people are familiar with models that use simple text prompting, where you describe everything about a composition using words only. Much can be achieved with these tools, but when it comes to exact composition and configuration, you are working at the model\u2019s behest. However, fewer architects are aware that you can now combine an image with a text prompt to further your creative control.<\/p>\n While these are enormously promising developments, it has been hard to understand exactly how an architect might be able to use these tools. How can we use them to augment the design and visualization processes we are already doing? In architecture, we work in the gritty reality, not the synthetic imagination of AI. Planning and construction is a messy business that requires precision solutions.<\/p>\n Yet, sketch-to-render is a different kind of approach and takes user control to the next level again, utilizing an additional step in the generation feature called Control Nets, which allow for far greater control over how an image is constructed and where the trained model will go to work on a composition. Think of Control Nets as a framework or bounding box within which the AI will go to work \u2014 it puts you in the driving seat of the model\u2019s explorations.<\/p>\n Sketch by author (of NWLND\u2019s \u201cRefuge\u201d project), render produced in Prome AI<\/em><\/p>\n<\/div>\n This is a 20-minute process and the idea here is to go straight from primitive line work to vivid render.<\/p>\n Midjourney can produce incredibly high quality and vivid imagery, but offers limited control over the exacting composition of the subject matter. For fields like architecture, the ability to fix the areas within an image around which the model will iterate is absolutely essential for actual tool adoption and use.<\/p>\n There are now various methods to combine generative image tools with \u2018fixed\u2019 image subjects and composition to give more exacting control over a single viewpoint and to then iterate ideas on top.<\/p>\n Here are some good emerging methods that are worth experimentation:<\/p>\n Example of varied rendered outputs produced from the original sketch<\/em><\/p>\n<\/div>\n This is a more complex process and takes about 30 minutes once you understand how to do it.<\/p>\n The direct sketch-to-render tools are great to use, but having experimented with them in detail, I felt they were best suited to interior design work only. When it comes to external envelope and massing, we always need to place our ideas in context and render with appropriate scale, visualizing the buildings and landscapes within which they sit.<\/p>\n Then, I discovered the idea of using two control nets together: one for a process called \u201cin-painting\u201d and the second for the sketch proposal. There is quite a bit of trial and error to get the workflow right, but it\u2019s made possible by running Stable Diffusion with ControlNet model on your local machine and is repeatable for any photo and sketch combo (provided you can draw).<\/p>\n Photograph taken by author of an imaginary development site while on a cycle ride home<\/em><\/p>\n<\/div>\n The aforementioned simple \u201csketch-to-render\u201d process works with one ControlNet active. However you can now use Stable Diffusion with a second ControlNet at the same time which can be used for a process called\u00a0 \u201cin-painting\u201d and this allows you to tell the model exactly which parts of a source image you want to experiment with and which you want to leave exactly as they are.<\/p>\n What I love about this application of Generative AI is that it relies solely on your direction and discretion as a designer and what you do with the pen. It takes out the \u201cmiddle-man\u201d of painstaking digital modeling of an idea and goes straight to vivid imagery. This is rendering without the hours of boring 3D modeling.<\/p>\n<\/blockquote>\n In a world where you can get straight to a high-quality render with just a site photo, a sketch and an ability to describe your idea in the form of an effective prompt, you immediately bypass the need to build detailed 3D models of initial concept ideas.<\/p>\n Design sketch by author, drawn quickly directly on top of original photograph.<\/em><\/p>\n<\/div>\n Arguably, most major practices working with developers have to run projects with a high degree of waste. Options are tested, digitally modeled in 3D, rendered, photoshopped and\u00a0 perhaps mocked up quickly in foam or card model for a client to review.\u00a0 Then we respond to comments, requests for changes, new constraints, new information and a continuous process of change occurs.<\/p>\n Throughout this design process, each rendition must be exhaustively conceived, drawn and modeled before it can be vividly represented in context; this means much of the previous work is discarded or inevitably thrown away almost immediately. Quick iteration tooling could massively reduce the waste and grunt work associated with the process we call \u201coptioneering.\u201d\u00a0 This term is not used affectionately in practice because it can feel so non-linear, but unfortunately some form of option testing is always required to discover a design and each option requires lots of time and energy. Screenshot produced by author showing process of in-painting using Stable Diffusion with 2no ControlNets \u2013 the black area tells the model which part of the image to experiment upon<\/em><\/p>\n<\/div>\n In a race to produce powerful concept imagery for a new feasibility study or competition, someone who can draw their ideas well is going to beat 3D modeling in turnaround time and speed of iterations using a tool like this\u200a.<\/p>\n Soon, clients will be getting rendered ideas in a matter of days from a commission, not weeks. Their expectations about what is possible in a given time frame are going to change \u2014 and quickly.<\/p>\n<\/blockquote>\n There remains loads of space for improvement, but the foundation is there for a very different approach to design and visualization that could be really empowering for architects (and clients too).<\/p>\n Final Image produced by author using Stable Diffusion with two ControlNets for an imaginary project in Herne Hill, London. The image was produced in less than an hour including sketching time.<\/p>\n<\/div>\n My first attempt is a bit rough but demonstrates the enormous potential here; imagine how powerful this will be for early stage feasibility work. I definitely don\u2019t love the outcome, but it\u2019s some version of what I was thinking in the sketch. I would still be happy to present this image to a client as an early study at feasibility stage to give them a more vivid sense of a project\u2019s massing and scale alongside a set of 2D drawings before developing the preferred option in fine detail myself.<\/span><\/p>\n In the example shown, I would estimate the model achieved about 50% of my line intentions and about 20% of my material intent on the fa\u00e7ades. However, the perspective, massing, lighting, context placement, reflections and sense of scale are all bang on \u2014 and all this is done with a general purpose, open-source model.<\/p>\n<\/blockquote>\n As this technology continues to improve, specialized architecture models will be trained on data sets that focus specifically on fa\u00e7ade and architectural composition. Different architectural styles and materiality options will be made possible and they will be far better at understanding fa\u00e7ade componentry such as floor zones, balustrades, windows, curtain walling and columns. The models will need to learn \u201carchispeak\u201d which will now be expressed using text prompt inputs and require architects to say what they actually mean in simple language.<\/p>\n As these models improve we will be able to discern discrete elements within the design concept, identify them as an architectural building component and then refine them directly with prompts as we work. We will be able to apply different prompts to different parts of the image, add people, change the lighting and mood \u2014 designing over and over in a live render environment \u2014 without modeling anything, all potentially driven from a sketch idea.<\/p>\n There will be many more experiments to come\u2026 and when Midjourney can do ControlNets too, it will probably feel like \u201cgame over\u201d for much of the traditional 3D model and rendering that we do currently.<\/p>\n Keir is an AEC Domain Expert operating at the intersection of architecture practice, sustainable development and software design. Connect on LinkedIn<\/a>.<\/em><\/p>\n Ismail Seleit<\/a> who was the first person I saw demonstrate this idea.<\/p>\n @design.input<\/a> who put out a great video that helped to describe each step of the process.<\/p>\n Hamza Shaikh<\/a> for pointing me at ControlNet in the first place when I got frustrated with Midjourney.<\/p>\n \n Architizer’s new image-heavy daily newsletter<\/strong>, <\/a>The Plug<\/a>,<\/strong> is easy on the eyes, giving readers a quick jolt of inspiration to supercharge their days. Plug in to the latest design discussions by subscribing.<\/strong><\/a>\u00a0<\/em><\/p>\n The post Plot Twist\u200a \u2014 \u200aGenerative AI May Actually Encourage Architects to Draw MORE<\/a> appeared first on Journal<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":" Keir is an AEC Domain Expert operating at the intersection of architecture practice, sustainable development and software design. With over 15 years in practice, he … <\/p>\n","protected":false},"author":1,"featured_media":453,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/posts\/450"}],"collection":[{"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/comments?post=450"}],"version-history":[{"count":3,"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/posts\/450\/revisions"}],"predecessor-version":[{"id":469,"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/posts\/450\/revisions\/469"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/media\/453"}],"wp:attachment":[{"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/media?parent=450"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/categories?post=450"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/smalldivine.com\/index.php\/wp-json\/wp\/v2\/tags?post=450"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nIncredibly, the development of Generative AI actually appears to be evolving faster than our agitated modern attention spans. Just as I was beginning to become nonplussed by the latest hybridization of Batman X The Simpsons, I discovered sketch-to-render.<\/p>\n<\/p>\n
Sketch-to-Render<\/h2>\n
\n
<\/p>\n
Sketch-to-Render-in-Context<\/h2>\n
<\/p>\n
\n
<\/p>\n
\n<\/span><\/p>\n<\/p>\n
\n
<\/p>\n
\n
\nWith thanks to:<\/h4>\n
\n