Apple is advancing its exploration of generative AI to enhance app development processes, focusing specifically on user interface (UI) generation. Recently, a new paper titled “Improving User Interface Generation Models from Designer Feedback” was published by the team behind UICoder, an initiative aimed at training AI to create functional UI code rather than merely focusing on design aesthetics.
In this study, 21 professional designers contributed feedback by assessing and refining AI-generated UIs through comments and sketches. Their input was essential in creating a reward model that aligns better with real-world design practices. The designers involved had diverse backgrounds, with experience levels ranging from 2 to over 30 years, and they frequently engaged in design reviews as part of their routine work.
During the research, the team gathered 1,460 annotations that contrasted the original AI-generated interfaces with the improved designs suggested by the participants. This data was instrumental in training the reward model, which evaluates UI outputs based on a rendered image and a descriptive text, ultimately producing a numerical score to guide the AI towards generating higher-quality designs.