I just read this. The conclusion is that creativity, in image diffusion models at least, stems from locality (generating then accepting a small part of the image) and translational equivariance (generating the next small local part and making it coherent with the last over and over). The researchers and are author are a bit coy about it, but imo this is how human creativity works too: progressive coherence, you assume some constraints, make something0 constrained by them, move beyond something0 and making something1 coherent with something0, over and over until the aggregate is something worthwhile and new.