Unsupervised Doodling and Painting with Improved SPIRAL

John F. J. Mellor, Eunbyung Park, Yaroslav Ganin, Igor Babuschkin, Tejas Kulkarni, Dan Rosenbaum, Andy Ballard, Theophane Weber, Oriol Vinyals, S. M. Ali Eslami

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


We investigate using reinforcement learning agents as generative models of images (extending arXiv:1804.01118). A generative agent controls a simulated painting environment, and is trained with rewards provided by a discriminator network simultaneously trained to assess the realism of the agent's samples, either unconditional or reconstructions. Compared to prior work, we make a number of improvements to the architectures of the agents and discriminators that lead to intriguing and at times surprising results. We find that when sufficiently constrained, generative agents can learn to produce images with a degree of visual abstraction, despite having only ever seen real photographs (no human brush strokes). And given enough time with the painting environment, they can produce images with considerable realism. These results show that, under the right circumstances, some aspects of human drawing can emerge from simulated embodiment, without the need for external supervision, imitation or social cues. Finally, we note the framework's potential for use in creative applications.
Original languageEnglish
Title of host publicationMachine Learning for Creativity and Design Workshop, Neural Information Processing Systems (NeurIPS)
StatePublished - 2 Oct 2019
Externally publishedYes

Bibliographical note

See https://learning-to-paint.github.io for an interactive version of this paper, with videos


  • cs.CV
  • cs.LG
  • stat.ML
  • I.2; I.4


Dive into the research topics of 'Unsupervised Doodling and Painting with Improved SPIRAL'. Together they form a unique fingerprint.

Cite this