Surprise Me: How DrawDick Trained Its Own Doodle Model

The short version: in the editor, open the Menu and press Surprise me, and a small drawing model draws you a doodle to start from, right there on the page. It is yours to change: draw on it, rub bits out, dress it up. DrawDick trained that model itself, on an openly licensed collection of doodles, in under an hour on one laptop. Here is how it works, how we made it, and where it falls short.

What Surprise me does

You will find Surprise me in two places: in the editor's Menu, under Your drawing, and in the drawing-idea card beside a blank page, where there is room for it.

1. The model loads, once

   The first time, a bar at the foot of the drawing shows the model downloading (about 1.9 MB), with a Cancel button. Your browser keeps it, so next time it is instant.

2. It draws

   The doodle draws itself in, line by line, in about a second. Press anywhere and it goes down whole; with reduced motion switched on, it just appears.

3. Make it yours

   It is ordinary pen strokes in your brush's color, size and opacity: draw over it, erase bits, move it, add stickers. Undo takes the whole doodle back in one step.

4. Or have another

   Another draws a fresh one. Done puts the bar away and leaves the doodle.

Surprise me starts a fresh page in place of the drawing you had: it does not add the doodle to it. A drawing saved to your account stays safe there. If the page holds a drawing not saved to your account, it asks first, and Keep my drawing is the default.

Or skip the menu: drawdick.com/c/surprise opens the editor with the prompt "Draw a dick from a surprise doodle" and a fresh doodle on the page. Add a background if you like: /c/surprise?bg=moon-mission puts it on the moon.

Why we trained our own

DrawDick did not invent the idea of a computer that draws these. A community project called dick-rnn trained exactly this kind of model and put a demo online. We tested it before building anything: it still runs, and in some cases it draws rather well.

But its trained weights, the numbers that make it draw what it draws, come with no license, and without one nobody else has permission to reuse them. So we trained our own from scratch, on openly licensed data, with training code we wrote ourselves. That is no criticism of the original: it is a fun piece of work, and it showed the idea was worth doing.

How a model draws a doodle

The model is a small sketch-rnn, a design published by David Ha and Douglas Eck at Google in 2017.¹ It never sees a picture. To sketch-rnn, a drawing is a list of pen moves: how far across, how far down, and then whether the pen stays on the paper, lifts, or the drawing is finished. Here is one real doodle the model drew, move by move. We picked this seed because a tidy doodle is the easiest to follow:

The same doodle’s first moves and its last, in the model’s own pixels (down is positive, so a negative “down” is up)

Move	Across	Down	Then the pen
1	-7.1	-53.3	stays down
2	-1.6	-43	stays down
3	3.7	-31.3	stays down
4	8.8	-19.6	stays down
…
33	16.6	-7.4	lifts
34	–	–	stops: done

One move at a time

At its heart is a long short-term memory network (an LSTM), a kind of neural network that reads a sequence one step at a time and keeps a running memory of it. Ours has 512 units of memory and 1,123,963 weights in all.

At each step it looks at the move it just made, and its memory of everything before, and predicts the next one. Not one answer, but a spread of possibilities: twenty overlapping guesses about where the pen goes next (technically, a mixture of 20 Gaussians), plus the odds that it stays down, lifts or stops. Then the computer rolls the dice, picks a move from that spread, draws it, and feeds it back in. Repeat until the model says it is done.

So every doodle is different. Give the dice the same starting number (a "seed") and you get the same doodle back, which is how the figures on this page were made.

Temperature: how adventurous

Before the dice are rolled, that spread can be sharpened or flattened: the temperature. Turn it down and the model sticks to its likeliest moves; turn it up and it takes long shots, livelier until the long shots win and the doodle falls apart.

There are really two dials. One says how freely the model picks among its twenty guesses, and whether the pen lifts or stops: which way the doodle goes. The other says how far each move may stray from the guess it picked: how wobbly the line is. sketch_rnn.js, the code that runs the model, turns them together by default. Surprise me sets them apart: it picks the guess almost greedily (at 0.15), so the doodle keeps to the model's main plan, and lets each move wobble at 0.25. In our blind tests that alone made more of its doodles good, from about 60% to 84%.

Here is the same hand-picked seed both ways. With one dial, it is speckled even at 0.1, in pieces at 0.25 and gone by 1. Held to its plan, a higher temperature only loosens the line.

One dial: the guess picked at 0.5 + temperature ÷ 2, as sketch_rnn.js does by default

Surprise me’s way: the guess always picked at 0.15

How we trained it

The data is the Quickdraw appendix, by the design studio Moniker: 25,209 doodles in the same format as Google's Quick, Draw! dataset, released under the Creative Commons Attribution 4.0 license.² According to Moniker, the first 10,000 were collected through Amazon's Mechanical Turk, and the rest came from their 2019 project Do Not Draw a Penis. We used the simplified version of the file, in which each drawing's lines have already been simplified to fewer points.

Cleaning and the split: we left out drawings with fewer than 10 or more than 200 moves, which kept 25,201. Those were shuffled and split three ways: 22,681 to train on, 1,260 to check the model against during training, and 1,260 held back for a final test.

The run: the model practiced on batches of 100 doodles, each stretched a little and with a few points dropped so it could not just memorize them (both tricks from the paper). Every 1,000 steps it was scored on the doodles it had never trained on. It stopped improving at step 15,000, so training stopped at 19,000. That took 31 minutes on an Apple M4 laptop, using its own graphics chip: no cloud computer, no money spent.

Then, standing them up: only about a third of Moniker's doodles are upright, shaft up and balls at the bottom. The rest lie on their side, lean or hang upside down, so the model drew every way up, and sometimes hung a shaft downward from a pair of balls. So we turned each training doodle upright, by the first of sixteen fixed angles that made it so (20,591 of 22,681 could be), and trained the model a little more on those: 13 minutes more. Then we trained it on the same upright doodles for 5 minutes more, starting its learning rate high again, with no new drawings. That is the model the site uses. It is a single 3 MB file, about 1.9 MB as your browser downloads it.

How good is it?

The model can also carry on from lines it is given, which is how Finish it works and how we test it: the same starting lines, the same seeded dice, many attempts per start, every result labelled good, partial or bad. The labels are given blind: the drawings being compared are shuffled together under random codes, and two separate judges (AI vision models, following a written rubric) label each one without knowing which model or setting drew it.

Sixty attempts per start for each, as the site shows them. "Before" is the first model as the site first sampled it, and it was already well ahead of the community model from a shaft or half a shaft. "After standing them up" is three changes together: the near-greedy dice above; standing the training doodles up; and, for Finish it, drawing up to four completions and keeping the first with about the right amount of ink for the lines it was given, since the bad ones either stop almost at once or scribble on and on.

The extra 5 minutes came later. In two more blind tests on fresh seeds, the model with them was good 85% of the time over all four starts, and the model without them 82%. It drew better from a shaft (83% against 75%), and no start was clearly worse. The difference is small enough that it could be chance, but it was in the same direction both times.

Standing the doodles up has a cost. The model now draws upright doodles, nearly nine in ten of them, and with less variety in shape than before. And if you draw a shaft lying on its side and ask Finish it to finish it, it does poorly: about one good completion in forty, where the first model managed one in four.

One detail mattered more than we expected. The training doodles had been simplified, so the model expects any lines it is given to be simplified the same way. Fed lines simplified differently, it did noticeably better from a shaft and worse from balls, so anything that hands it someone's lines has to simplify them as the training data was.

Where it goes wrong

Here is the evidence: the first twelve seeds as Surprise me draws them, nothing picked out.

• Sometimes it scribbles where the balls should be, going round and round the same loop. Picking its guesses almost greedily makes that more likely: having started a loop, it keeps choosing the likeliest next move, which is more loop.
• Sometimes it never stops. So every doodle from nothing stops at 250 moves, a little over the longest training drawing, and every Finish it completion at 100, far more than a good one needs. Drawing from nothing, about 1 in 17 doodles runs to that limit, and about 1 in 7 either does or scribbles on.
• Sometimes the balls sit apart from the shaft, or one is much smaller than the other.

Surprise me checks every doodle before you see it. One that ran to the limit, scribbled on, went to speckles, or came out too small or thin is thrown away, and the model draws again from the next seed, up to six times. Above, that happened once in twelve, a little less than usual: over 200 seeds, it was about one in seven.

How it runs on the site

• Only when you ask. None of it loads on your first visit. It is fetched the first time you press Surprise me (or open the link), and your browser keeps it.
• In the background. It runs in a Web Worker, a separate thread, so the page stays smooth. Working out a doodle takes a fraction of a second; the drawing-in is for show.
• On your device. Surprise me draws from nothing, so it needs nothing from you, and nothing you draw is sent anywhere for it. Beyond fetching the model from DrawDick's own site, all it sends is a count that a surprise was drawn, and whether a drawing was started from it: the same cookie-free tally the site keeps for every ready-to-draw link.
• Unchanged code. It runs on Magenta's open-source sketch_rnn.js,³ unchanged, on a small math library DrawDick wrote to replace one the site's security settings forbid. It draws the same seeded doodles either way.

It does not judge your drawing or learn from it. It draws one doodle when you ask, and the rest is up to you.

Credits and licenses

• The training data: drawing model trained by DrawDick on the Quickdraw appendix by Moniker, used under CC BY 4.0. That credit is in Surprise me's bar, under every doodle it draws, too.
• The runtime: sketch_rnn.js from Google's Magenta demos, Copyright 2017 Google Inc., under the Apache License 2.0.
• The design: David Ha and Douglas Eck, "A Neural Representation of Sketch Drawings" (2017).¹ The model and its training code are DrawDick's own.

DrawDick's own art and words are made with AI tools, and this article was drafted with them too, then checked by a person before publication, like every article: how the Dicktionary is written says more. The doodle model is something different: a small model we trained ourselves, on doodles that Moniker collected and shared under an open license.

Sources and notes

1. David Ha and Douglas Eck, "A Neural Representation of Sketch Drawings", arXiv:1704.03477 (2017). arxiv.org ↩ ↩²

2. Moniker, Quickdraw appendix, README and license (CC BY 4.0). github.com ↩

3. tensorflow/magenta-demos, sketch-rnn-js, Apache License 2.0. github.com ↩