Comonads are neighbourhoods, not objects

Gabriel Gonzalez, the author of the well-known pipes package, has written a blog post in which he claims that comonads are objects. I think he is mistaken, but I can't blame him; I used to be quite confused about comonads myself.

Comonads are not objects

To motivate the purported equivalence between objects and comonads, Gabriel gives detailed implementations of three classic object-oriented patterns (Builder, Iterator, and Command). Then, he reveals that all three implementations are comonadic. His examples are contrived, but more importantly, they are also misleading! Although not a direct quote from his post, he clearly intends the Haskell session

>>> t1 = thermostat 3
>>> t2 = t1 # up'
>>> t3 = t2 # up'
>>> toString t3
"5 Kelvin"

to be equivalent to the following Java session.

>>> t = new Thermostat(3);
>>> t.up();
>>> t.up();
>>> t.toString();
"5 Kelvin"

With this particular sequence of methods, the two sessions indeed compute the same result, but this is only because the up and down methods commute with each other. By adding the method square to the list, the behaviours of the two sessions quickly diverge. Strangely enough, in the Haskell version, each new method call gets applied before all the previous calls!

Haskell version (inverted method call order)

>>> t1 = thermostat 3
>>> t2 = t1 # up'      -- 3+1
>>> t3 = t2 # up'      -- 3+1+1
>>> toString t3
"5 Kelvin"
>>> t4 = t3 # square'  -- 3*3+1+1
>>> toString t4
"11 Kelvin"
>>> t5 = t4 # up'      -- (3+1)*(3+1)+1+1
>>> toString t5
"18 Kelvin"

Java version (normal method call order)

>>> t = new Thermostat(3);
>>> t.up();            // 3+1
>>> t.up();            // 3+1+1
>>> t.toString();
"5 Kelvin"
>>> t.square();        // (3+1+1)*(3+1+1)
>>> t.toString();
"25 Kelvin"
>>> t.up();            // (3+1+1)*(3+1+1)+1
>>> t.toString();
"26 Kelvin"

I hope this counter-example convinces you that comonads are not objects. But if that is true, what are comonads, then? What use do we have for methods which get applied in reverse chronological order?

Comonads are neighbourhoods

The answer, I believe, is that a comonadic computation is similar to a cellular automaton. At each step, the computation for a cell computes its next value based on the value of its neighbours at the previous step. My interpretation of

extract :: w a → a

is that w a is a neighbourhood containing a number of values of type a, at various locations around the current cell. A given comonad may provide functions to inspect, say, the neighbour immediately to the left of the current cell, or the neighbour from one timestep ago, but the bare minimum is that it must be possible to extract the value of the current cell itself.

Similarly, my interpretation of

extend :: (w a → b) → w a → w b

is that w a → b computes one step of the cellular automaton by examining a local neighbourhood and producing the next value for the current cell. This single-cell computation is then extended to the entire neighbourhood, updating each cell as if it was the current one.

Comonadic thermostat

In our thermostat example, there is one cell for each possible temperature, and the value of each cell is also a temperature.

So far, I have stuck to Kelvin degrees in order to minimize the number of moving parts, but now that we have two completely different uses for temperatures, let's follow Gabriel by using Celsius for the cell contents, and Kelvin for the cell labels.

Since temperatures are floating point values, our thermostat operates on a neighbourhood which is continuous instead of discrete. Continuousness is also the premise of the above Game of Life variant. Unlike comonadic computations, which apply their operations one at a time, the above cellular automaton also operates on a continuous time domain. I encourage you to watch the video, it looks awesome.

If our thermostat had an object-oriented implementation, each operation would simply modify the current cell by applying the operation to the Celsius temperature it contains. In fact, if this was an object-oriented implementation, we would only need one cell! Since this is, instead, a comonadic implementation, the operation is instead applied to the Kelvin temperature which labels the current cell. The resulting Kelvin temperature is interpreted as a pointer to another cell, and the final result is the Celsius contents of that cell.

When the available operations are restricted to just up and down, this scheme works just fine. Originally, each cell contains the Celsius equivalent of their Kelvin label. Then, each time the up method is applied, each cell takes on the Celsius value of the cell one unit above it, which effectively increases the temperature by one everywhere. But this only works because up and down preserve the invariant that cells which are one Kelvin unit apart contain values which are one Celsius unit apart!

One way to visualize this is to imagine a sheet of graph paper annotated with Celsius temperatures, over which we overlay a transparent sheet of graph paper annotated with Kelvin temperatures. One of the transparent cells is our current cell, and we can read its Celsius contents through the transparent paper. When we call up and down, we offset the two sheets of paper from each other, causing the apparent contents of all the cells to increase or decrease simultaneously.

Reverse chronological order

Similarly, if each cell contains its own temperature and we apply the square method, each cell squares its Kelvin temperature, looks up the corresponding cell, and ends up with the (Celsius representation of the) square of its original Kelvin value.

But if we apply up after the temperatures have already been squared, the values will change as if the values had been incremented before they were squared! How is that even possible? What is the secret of this reverse-chronological method application?

If there was only one cell, anti-chronological application would not be possible because the original, unsquared value x would been lost by the time the up operation was applied. Our comonadic implementation, however, has a lot of cells: in particular, one unit above the current cell, there is a cell whose unsquared value was x+1. Since the square operation affects all cells uniformly, that cell now contains the squared value (x+1)². Therefore, it is no surprise that when the up operation offsets the two sheets of paper by one unit, the old x² value scrolls out of view and gets replaced by (x+1)² instead.

Ironically, Gabriel's original implementation of up' was applying its operation in the correct chronological order.

>>> up' (t, f) = (t + 1, f)

But later on, while trying to shoehorn his object into the comonadic mold, he proved that the correct definition should have been as follows.

>>> up' (t, f) = (t, \t' → f (t' + 1))

While that is indeed a correct comonadic implementation, this is not a correct object-oriented implementation because it applies its operation in reverse chronological order. In particular, notice that it modifies the input of f; since f applies all the other operations which have been accumulated so far, applying the new operation to its input has the effect of inserting that operation before all the others.

Visiting the neighbours

Why do comonads apply their operations in reverse chronological order? Is that ever useful?

In fact, this reversed order business is not the full story. In a typical comonad, the cell labels and their contents would not both be temperatures. Instead, if we were trying to implement a one-dimentional cellular automaton such as rule 30, we would label the cells with integers and their contents with colors.

Wolfram's rule 30 automaton, from his
controversial book A New Kind of Science.

Now that the input and output types are distinct, we can see that \t' → f (t' + 1) is not even attempting to modify the contents of the current cell, neither chrono- nor anti-chronologically. Instead, it is modifying the index of the cell on which f applies its computations, thereby allowing us to observe which color has been computed for a neighbouring cell. This is, of course, the first step to compute the color which the current cell will take on the next step.

Once we have wrapped the steps necessary to compute our next color into a function of type w Color → Color, we can extend this function to all the cells in our simulation, modifying the entire row at once by modifying f.

Objects are monads

We have seen that comonads are not objects, but are instead neighbourhoods in which cellular automata can evolve. But that is only half of the question. If comonads are not objects, then what are objects? Is there another way to represent objects in Haskell?

There are many facets to modern object-oriented programming, but the aspect which I think Gabriel was aiming for in his post is objects as encapsulated data plus a bunch of methods to modify this data. To me, those features don't sound comonadic at all; rather, I would implement them using a State monad!

>>> type Thermostat a = State Double a
>>>
>>> getKelvin  = get
>>> getCelsius = fmap (- 273.15) getKelvin
>>> 
>>> toString = do c ← getCelsius
>>>               return (show c ++ " Celsius")
>>> 
>>> up   = modify (+1)
>>> down = modify (-1)

Since we're using monads, the syntax for calling a few of those methods one after the other is just do-notation, which is even shorter than the this # notation advocated by Gabriel.

>>> up3 :: Thermostat String
>>> up3 = do up
>>>          up
>>>          up
>>>          toString

Notice the type of the above code: Thermostat String doesn't mean that there are many different kinds of thermostats, and that this particular code is using or producing a "string thermostat". Rather, the above code is an object-oriented computation having access to a single Thermostat object and producing a single String value.

Okay, so using one monad, we managed to represent a computation manipulating one object. How would we manipulate two thermostats? Using two monads? Yes! Using more than one monad at once is precisely what monad transformers are about.

>>> type ThermostatT m a = StateT Double m a
>>> obj1 = id
>>> obj2 = lift
>>> 
>>> up_both :: ThermostatT Thermostat String
>>> up_both = do obj1 up
>>>              obj2 up
>>>              s1 ← obj1 toString
>>>              s2 ← obj2 toString
>>>              return (s1 ++ " and " ++ s1)

This time we have an object-oriented computation having access to two Thermostat objects, again producing a String value.

Duality

Since monads and comonads are duals, we would expect comonads to also have transformers. To figure out what comonad transformers are, let's spell out the similarities and differences of this duality.

First, monads are computations. Comonads are also computations, but of a different kind. The goal of both kinds of computations is to produce a value, but their means to obtain it are different. The main difference is that monads can cause side-effects, which are visible at the end of the computation, while comonads can observe neighbours, which need to be given before the computation can begin.

One of the characteristics of monadic-style computation is that there is a distinguished statement, return, which produces a given value without causing any side-effects. The equivalent characteristic for comonadic-style computation is that there is a distinguished observer, extract, which returns the current value without observing any neighbour.

In addition to this distinguished statement, which all monads share, each monad instance typically offers a number of monad-specific statements, each producing their own special side-effects. Correspondingly, each comonad instance typically provides a number of observers, each observing their own special kinds of neighbours. You can also observe the shape of the neighbourhood; for example, if your automaton runs on a finite grid, you can check whether the current cell lies on the boundary of the grid.

With monad transformers, it is possible to construct a combined monad in which the special statements of both component monads may be used. And now we have it: with comonad transformers, it must be possible to construct a combined comonad in which the special observers of both component comonads may be used.

For example, if we have a 1D row of cells, that is, a comonad in which left and right neighbours are available, and another 1D comonad in which up and down neighbours are available, then the combined comonad is the cartesian product of its components: a 2D grid in which all four direct neighbours are available.

If you are still curious about comonads, comonad transformers, and the above 2D grid trick, more details are available in my Haskell implementation of Conway's Game of Life.

From game developer to game reviewer

For one month, I was a game developer. The month after that, I became a game reviewer. This month, I'd like to tell you how that went.


Developer, reviewer, storyteller; those three roles are more similar than they seem. In all three cases, I have an audience. That's the part you fill in right now. Also, each role is partly about deciding what to share: choosing the most interesting game features, the most entertaining games, or the best anecdotes. Here's an example.

When me and my teammates were designing our game, we envisioned teleporting blocks, entanglement badges, decaying walls, a lot of good game mechanics which we had to discard because we had a lot more ideas than we had time for implementing them. Our tight deadline was due to the fact that we were competing in the GitHub Game Off, a programming competition inviting teams and individuals to create a web-based game in one month.

Finishing a game in one month is harder than it sounds. In fact, the overwhelming majority of the contestants failed to deliver a playable game at the end of the month. All of these flops became problematic during my second role, as a game reviewer: if I had not been smart about this, I could easily have spent more time discarding failed projects than playing and reviewing actual games.

Dealing with an overwhelming number of possibilities is a common problem, which I'm sure you also have to face sometimes. Here's how to fight back.

Ask other people for advice

When we were working on our game, we knew that our goal was to make a game which players would appreciate. We therefore decided that our first priority was to create a small, but playable version of the game, so that we could show it to our friends and ask them what to improve.

I thought I would have had to compile many suggestions and identify the most frequent requests, but to my surprise, almost everybody reacted to the game in almost the same way! Sadly, this common reaction was incomprehension, because our game mechanic was very complicated and not very well explained.

It turns out I was so familiar with the game that I was blind to its complexity. Play-testing early was a great plan, as it allowed us to discover the problem early on and to improve the clarity of our game throughout its development.

Did we succeed? See for yourself by playing it here!

Look around for inspiration

After our game was complete, I compared it with the competition. As I played one game after another, looking for my strongest competitor, patterns started to emerge.

There was a recurrent hero: Octocat, the mascot of GitHub. The contest page featured a few Octocats dressed as various game characters, so it was an obvious choice.

Our game was also using an Octocat until the last day of the competition, when we learned that this was against the rules! We changed the artwork, renamed the game, and then the server refused our changes because we ran out of disk space, something which shouldn't even happen on GitHub. It was a very dramatic climax at the end of the race.

Practice

Another common occurrence was games which were clearly aiming to emulate existing well-known games, like Osmosis and World of Goo. Imitating a game concept which has already proved to be a lot of fun sounds like a winning strategy, but it isn't, because games are about mastering new skills. In this case, however, I suspect it was not the players who were supposed to learn new skills.

When I first learned to code, I wrote a Pac-Man clone. Then I wrote a second one, to see if I could get the ghosts to animate smoothly instead of jumping from cell to cell. Then I wrote a Mario clone, and so on and so forth.

We game developers learn how to make games by copying the work of the masters, much like musicians begin by learning how to play famous songs before they become ready to compose their own.


There were also many documents in which contestants described their dream game in detail, without actually reaching the point where they would start implementing it for real. Again, I've been there. When I was a kid, I would draw detailed, life-size Mario-style levels... and then I would lay down the pages on the floor and jump on them. Always play-test your game as early as possible, kids!

I think the trend here is that learning to develop games is a long process, and those artifacts represent the different stages you have to go through before you become ready to create your own original games. So sure, as a game reviewer I have to say that the clones didn't introduce enough novelty and that a design document is in no way a substitute for a game. But as a fellow game developer who has been through those stepping stones, I say keep up the good work! Don't give up, you're on the right track.

Use tools

After comparing my game with all those clones and design documents, I was sure that we were going to win... but I was wrong. I had not compared our game against strong enough competitors.

To find my strongest competitor, I was willing to play through many clones and incomplete prototypes, but I quickly grew tired of the non-playable entries.

To help me on my quest, I used the GitHub API to obtain high-level information about all the contestants. I tried to filter the entries by code size and by the date on which the contestants had stopped working on their game, but neither metric clearly separated the games from the flops. Instead, I simply eliminated the entries which did not provide a URL at which their game could be played, and that turned out to be enough.

After encountering many comments on the internet from potential players who couldn't find the games among all the flops, I realized the value of my reduced list, and I decided to publish it.

Be persistent

Even with the list reduced to a tenth of its original size, there was still a large number of games to go through.

I kept looking at more competitors because every ten games or so, I would encounter a game which was genuinely fun and original, and this would motivate me to keep looking for more gems.

The fact that good games were so rare made me realize that publishing a list of all the games, even after filtering out all the flops, was not going to be very helpful. I had to sort the list! It is at this point that I took on the role of an amateur game reviewer.

Drop information

Comparing the games turned out to be more challenging than I had imagined. Among the top contenders, there was no game which was better than the others in every single respect; rather, I would find one game with impressive graphics, another game with stimulating music, and another one with an original new mechanic. It was like comparing apples and oranges.

Professional game reviewers often solve this issue by rating each game aspect in isolation. They then combine the individual ratings into one unified score, using a fixed weight for each aspect. That's a good idea, but with more than 150 games to review, I needed a more expedient scoring method!

I ended up separating the games into categories, then sorting the categories. For example, all the games with good ideas but poor execution ended up in one basket, and then I only had to decide whether I favoured games with great execution or games with great ideas. When I published the list, I concatenated all the lists and removed the category names, so that I wouldn't hurt anyone by telling them that their game was in the "poor execution" category.

Conclusion

Only sorting the baskets meant that games belonging to the same category would appear in an arbitrary order, falsely conveying my preference for one over the other. I decided that this did not matter. After all, my goal was not to share my preferences, but to help gamers find the most worthwhile games. No order I could have came up with would have precisely matched the preferences of all my visitors anyway.

In fact, once the results of the contest came out, I discovered that my preferences were even less representative than I thought. Some of the games which the judges decided to highlight had appeared very low on my list, and conversely, some of the games at the top of my list were not mentioned at all.

Do you agree with the choices made by the judges, or is it my list which most closely matches your taste? Check out the official GitHub Game Off results here, and my much longer list of all the games here.

Every single game from GitHub Game Off 2012

From the 1285 forks of GitHub's official game-off repository, only 182 contestants bothered to change the URL at the top of their repo page. Of these, only about 140 URLs actually point to a game you can play. Not many of those games are worth playing.

I know this because I have used GitHub's API to obtain basic information about all the forks. And because I have visited all 170 non-blank URLs, and played every single game. And now, so can you!

Well, I don't really recommend trying all 170, but for your convenience, I tried to list the most worthwhile games at the top. Obviously, my opinion as to which games are the best might not match yours, but as you go down the list, if you encounter a point where every single game feels sub-par, you should be able to stop and be confident that you are not missing a rare gem hidden much further down. That being said, if you want to maximize your enjoyment, better start by the end, and play the most polished games last!

Enjoy.


(full disclosure: my own game entry is somewhere in that list.)

1. linkboy992000/game-off-2012
   challenging, polished puzzle game about cloning. story and music! Do *not* press 'A'.
2. svenanders/jetmanjr
   hard exploration platformer.
3. searlm/game-off-2012
   virus-themed shoot-em-up where you need to balance progress against collecting ammo.
4. redmanofgp/game-off-2012
   hard, but fun shoot-em-up game where you can concentrate your mind power.
5. gamefrogs/game-off-2012
   a puzzle version of pipe dream, on an hexagonal grid.
6. flypup/game-off-2012
   short fighting game in which your opponent spawns copies of himself.
7. ondras/star-wars
   hard, fun, ascii fighting game.
8. jpfau/scrum
   a mix between a kind of tetris and a 3D shoot-em-up. on a GBA emulator.
9. mindd-it/game-off-2012
   bird-themed single-button avoid-the-obstacles game
10. wtjones/game-off-2012
    unique clone-climbing game.
11. fragcastle/rock-kickass
    megaman-style game in which you steal the abilities of normal enemies
12. sdrdis/hotfix
    single-button platformer with very nice increasingly difficult yet randomly-generated levels. upgrades are way too expensive.
13. lulea/game-off-2012
    3D sokoban
14. adhicl/game-off-2012
    puzzle game with a unique clone-and-lure mechanic
15. KriScg/Waveform
    circuit-simulator puzzle game.
16. 502Studios/game-off-2012
    mine-themed sokoban-style puzzle game.
17. RonanL/game-off-2012
    a nice platformer about escaping from your clones.
18. mvasilkov/game-off-2012
    typing game with gratuitious physics, space invaders, music, anime, oh my!
19. Eugeny/foku
    an exceedingly pretty, but hard to control game about a magic fork.
20. tapio/plasma-forks
    a first-person shooter.
21. visualjazz-ngordon/Play-dot-to-dot
    short connect-the-dot game.
22. tsubasa-software/game-off-2012
    pirate themed obstacle-avoiding game.
23. duncanbeevers/game-off-2012
    over the top maze game.
24. gelisam/game-off-2012
    hard sokoban variant with a confusing rewinding mechanic.
25. loktar00/game-off-2012
    bejeweled-style game with a resource management side.
26. AD1337/ForKingGame
    hard physics-based platformer.
27. Seldaek/split
    obstacle-avoiding game with a unique splitting mechanic.
28. dakodun/game-off-2012
    strategy game with a unique unit-pushing mechanism.
29. volrath/game-off-2012
    a shoot-em-up game with powerups.
30. cdata/solstice-submarine-ctf
    capture-the-flag game with an underwater theme
31. ViliusLuneckas/Pipe-slime-adventure
    pipe-themed avoid-the-obstacles game.
32. gbatha/PolyBranch
    3D avoid-the-obstacles game in a tunnel.
33. kicktheken/game-off-2012
    an isometric exploration game in which you accumulate resources of different types.
34. zombiebros/game-off-2012
    rambo-themed shoot-em-up
35. Jidioh/SkyScaler
    short labyrinth with a world-flipping mechanic.
36. lazor-base/fused-holiday
    a platformer in which you can push and pull crates.
37. AntPortal/game-off-2012
    isometric quiz questions about git.
38. Dover8/game-off-2012
    a puzzle where actions in one world affect the other copy.
39. icebob/game-off-2012
    3D pong.
40. Zolmeister/avabranch
    hard to control fork-and-avoid-the-obstacles game.
41. eric-wieser/game-off-2012
    a variant of snake in which you can divide and control multiple snakes simultaneously.
42. incorrectangle/game-off-2012
    astronaut-splitting, blob-merging puzzles. too short. a bit buggy.
43. cnnzp/game-off-2012
    short track-building puzzle game.
44. nojacko/game-off-2012
    strategy game in which you need to defend your buildings from the zombies.
45. thehen/game-off-2012
    exploration game based on a World-of-Goo-style building mechanic. quite short.
46. rozifus/game-off-2012
    hard sokoban variant with a color merging mechanic
47. lantto/game-off-2012
    unique clone-spotting and killing game.
48. jlongster/octoshot
    3d shooter on a small map
49. RothschildGames/release-cycles
    circular avoid-the-obstacles game
50. appleskin/game-off-2012
    block-carrying puzzle game.
51. etamm/game-off-2012
    top-down zombie shooter with interesting alter-your-world mechanic.
52. begillespie/cloned-game
    move in both copies of the world. Again.
53. fengb/game-off-2012
    a unique drawing puzzle game, once you finally figure out what you're supposed to do.
    hint: click on the black strokes.
54. sjthompson/game-off-2012
    a strategy game in which units are spawned off other units, not buildings.
55. xSmallDeadGuyx/game-off-2012
    a nice little light-bot-style game.
56. Gagege/game-off-2012
    unique, fast-paced memory game.
57. notsimon207/game-off-2012
    physics-based platformer, clearly inspired by Gish. too hard to control.
58. danfischer87/game-off-2012
    hard git-themed puzzle game. a bit buggy. too short!
59. jeffreypablo/game-off-2012
    fighting game. terrible graphics, buggy, but the gameplay is there!
60. scriptfoo/game-off-2012
    a game about learning Javascript. Incomplete? I got stuck after the toLowerCase() quest.
61. petarov/game-off-2012
    distract your opponents while you pick up carrots. strangely-placed controls.
62. Dave-and-Mike/game-off-2012
    short alien-cloning game. strangely-placed controls.
63. murz/game-off-2012
    top-down shooter with multiple gun types.
64. jsonsquared/game-off-2012
    multiplayer top-down shooter.
65. DangerMcD/game-off-2012
    unique multitasking, block pushing game.
66. Choctopus/game-off-2012
    guitar-hero-style game.
67. gilesa/game-off-2012
    unique obstacle-avoiding game about writing software.
68. vladikoff/game-off-2012
    complicated shooter.
69. Chleba/game-off-2012
    short starwars-themed fighting game
70. JasonMillward/game-off-2012
    hard obstacle-avoiding game.
71. denniskaselow/game-off-2012
    asteroids-style game.
72. mapmeld/game-off-2012
    missile command clone with an interesting "change the rules" mechanic.
73. onethousandfaces/game-off-2012
    bonsai-growing pseudo-game. would be an interesting mechanic if there was a goal shape.
74. vrgen/game-off-2012
    car-themed avoid-the-obstacles game
75. gitdefence/game-off-2012
    complicated allele-sharing tower-defence game
76. ChickenChurch/game-off-2012
    punch-the-obstacles game with annoying controls
77. AreaOfEffect/game-off-2012
    food-themed shoot-em-up
78. asswb/game-off-2012
    a bug-tracker simulator, where you solve issues by being patient.
79. eleventigers/echo
    hard 3D platformer about capturing sounds. too hard for me.
80. forrestberries/game-off-2012
    a online multiplayer version of the board game "Apples to apples".
81. NetEase/game-off-2012
    diablo-style game. not worth the very long loading time.
82. DancingBanana/game-off-2012
    platform game with a chronotron-like cloning mechanic. SaveTheClones, below, is the same idea but with more levels and... different graphics.
83. lrem/kobo-san
    sokoban variant in which some blocks can be pulled.
84. SUGameOffTeam/game-off-2012
    hard tank-wars clone with a kitchen theme. only one level.
85. jisaacks/game-off-2012
    food-themed asteroids-style game.
86. pce/game-off-2012
    short dream-themed obstacle-avoiding game.
87. scurryingratatosk/game-off-2012
    two player Qix variant.
88. bverc/miner-trouble
    sokoban variant with gems and bombs.
89. Psywerx/game-off-2012
    obstacle-avoiding hipster-themed game.
90. AmaanC/TinyWings
    tiny wings clone.
91. Andersos/Meatballs
    meatball-themed obstacle-avoiding game.
92. yosun/game-off-2012
    propel clones at your enemies. instructions would have been useful.
93. Popcorn-Web-Development/Beaver-Words
    beaver-themed typing game.
94. lessandro/dave
    short plaftormer with diamonds and a jetpack.
95. dakk/game-off-2012
    nyan-cat themed obstacle-avoiding game.
96. binary-sequence/game-off-2012
    firefighting game.
97. condran/game-off-2012
    shoot-em-up with very limited ammo.
98. Jacic/game-off-2012
    platform game with a chronotron-like cloning mechanic
99. sourrust/game-off-2012
    double-jump platform game. only one level.
100. MonsterGuden/game-off-2012
     single-button puzzle platformer
101. JamieLewisUK/game-off-2012
     shoot the balloons for points.
102. JuggleTree/game-off-2012
     catch fruits and throw them into a basket.
103. mkelleyjr/game-off-2012
     snake clone.
104. dafrancis/SpaceHam
     non-sequitur-themed shoot-em-up.
105. playtin/game-off-2012
     wario-ware-style mini-games.
106. gplabs/game-off-2012
     a tetris variant with more annoying controls. use space to attach/unattach to a piece.
107. jimmycuadra/pushing-hands
     bejeweled variant where you swap whole rows at a time.
108. doowttam/game-off-2012
     unique fix-the-pattern game.
109. sunetos/game-off-2012
     unclear body simulation game.
110. dicksontse/game-off-2012
     snake, without the snake.
111. imagentleman/hackris
     original, but pointless incorporation of cloning and pushing into a typing game.
112. brooss/game-off-2012
     answer text-questions, and hang out in a chat room.
     
     
     
     Games after this point did not make it into the list, but maybe it's just me.
     
     
     
     Not-quite-games
     
     I don't agree that those entries count as "games", so I couldn't compare them fairly.
     
     
113. timehome/game-off-2012
     a clone of robocode; which, as you know, is not a game, but an AI competition.
114. pkukielka/radiance
     a psychedelic experience, to be sure, but is this a game?
     
     Annoying
     
     I could not stand playing those games for long enough to give them a fair comparison.
     
     
115. mosowski/game-off-2012
     a 3D game designed to give you motion sickness? why??
116. gnius/droplet
     avoid the obstacles. annoying alert bomb each time you die.
117. abrie/game-off-2012
     unique finger-twitching game with intentionally irritating controls.
118. ess/game-off-2012
     platformer with intentionally irritating controls. too hard and annoying for me.
     
     Buggy
     
     I could not play those games either.
     
     
119. heisenbuggers/game-off-2012
     buggy boggle variant.
120. Dlom/game-off-2012
     buggy duck shooter with a silly intro.
121. TARGS/game-off-2012
     buggy bejeweled variant.
122. CalPolyGameDevelopment/ettell
     mini-games
123. shinriyo/game-off-2012
     3D choose-your-character and sink-into-the-ground?
124. dawicorti/helping-pibot
     you're supposed to be able to create new pieces, but they get messed up.
     
     Technical difficulties
     
     I am not entirely sure that those games would also fail on your computer.
     
     
125. Finjitzu/Archetype
     placeholder?
126. nuclearwhales/push-the-box
     doesn't work, even with "chrome native client" enabled and relaunched?
127. drabiter/magnet
     requires access to my computer to run?
128. py8765/game-off-2012
     abandoned Draw Something clone?
129. MS-game/game-off-2012
     maybe my Java player is too old?
130. reedlabotz/game-off-2012
     the "Create new game" button doesn't do anything?
131. OpenSismicos/game-off-2012
     applet requesting access to my computer?
132. wprogLK/4thDimensionGame
     failed to run.
133. jamescostian/game-off-2012
     got a blank page, but this is supposed to use a 3D library?
134. BumbleBoks/game-off-2012
     unclear path-drawing game.
135. prgnization/game-off-2012
     you're supposed to be able to chat (as a core game mechanic!), but the text field doesn't respond to my keyboard.
136. elmariofredo/game-off-2012
     the video shows a working level, but I my character doesn't go past the zeroth level.
137. ThatsAMorais/game-off-2012
     a strategy game, but the units don't move unpredicably?
     
     Incomplete
     
     I couldn't play those games until the end, which is sad, because some were very promising.
     
     
138. publysher/game-off-2012
     incomplete text adventure about eating a steak.
139. MikeRogers0/game-off-2012
     short platformer with no ending. interesting shoot-your-own-platforms mechanic.
140. SoftlySplinter/game-off-2012
     a shoot-em-up without things to shoot.
141. mhluska/game-off-2012
     incomplete olive bouncer.
142. Annovean/game-off-2012
     incomplete osmosis clone.
143. leereilly/follow-dem-game-off-forkers
     item collection game with no way to lose.
144. ozh/alchemy
     doodle-god clone
145. Blipjoy/game-off-2012
     incomplete UFO-themed object-dragging game?
146. FluffyCode/game-off-2012
     octopus-in-the-desert-themed top-down shooter
147. ImmaculateObsession/game-off-2012
     some level editor with no way to play?
148. devNil/game-off-2012
     a so-called "inverted tower-defence" which you can win by repeatedly clicking the "warrior" button.
149. EpicFailInc/game-off-2012
     destroy walls using bombs and lose HP for no reason.
150. ihcsim/game-off-2012
     top-down shooter in which you can't win nor lose.
151. gcoope/game-off-2012
     hang-glider inverse shoot-em-up with no way to win.
152. vespertinegames/game-off-2012
     prototype tile-based movement.
153. wskidmore/game-off-2012
     for a game in which you have "endless destructive powers", there sure aren't many things to destroy.
154. jamestomasino/game-off-2012
     just a grid.
155. freejosh/game-off-2012
     the engine for a platformer.
156. dparnell/game-off-2012
     a maze, but you no way to explore it.
     
     Access denied
     
     Maybe it's just a server configuration issue?
     
     
157. superally/game-off-2012
     access denied.
158. DarkForestGames/game-off-2012
     403 forbidden.
159. lazyeels/game-off-2012
     forbidden
     
     Abandoned
     
     Some people wrote down their game ideas, but never got around to implement them. Others picked a URL, but never uploaded anything there.
     
     
160. strugee/game-off-2012
     abandoned placeholder.
161. cmdkeen/game-off-2012
     abandoned game concept.
162. Willshaw/game-off-2012
     abandoned game concept.
163. Vbitz/game-off-2012
     placeholder.
164. EvilSpaceHamster/game-off-2012
     page not found
165. fabriceleal/game-off-2012
     abandoned placeholder
166. CodingEffort/game-off-2012
     page not found
167. Jorjon/game-off-2012
     page not found
168. mbl111/game-off-2012
     placeholder
169. matthewtole/game-off-2012
     404
170. will3942/game-off-2012
     placeholder

A little bit of Waterfall

Agile, what have you done! I used to believe in you.

There I was, happily finishing iteration N of my online board game library, unaware of the terrible event which would shatter iteration N+1. While thinking about which features I would like to add next, I stumbled upon a major feature for which I was not ready yet, and probably never will! Not with this codebase, anyway.

This was a board game library. I was working on a board game library because recently, I have been playing a lot of digital board games on my iOS devices. I was mostly playing them in the subway. So, it would make sense to assume that a major use case for my online board game library would be for a group of friends to play together in the subway, or on the train, while waiting to get to their destination.


The problem is, there is no internet in the subway.

My codebase was an HTTP server written in Haskell, so I pretty much had to scrap the project and start anew. I'm trying node.js this time, this should allow me to use the same game logic on the server (for distributed games), as on the browser (for local games).

The point of this post is not that node.js is better; I don't know that yet. The point of this post is that if I had done a bit more planning at the beginning of the project, the "also works offline" requirement would have jumped at me as being the serious technology-limiting, can't-do-it-in-Haskell-then factor that it is. I didn't spot it early because I didn't plan that far ahead: being a good Agile developer, I only planned the features I could complete within the first iteration.


Clearly, I am not going to ditch Agile on the first offence, but next time, before I even begin iteration 1, I am certainly going to spend some time collecting requirements. And throwing them in the garbage. After all, the details are clearly going to change midway during the project, that's why we use many iterations. But by looking at a large enough sample of example requirements, I should be able to pick a more appropriate technology next time.


That is, next time, I will put a bit of Waterfall in my Agile wine.

Agile vs Scrum

Recently I have been thinking more abstractly about what Agile is, and why (or whether) it is good. The first thing I have discovered is that I was not, in fact, following Agile's practices, but Scrum's!

The difference between the two is less subtle than I thought.


According to the Scrum guide, Scrum consists of "roles, events, artifacts, and the rules that bind them together. [...] Scrum’s roles, artifacts, events, and rules are immutable and although implementing only parts of Scrum is possible, the result is not Scrum."

That is, Scrum has rigid rules which cannot be broken, mandatory meetings with strict time limits, and long lists of responsibilities for each role.


According to the Agile Manifesto, Agile is a short list of four value judgments.

That is, Agile lets you make your own decisions, but leads you in the right direction by (1) emphasizing the important goals, and (2) reminding you that achieving those goals may require you to sacrifice other goals.


Clearly, Agile is much more flexible than Scrum, while Scrum is much more complete and precise. Since I like to tweak my workflow from iteration to iteration, I think Agile is more appropriate for me.

Surprisingly, Agile by itself doesn't even mandate the work to be divided into iterations, and neither of them requires the work to be divided into stories! So... whose recommended practices have I been following, then?

I feel like there is a third contender which I have yet to discover.

Specification aphorism

I would rather have an imperfect program which does the wrong thing than a perfect specification which doesn't do anything.


That is all.

Thank you, KeyRemap4MacBook!

At long last! I managed to fully reproduce my Linux xmodmap layout on OS X.


I don't think I have particularly peculiar typing needs. I often type code, which frequently uses characters like "[", "{", and "<". Those are easy to type using the US keyboard layout. I also often type French, which frequently uses accented characters like "é", "à", and "û". In that situation, most people opt for switching between keyboard layouts depending on the situation. I, however, opted for a custom keyboard layout.

My custom keyboard layout is mostly the US keyboard layout, plus some Alt+key combinations for dead keys like "´", "`", and "^". On Linux, this was easy to setup using xmodmap. Implementing the keyboard layout on OS X was a bit more complicated, because its XML representation of keyboard layouts is very long and it's not easy to test changes incrementally, but I eventually managed to do it.


Then one day, I decided that the modifier keys were too low.

I am not even using Emacs, but typing Ctrl with my pinky is really twisting my hand in a way I don't like. So I bought an ergonomic keyboard. It didn't help.

One of the reasons I chose this particular keyboard was because of the two little extra arrow keys below the space bar. I thought I could remap them to Ctrl and Shift, thereby allowing me to type those troublesome modifiers using my thumb instead of my pinky. That didn't work either. The keyboard preferences which were installed with the keyboard helpfully allow me to remap those to any other key... except modifiers. No luck directly modifying the layout files either, not even through keyboard layout creation tools like Ukulele. I did manage to turn Caps Lock into an extra Ctrl, but the other modifiers looked like they would be stuck in place forever.


I had no such problems with Linux. Again using xmodmap, I converted the key above the enter into an extra Shift key, much to the relief of my right pinky. The only annoyance with this solution was that I kept trying to use that key as a Shift on the Mac computer too, only to be presented with a disappointing backslash character as a result.

Well, those days are now behind me! Today, I downloaded KeyRemap4MacBook, a very helpful tool which managed to accomplish what so many others had failed. I am now typing this text on my Mac, using the backslash key as a Shift! And my right pinky is very grateful.


Thanks, KeyRemap4MacBook!

Building qtHaskell on OS X

Update

The author of qtHaskell recommends a simpler approach. In addition to the "slotReject" patch below, simply change lines 308-309 of build.pl as follows:

$ghcdv = ((($ghcmv == 6) && ($ghcsv >= 12)) || ($ghcmv > 6)) ? 1 : 0;

$ghcfv = ((($ghcmv == 6) && ($ghcsv < 12)) || ($ghcmv < 6)) ? 1 : 0;

Original post

Finally! I spent the entire weekend on this, but I finally managed to compile qtHaskell on OS X. I spent a lot of that time on the internet googling for the error messages I encountered, to no avail. So, to save some time to the next person who google them, here is what worked for me. The qtHaskell documentation recommends that you use their ./build perl script. That almost works, but not quite. Thankfully, by giving arguments to the scripts, we can run the parts which work, and skip over the parts which don't. Speaking of arguments, one the most annoying errors is the "scattered reloc r_address too large" error message which only occurs at the very end of a very length compilation. I stumbled upon a page recommending to use "--extra-ld-opt=--architecture=x86_64" to fix a similar error message, and since then I have superstitiously stuck the argument everywhere I could, just to be sure. So, let's build qtHaskell, shall we? First of all, qtHaskell is a set of Haskell bindings for Qt, so you need to install Haskell and Qt. I used Qt 4.7.1, qtHaskell 1.1.4, and GHC 7.0.4 (64 bits). Actually I had the 32 bits version of GHC at first, and that let to the aforementioned r_address problems, followed by linking problems afterwards. Better upgrade to 64 bits!

brew uninstall ghc
brew install ghc --64bits

We should now be ready to compile qtHaskell proper. There are actually two parallel hierarchies to be compiled, the C++ implementation and its Haskell FFI interface. Both are very large, because, well, Qt is large. First, the C++ part: no problems here, the build script works just fine for this part.

./build user cpp qmake cmake \
    --extra-ld-opt=--architecture=x86_64
./build user cpp-install \
    --extra-ld-opt=--architecture=x86_64 --no-sudo

Note that even though the "user" flag was given, the build script will still install the files globally in /usr/local/lib/libqtc_*. Presumably, the flag is only for the Haskell part, for which the steps would ideally be as follows.

./build user haskell configure \
    --extra-ld-opt=--architecture=x86_64
./build user haskell build \
    --extra-ld-opt=--architecture=x86_64
./build user haskell-install \
    --extra-ld-opt=--architecture=x86_64 --no-sudo

Unfortunately, the middle step fails with the following error message.

unrecognised command: makefile (try --help)
runghc Setup.hs makefile: No such file or directory

Fear not! This cryptic-looking error message is actually a clue towards the solution. Apparently, "runghc Setup.hs makefile" is the command used for building the Haskell part. Using the help, as recommended in the error message, leads to the following alternative build commands.

runghc Setup.hs configure --user \
    --extra-ld-opt=--architecture=x86_64
runghc Setup.hs build

Unfortunately, that doesn't work either, at least not with a recent GHC. There is a problem in the qtHaskell code, which leads to the following error message.

Qtc/Core/Attributes.hs:583:13:
    Could not deduce (Qstt a (QDialogSc b))
      arising from a use of `slotReject''
    from the context (Qstt a (QDialogSc b1))
      bound by the instance declaration
      at Qtc/Core/Attributes.hs:581:10-52
    Possible fix:
      add (Qstt a (QDialogSc b)) to the context of
        the instance declaration
      or add an instance declaration for (Qstt a (QDialogSc b))
    In the expression: slotReject'
    In an equation for `reject'': reject' = slotReject'
    In the instance declaration for `QsaSlotReject a'

If you are familiar with Haskell, you should be able to fix the problem on your own. Alternatively, you could grab Uduki's hsQt fork of qtHaskell, which comes pre-patched. Or you might apply the following patch yourself:

diff --git a/Qtc/Core/Attributes.hs b/Qtc/Core/Attributes.hs
index 197c506..217d585 100755
--- a/Qtc/Core/Attributes.hs
+++ b/Qtc/Core/Attributes.hs
@@ -580,7 +580,7 @@ class QsaSlotReject w where
 
 instance (Qstt a (QDialogSc b)) => QsaSlotReject (a) where
   slotReject' = (Qslot "reject()", \_ -> ())
-  reject' = slotReject'
+  reject'     = (Qslot "reject()", \_ -> ())
 
 class QsaSignalRejected_nt_f w x f where
   signalRejected', rejected' :: x -> SltConf w f

Now that the source is patched, the Haskell part should finally compile.

runghc Setup.hs build

If you run out of memory during this phase, just run the command again. And again. As many times as it takes to compile all the targets. When you get the the very last target, qtHaskell is finally linked... and then, like me, you might get one of those nasty "scattered reloc r_address too large" errors. Did you? If so, you might have skipped the very first step and be stuck with a 32 bits version of GHC. Check using "ghc --info". If you insist on using the 32 bits version, you could try the following variant of the build command, which got me through that step. Still got troubles when linking programs using qtHaskell, though.

runghc Setup.hs configure --user --disable-library-for-ghci

Now that the Haskell part is built, we can finally install it! This time, the "user" flag is honored.

./build user haskell-install \
    --extra-ld-opt=--architecture=x86_64 --no-sudo

Let's test this! The qtHaskell helpfully provides the following Hello World code, just put it in a file with the "hs" extension and build it into an executable using "ghc --make".

module Main where

import Qtc.Classes.Qccs
import Qtc.Classes.Gui
import Qtc.Gui.Base
import Qtc.Gui.QApplication
import Qtc.Gui.QPushButton

main :: IO Int
main = do
  qApplication ()
  hello <- 60::int="" ello="" hello="" nt="" pre="" qapplicationexec="" qpushbutton="" qshow="" qthaskell="" resize="" world="">
Tada!
At long last. A button!

As the story dragged on

The main reason I decided to experiment with non user-visible stories was to keep stories short. That part worked great; as long as they were short, I breezed through the stories and was eager to start the next. But then one particular story — I'm looking at you, story 19! — turned out to be a lot more work than expected. As the story dragged on, I lost my motivation... and then it took even longer to get it done. I would like to avoid this situation in the future, if possible.

But how? By adjusting my process, of course! Ideally, I should be able to complete each story in one sitting; and now that stories don't need to be user-centric, there is no reason not to make them as small as needed. If they aren't, then the plan needs to be refined. Thus, this time, I am going to break something even more fundamental than the user-centric nature of stories: the fixed-length iterations! How naughty.

This iteration will end in one week, or when a story drags on for more than a day, whichever happens first. If it is the latter, I will have the opportunity to adjust the offending task, by dividing it into smaller chunks.


Alternatively, I could rethink my design so that the offending task is not needed, or vastly simpler. Or just different. Or more fun to code.

I mention this possibility because I stumbled upon a post from a writer explaining how she quintupled her productivity by optimizing her process, and she found out that she was more productive when writing scenes she was enthusiastic about. Well, I was enthusiastic about Story 20. And hated Story 19. I'm not sure why.

In order to figure it out, from now on I am also going to write down the best and the worst story of each iteration. Hopefully, a pattern will emerge. Although, to be honest, even if I had an easy criterion to distinguish stories which are going to be fun from those which are going to be boring, I don't really know what I could do with that information. Sure, some stories are less pleasant than others, but they still need to get done, right? Unlike passages in a book, if the code is boring to write, it doesn't mean the user won't enjoy the result. Who knows, maybe I'll find a different software organization which minimizes annoying tasks?


Iteration 5

As yet another crazy departure from conventions, I'm dropping story numbers. They haven't been very useful so far.

• Setup continuous compilation (using Substrate to detect when the source files change).
• Use signals to propagate errors.
• Implement an InputNode which can bake itself.
• Implement a Gui which links the input nodes with the input pane widget.

Who cares about the user?

As promised, I am going to play with my process. Unlike what I promised, I am not going to waste an entire iteration on this. I am going to waste several!

I have already tried one change: scheduling optional stories. It didn't work, but I'm glad I tried. I am not on a tight schedule, so I can take the time to experiment and see what works. I even tried the Spiral idea of trashing all my code and starting from scratch using my newly accumulated experience: that's what I did on the very first iteration, when I decided to drop everything and start fresh, Agile-style this time. What should I try for the next iteration?


Well, is there something from the last iteration which could be improved? Let's see, was there anything at all which went wrong? Let me think. Oh, right. Maybe the fact that I totally failed to produce anything at all?

Yeah, during the past iteration, I didn't accomplish much. Or rather, that's what you would think if you only looked at the number of stories I completed: zero. And indeed, when I run Substrate, it looks exactly as it did two weeks ago. But that is only part of the picture! Sure, the look didn't change and there are no new buttons to click on, but I did spend a lot of time working on Substrate, trying out new architectures and comparing design approaches. Even as I worked, I felt like I was wasting precious time, because days went by and my metric wasn't improving.

Well, screw the metric. Screw the users, I mean. Erh... that's not what I meant either.


This iteration, I am testing a variant on story descriptions: instead of focusing on user-visible features, I am going to focus on developer-visible features. If code needs to be written, then I don't care whether that code is producing buttons, menus, bells, or whistles: by golly, this code is going to be divided into small story-like chunks and assigned to an iteration!


Iteration 4

This iteration is still about geting Substrate to interact with external files, but the steps to reach this goal are more detailed. Stories 18 and 20 have no user-visible impact.

Story 10 (1 story point): I can save the current project. Use a version number in case the format changes.
Stories 11 to 16: Postponed.
Story 17 (1 story point): I can reload and run the entire project.
Story 18 (1 story point): Simple filesystem-backed key-value store with one file per key.
Story 19 (2 story points): The project reloads when the filesystem copy of the store changes.

Story 20 (2 story points): Specialized store interface for strings, integers, and string arrays.

Story 21 (1 story point): The project stores the active input tab.

p.s.: Don't take the title of this post seriously. The only user Substrate ever had, so far, is myself.

Unifying the Agile and Spiral models

If, like me, you have never heard of the Spiral model of development, here is a summary: it is a bit like Agile, except that all the code is thrown away at the end of the iteration. Also, the scope increases each time, so the iterations grow longer and longer.

Yes, I know this sounds like a terrible idea, but that's only when compared to Agile as a reference point. From that angle, Waterfall is the model where you only get one, very big, very long iteration, and hope to get it right the first time. When put in the context in which it was invented, doesn't the Spiral idea of starting with a smaller version of the project seem like a great improvement?

Not throwing away the code all the time sounds like an even better improvement, but when I first heard of the Spiral model (just a few hours ago), I thought the opposite. Throwing code away! What a novel and intriguing idea! Should I try? Maybe? Being in an Agile team doesn't guarantee good design, after all. And changing a bad design is sometimes a large endeavor. With Substrate, for example, I haven't been able to implement any story at all this iteration, because I was busy refactoring the entire application to use MVC.


Instead of being forced to either drop all the existing code or keep all of it, I suggest we should be allowed to do either. And those two possibilities are just two ends of a continuum: using version control, we can easily revert to a number of intermediate steps between the two extremes, whichever is most convenient.


An Example

Let's start with an empty project, in the middle of Design Space, the space of all possible designs.

We discuss the product idea, draw a bit of UML, and end up with a particular design (a particular point in Design Space).

A few iteration go by, moving the project toward the goal established in the design.

While implementing new features, suppose we come up with a new design for the project. Could happen! Otherwise, we would still be doing big design up front and it would work great. The goal is moved a bit to the left.

Since we are using iterations, we adapt to this unexpected change without a sweat. We just start writing the new code with the new code in mind!

But what if the new design was very, very different from the first?

In that case, starting from scratch is less work.

But this is not an all or nothing decision. Yes, you can start from scratch, but you can also start from any other past point. Just pick the one which represents the least amount of work!

Future Work

Now, as I said, I only heard about the Spiral model today, so there are still many rough corners in my generalization of it and Agile. For one thing, going back in time means removing some features, and if those features have shipped already, maybe you don't want to remove them. Feature Space is not necessarily aligned with Design Space.

The other big problem is that history is linear, while Design Space is multidimensional. When your goal changes its position, it probably won't move across all the dimensions, only some of them. So while it is true that some of your code can be thrown away, most of it probably shouldn't; in order to reach the commit you want to restart from, you will have to throw away some commits which are still good, just because they were submitted after the restarting commit. I recommend using a history-rewriting tool such as git, but when I tried, solving all the merge conflicts was still a lot of work.


Ideally, each new feature could be implemented on a bare-bones version of the application, containing only the elements on which the new feature builds. In that setting, implementing the new feature is much easier, because there are no unexpected interactions with the other features, the codebase is small enough to be understood all at once, and the debug-edit-compile cycle is fast. Theoretically, using feature branches, maybe this ideal could even be reached!

The problem, of course, is that the unexpected interactions bite back once we try to merge all of the feature branches into a complete product.

I think I am going to try anyway. If I can afford wasting an entire iteration redesigning my code, maybe I can also waste one redesigning my process?

Still not ready for private consumption

Good news! This time, I did manage to find a use for Substrate. It turns out that if you want to suck the motivation out of your programmers and make them want to quit, forcing them to use Substrate is a really good way to achieve that!

So, yeah. Eating your own dog food is a great way to find flaws in your own software, but not necessarily a good way to get them fixed. I mean, I want to fix them, I do, but the idea of eating my own dog food is that I should be using Substrate to manipulate its own source code. So I'm supposed to fix those debilitating flaws using a debilitatingly flawed software. No fun there! It would also take a while, because the sub-par tool would drag me down. Better do it all in vim and forget about the dog thing.


Iteration 3

This iteration is about using Substrate and vim, together. My goal is to make Substrate a good complement for vim.

My goal was never to replace vim. That would be silly! I was not thinking of Substrate as a text editor at all, but as a text manipulator. Sometimes I would need an editor, and sometimes I would need a manipulator. But I have now realized that I won't be developing Substrate using Substrate, I will be developing Substrate using vim! Mostly vim. So if I want to keep Substrate in the loop, to eat it as dog food, then I need Substrate and vim to play along.

And to do that, I need Substrate to interact with the external files I'll be editing in vim.


Story 10 (1 story point): I can save the current project. Use a version number in case the format changes.
Story 11 (1 story point): I can run the script with a particular file as input.
Story 12 (1 story point): I can run the script with each file from a directory (recursively) as input.

Story 13 (1 story point): I can run the script with only the changed files as input.

Story 14 (2 story point): I can perform story 13 from the command line, or when a file change is detected.
Story 15 (1 story point): I can write multiple parallel scripts (all with the same input).
Story 16 (1 story point): I can write multiple sequential scripts (feeding one into the other).

No "bonus" stories this iteration, not anymore. Listing the stories in advance gives me a short term goal to accomplish, and that's motivating. There no motivation in completing optional stories: who am I trying to impress? I just want to be done with the mandatory stories as early as possible, and play video games for the rest of the week.

What one hat said to the other

Developers know which features would be easy to add, but it is users who know which features would be useful to add. This is why, after completing what I thought was the Minimal Viable Product version of Substrate, I decided to trade my Developer hat for a User hat. This turned out to be a very, er, humbling experience.

I wanted to know which lack-of-a-feature was causing the most pain to my non-existent users. To find out, I tried to use Substrate for all the tasks it could handle. And the main conclusion of my experiment was that there were, in fact, very few tasks it could handle. Or, as User hat would put it: What a crappy, buggy, useless software.

Well, thanks for your feedback, User hat. For what it's worth, Developer hat here is more worried about "useless" than about "buggy". That's because I, too, aspire to be a User some day! I want to eat my own dogfood. Substrate is a text manipulation tool, code is made of text... surely one day a version of Substrate will be useful enough for me to use it while developing later versions?

One day, perhaps, but not yet. The current Substrate is great for twea— ok, the current Substrate is a crappy, buggy program aiming to make it easy to tweak long chains of bash commands. But even if it wasn't buggy, the bigger problem is that very few tasks can actually be accomplished through a single long chain of bash commands.


I'm really glad I tried this MVP approach, because this is a major flaw which I could only have discovered by actually using the software, as opposed to merely developing it. I have long paid lip service to The Unix Way, and I am not alone in thinking that a few carefully chained Unix commands can go a long way. But not, it would seem, all the way.

Since my long term vision for Substrate was a GUI for creating complex chains of text-manipulating nodes, I need to revise my approach. And that's precisely what short iterations are for, aren't they?

Iteration 2

In this iteration, I will focus on adding new features, not on fixing the many bugs User hat found. Hopefully, by the end of the iteration, Substrate will evolve into a crappy, buggy, marginally useful product. Useful enough, I hope, for Developer hat to use.

Story 5 (1 story point): I can enter different inputs for the script.
Story 6 (1 story point): I can run the script on a series of different inputs.
Story 7 (2 story point): I can save the script and its inputs as a single project.
---
Story 8 (2 story point): I can write multiple scripts which call each other.
Story 9 (2 story point): I can write a script which applies multiple inputs to another script.


Adjusting from last iteration, this time I intend to finish the first three stories in two weeks, not one month. I have also added two optional stories, in case I finish early again. This is my personal extension to the Agile process: even if your velocity indicates that you can only deliver 4 story points per iteration, you should still schedule a bit more than that. I'll let you know if it's a great way to empirically measure by how much to increase your velocity, or just another idea which sounds better to one hat than to the other.

Success!

Has it only been a week? Wow. This challenge was both easier and harder than expected.

Easier, because I expected to finish in a month, rather than a week. I'm really bad at making time estimates.

Harder, because I ended up writing much more code than I expected. I initially thought that I would implement the four stories back-to-back, with a few lines of code each. That was the plan. But I ended up organizing my code a lot, and implementing a few visual niceties which were not in the original stories. I'm really bad at sticking to a plan.


In any case, tada! Here is Substrate, iteration 1.

The greyed-out text is one of the extra visual niceties
I couldn't help but add. This shows you which lines
are going to run when you press F5.

That's right! I escaped from my curse, and completed a project: an MVP version of Substrate. I think laying down a plan at the beginning really helped, even if I didn't follow it too closely. I'll definitely need to write down new stories for iteration 2. But not yet.

For now, I need to play a bit with the product. Only then, as a user, will I have the authority to decide which parts of the product could be improved.

May the day I actually complete something finally come

Completing projects is hard

I have a curse.

My curse is that I have too many great ideas. Now, I know that ideas are cheap, so I don't just publicly announce my ideas and then complain that nobody is acting on them. I know that if I want my ideas to have an impact, I must act on them myself. I do act! But I'm still cursed.

Often, my idea is something that I can code on a computer. That's great! I have a computer, and I know how to code. I can act. So I put everything aside, fire up my favorite text editor, and enthusiastically put in the hours necessary to create something out of nothing. But it doesn't work! I'm still cursed.

The main problem with this "put everything aside" approach is that the main thing I put aside is usually the code I was writing in order to implement my previous idea. And by the time the code for my new idea becomes something usable, it will in turn be shoved aside by an even newer, better idea! That's why great ideas are my curse. If only I was coming up with worse and worse ideas instead of better and better ones!


Well, this curse ends now.

I was discussing Agile with Nadya yesterday, and I asked her how small she thought a team could be while still being Agile and benefiting from it. Since she and I love to work on common projects, I expected her to answer "two", suggesting that our common projects should use Agile. But to my surprise, she answered "one".

I guess it makes sense. If following the Agile methodology can help a team to ship, why couldn't it help me complete my personal projects? Let's give it a shot.


The project I will actually complete

My current project is Substrate, a text processing tool making it easy to build text processing pipelines.

I got that idea while using Houdini, a 3D software package making it easy to build 3D-model and 2D-image processing pipelines. It's very different from other 3D packages. Using Houdini was much more fun than using 3DS Max. Houdini felt more like programming than modelling, but at the same time, it also felt more fun than programming. Very few things do! I suddenly wanted all of my tools to be like Houdini.

I started immediately, and that was my first mistake. I should have taken the time to figure out what it was that made Houdini so different. Instead, I immediately started working on a Houdini clone, with the major exception that my clone would manipulate text, not 3D models.

So far, I have a pane system where I can open and close tabs, split panes horizontally and vertically, and... well, that's it, really. No text manipulation yet. I just copied Houdini's pane system because Houdini had it. From an Agile perspective, the state of the project is highly embarrassing: totally unshippable, with the "team" focussing on miscellaneous GUI features over the core functionality.


Well, this ends now.

I still don't know what it is that makes Houdini feel different, but I will find it. Iteration after iteration. And each iteration will be shippable. Amen.


Iteration 1

Since I am working on this during my free time, I will need relatively long iterations in order to have enough coding hours to produce anything. Let's say one month: I shall have a shippable MVP at the end of February. And here is what this MVP will contain.

Story 1 (1 story point): I can load a Bash script and view its contents.
Story 2 (1 story point): I can run the currently-loaded script and run it using a command from the menu. I will see the script's output on stdout.
Story 3 (1 story point): I can run a prefix of the script using a command from the menu. I will see the output of the truncated script on stdout, or an error if the truncated code doesn't represent a valid Bash script.
Story 4 (1 story point): I can successfully run a script prefix even if the last line ends with a pipe character.

The hypothesis tested by this first iteration is that while working on a pipeline, the ability to see intermediate results from the middle of the pipeline will make it much easier to debug the pipeline. The Houdini feature which corresponds to this is the ability to click on any node to see what the 3D model looks like at this point.

I'm not convinced that this is really what makes Houdini fun to use, but it's certainly a core feature, and it's one which I am confident I can implement quickly.


Well, let's get to work!

An unusual "terms and conditions" page

You might have noticed my new "Terms and conditions" page, above. But then again, if you're like most users, you probably haven't.

Which is the whole point.


Nobody reads those pointless agreement pages, yet they are legally binding. This is crazy! I don't want to be legally bound by contracts I haven't even read.

For this reason, today, I counter-attack using my own "Terms and conditions" page. I am not a lawyer, but if I ever get into trouble with one of the many agreement pages I have not read, my opponents should not be able to claim that I am bound by their terms and conditions without accepting that they are also bound by mine.


It is not legal to encourage someone to sign a contract without reading it. Similarly, it should not be legal to encourage a user to click "I agree" without reading and understanding the conditions he or she is agreeing to. Yet, that is precisely what service providers are doing! Successful service providers streamline their website's user experience to encourage a particular behaviour, for example, signing up for the service. And my experience with online agreements is that the encouraged behaviour is always to click "I agree" without reading anything.

For example, service providers often go to great lengths making sure the product offering is crystal clear, by using short punchy sentences or video presentations. By contrast, those same service providers use many pages of long-winded, cryptic legalese to describe their legal agreement. And even when I do take the time to read and consider such terms, I am often rejected with a "this page has expired" message.

The only way to reach the service, in this case, is to click "I agree" without reading the contract. Since that is the expected behaviour, that's what I am going to do from now on. But since you don't expect me to read your contracts, don't expect me to be bound by them.

Our Life in 3D

By the way, my girlfriend and I will be updating the blog Our Life in 3D, a blog about our common project: modelling her entire condo using Houdini and 3DS Max.