Anyways, here is a short video about what we’ve been doing. Enjoy.

Following some interesting comments on our previous blog post we thought it would be appropriate to emphasize the added value of our logging approach (which we now realize we should call “complete tracing”) compared to the others mentioned. To demonstrate, let’s observe some simple code for manipulating sets. As background, a partition is defined to be a set of mutually disjoint sets. Each of the sets in a partition is usually called a class. For example: If this is the partition #{#{1 2} #{3 4 5}} then #{1 2} is a class. What makes this a partition is that no two classes share a member.

Now, consider the following functions that carry out the task of “linking” two simple elements (the ones that belong to the classes which are inside the partition) to a given partition:

(defn add-to-class [partition class el]
  (conj (disj partition class) (conj class el)))

(defn containing-class [partition el]
  (first (for [class partition :when (some #{el} class)] class)))

(defn link-elements-in-partition [partition el1 el2]
  (let [c1 (containing-class partition el1)
        c2 (containing-class partition el2)]
    (cond

      (and (not c1) (not c2)) ; neither are in any class (we add it to c1)
      (conj partition #{el1 el2})

      (and c1 (not c2)) ; el1 was found in a class (c1) while el2 was not (we add it to c2)
      (add-to-class partition c1 el2)

      (and c2 (not c1)) ; el2 was found in a class (c2) while el1 was not
      (add-to-class partition c2 el1)

      (and (and c1 c2) (not= c2 c2)) ; both were found in different classes (we join them)
      (conj (disj partition c1 c2) (union c1 c2))

      :else
      partition)))

For example, (link-elements-in-partition #{#{1 2} #{3} #{4 5}} 2 3) should return #{#{1 2 3} #{4 5}}) and (link-elements-in-partition #{#{1 2} #{3 4 5}} 3 2) should return #{1 2 3 4 5}. Yet if you try to run this code it returns #{#{1 2} #{3 4 5}}. There is a bug.

Let us examine the output of the clojure-contrib trace library (after re-defining the functions with “deftrace”) and see if it can help us find the source of the problem.

TRACE t32800: (link-elements-in-partition #{#{1 2} #{3 4 5}} 3 2)
TRACE t32801: | (containing-class #{#{1 2} #{3 4 5}} 3)
TRACE t32801: | => #{3 4 5}
TRACE t32802: | (containing-class #{#{1 2} #{3 4 5}} 2)
TRACE t32802: | => #{1 2}
TRACE t32800: => #{#{1 2} #{3 4 5}}

(start-complete-tracing)
(link-elements-in-partition #{#{1 2} #{3 4 5}} 3 2)
(end-complete-tracing-and-print-org-mode)

**** (and (and c1 c2) (not= c2 c2))
**** `=> Result:
false
***** (and c1 c2)
***** `=> Result:
#{1 2}
****** c1
****** `=> Result:
#{3 4 5}
****** c2
****** `=> Result:
#{1 2}
***** (not= c2 c2)
***** `=> Result:
false
****** c2
****** `=> Result:
#{1 2}
****** c2
****** `=> Result:
#{1 2}

We are all familiar with the following scenario – You write some code. Everything is working. You change your code just a little bit…

Incidental logging and step-by-step debugging are basically the same thing. You make your way through your code in a divide-and-conquer search for the cause of an unexpected result. You add a log here, you debug a function there, you step-into-outta-over-and-out. And then you finally find it – the function that returns the wrong value. And then, hopefully, the line that messes everything up. It took you almost forever and you know that with the next bug everything will happen all over again. But you always have a feeling that there is some reason guiding you through this weird process. And it’s true – Logging is clearly a job for a computer. So we asked ourselves – why not automatically log all arguments and return-values for any function call?

If you were to try implementing this in other languages you would find out you need to pursue one of two approaches, both fundamentally flawed. One would be to preprocess your code and inject logs. Another approach would be to implement some hideous reflection mechanism to inject the logs in the right places. The first approach would require heavy coding and either text-parsing or some use of a code analysis library, the second approach would require compiling code at runtime. Luckily for us, a preprocessed code-walking in Lisp is as simple as writing a function that iterates a list. Well, It is iterating a list!. Moreover, what would be the right places to inject logs at? Well, if you write in functional style, you really know for certain that you only need to log the arguments and return value of every function (or a sub-special-form of a function). In a non-functional programming language (any language that allows modifying variables) this certainty doesn’t exist. Therefore, any attempt to automatically generate all needed logs would require the logging of all variables after each and every single line of code.

In Lisp, a code-walker macro can be written that will iterate a function’s s-expression and “inject” log calls before every sub-expression call. We had some issues with handling Clojure’s special forms like “let” and “for”, but victory was eventually ours. It ended up looking like this.

The question still arises: How will we view such a monstrous amount of logs? We would really want to be able to view it as if its a simulation of the actual run of the program. So we wrote a small Javascript application to display the results of the logging in such a way. And that looks something like this:

With that said, we are aware that as our project will grow in complexity it may not be feasible to continue with this paradigm. However, thanks to Clojure, we believe that we will be able to find ways to maintain some parts of it by improving the logging mechanism to be less verbose and more precise.

Some people have already experimented with writing robots in Clojure (for example here and here). However, these approaches still assume that development is done on Google App Engine (GAE). This means that you have to redeploy each time there is a new version of your robot (contrary to the common Lisp way which is just evaluating code into a remote process, known as a REPL). We also wanted to use an in-memory database instead of dealing with database schemas and other irrelevant issues for prototyping. This can’t be done with GAE since each robot event gets sent to a different machine running the robot code and the memory is not shared.

In order to write in Clojure and use an in-memory database, we had to find a way to write our code outside of GAE. Google currently allows robots to only be run on GAE so the solution was writing a relay robot to manage the communication between the Wave server to our Clojure server. More technically, our relay robot is written in Java and all it does is intercept the JSON objects coming in from the Wave server, post them to a web server running Compojure (a Clojure web server), wait for a response (the JSON object describing the operations to be done on the wave) and pass it back to be processed.

If you still want to encode and decode JSON, you can use Wave’s HTML 5-compliant JSON functions (JSON.stringify and JSON.parse instead of MooTools’ JSON.encode and JSON.decode).

Update: Rasmus Fløe brought up a much simpler solution which allows you to still use MooTools’ JSON API. Instead of ommitting MooTools JSON from Core, just extend the JSON object: $extend(JSON, {stringify: JSON.encode, parse: JSON.decode});

Here’s the source code for task list gadget and the original blog post about it.

As I started the implementation, I realized that there are certain difficulties with gadget state management with regard to Wave’s real-time concurrency. Here is an attempt to recreate the learning process I went through, with the difficulties I encountered and solutions I put in place to make this gadget work.

• This is just a preliminary version of the gadget – I am still tweaking the user interaction so that concurrency is completely flawless.
• I do not have access to the real Wave sandbox. All my development is done using the gadget emulator. As far as I know, this does not work on the real Wave server (but it’s probably a small fix if I could debug it)
• This is long and technical. Think of it as a guided discussion.

Background

Gadget states consist of a set of key-value pairs (like a hash table, associative array or Javascript object). It is important to note that as opposed to Javascript objects, gadget state values are always strings (and not arrays or other objects). If you want to see what some gadget states look like, try the gadget emulator – it shows the gadget state besides the gadget itself. If you take a look around, you will see a whole assortment of hacks to manage gadget state even though it only allows strings as values. When I asked in the google group about this, I was suggested to JSON encode any objects or arrays I want to store and use that string as the value in the state.

JSON encoding objects and arrays into strings is a hack, but that’s not the real problem. The issue is how that works with the way gadgets modify their state. In the gadget API reference you will find that gadgets modify their state by submitting a delta. A delta is just another set of key-value pairs that state which keys to modify, and which new value to give them. If a null value is submitted then that key is removed from the gadget state.

Approach #1

Let’s start by trying to implement a simple task list without reorderability. We have to think of our gadget state scheme. How about using the idea to store the array of tasks by JSON encoding it? Then the state will look something like this:

tasks: “['Go to store', 'Clean home', 'Finish blog post']“

Seems simple enough. Let’s imagine two people working on the same gadget instance and trying to add a new task at the same time. Participant #1 submits the following delta:

tasks: “['Go to store', 'Clean home', 'Finish blog post', 'Go to sleep']“

while participant #2 submits:

tasks: “['Go to store', 'Clean home', 'Finish blog post', 'Check xkcd']“

In effect, the Wave server will receive one of the deltas followed by the other, which means the end result will be whichever delta was applied by the server last. One of the new tasks will necessarily not appear (although it will be visible in the gadget history right before it got erased). And actually, no concurrent modifications will ever work with this scheme.  Can we find a different approach that will not have this problem?

Approach #2

We can try to put the different tasks in separate keys of the state, for example:

task_1: ‘Go to store’
task_2: ‘Clean home’
task_3: ‘Finish blog post’

This approach will allow modification of specific tasks to work in a concurrent fashion, but adding a new task will have the same problem as approach #1.

Approach #3

The problem with approach #2 is that two participants will try to modify the same key when they each add a new task. If we want to solve this, we need to make sure that the key sent by both participants is different. Each participant has a unique participant id which is accessible from the gadget API. Let’s augment the keys with the participant id who added them. So, the initial state will look something like this (assuming the two participant id’s are 722931 and 199882):

task_1_722931: ‘Go to store’
task_2_199882: ‘Clean home’
task_3_722931: ‘Finish blog post’

Now, the two deltas being submitted for the addition of the fourth task would be:

task_4_722931: ‘Go to sleep’

and

task_4_199882: ‘Check xkcd’

Which means the gadget state after both deltas were submitted would be:

task_1_722931: ‘Go to store’
task_2_199882: ‘Clean home’
task_3_722931: ‘Finish blog post’
task_4_722931: ‘Go to sleep’
task_4_199882: ‘Check xkcd’

That has the slight weirdness of having two task_4_* keys but there’s no need for the UI to expose that in any way, so concurrent modifications worked as far as the user is concerned (which is the only thing that really matters).

But, actually, there is a problem with this approach – it would fail if the same user is trying to modify the gadget from two different computers. But that can be easily amended, by using a randomly generated code instead of the participant id. If we use, let’s say, 96 possible characters with a code length of 6 characters, we can reach over 700 million options for each code. That basically means you’ll never have any concurrency problems. And if you did, it will probably never happen again to you (remember, this could only even perhaps cause an issue if two people are adding a new task at the same time anyways.)

Approach #4

Let’s take on reorderability now. If we take the gadget state after the two new tasks were added, now we want to move ‘Finish blog post’ to come right before ‘Clean home’. What delta would have to be submitted? The following delta seems like it may work:

task_2_199882: ‘Finish blog post’
task_3_722931: ‘Clean home’

Let’s see what happens if we would use this scheme and another user tries to move ‘Clean home’ to come before ‘Go to store’ with the following delta:

task_1_722931: ‘Clean home’
task_2_199882: ‘Go to store’

If the two deltas are submitted at once and the first gets applied before the second, we get the following gadget state:

task_1_722931: ‘Clean home’
task_2_199882: ‘Finish blog post’
task_3_722931: ‘Clean home’
task_4_722931: ‘Go to sleep’
task_4_199882: ‘Check xkcd’

Oh no! Clean home appears twice and Go to store disappeared!

If you are following this discussion and you find it interesting, now is probably a good time to stop reading, grab a piece of paper and a pen (or a chalkboard if you have one around) and  try to find a solution for these concurrent reordering issues. Take into account that the scheme should also allow for modification or removal of a task at the same time as it is being moved around. Also, how would we be able to move ‘Finish blog post’ to go between ‘Go to sleep’ and ‘Check xkcd’? They are both “task_4″!

Solution

Let me start by saying that this is the solution I came up with and the paradigm used in my implementation. If you think of other possible solutions, I’d love to hear about them. They may be better than mine.

The main observation is that since any two deltas may interfere with each other if they touch the same attributes, we must ensure that the attributes modified properly represent the action taken by the user. For example, the last example of a possible delta to represent moving ‘Clean home’ to come before ‘Go to store’ contains two attributes, even though the user only acted on ‘Clean home’. It’s good to try to put it in words. The delta submitted means “Put ‘Clean home’ at position 1 and ‘Go to store’ at position 2″ when our scheme should allow for a delta that means “Put ‘Clean home’ before ‘Go to store’”. Also, if possible we should be able to allow for one user to modify the text on a task at the same time as it is being moved. That means that we must ensure that attributes in a delta modifying the position of a task do not conflict with the attributes in a delta modifying the text of a task. And in general, it is a good idea to be able to represent user actions as modifying one attribute only. This gives the best chance that concurrent modifications won’t conflict.

The solution has two attributes for each task – one for the text and one for the position. Each task would have to get a random id generated. Here is a sample gadget state in our scheme (remember that only strings are allowed as values):

task_826721_text: ‘Go to store’
task_826721_position: ‘1′
task_612229_text: ‘Clean home’
task_612229_position: ‘2′
task_172034_text: ‘Finish blog post’
task_172034_position: ‘3′

Modifying the text of a task would submit a delta only modifying the _text attribute. In order to move ‘Clean home’ to come before ‘Go to store’ we submit the following delta:

task_612229_position: ‘0′

In order to move ‘Go to store’ to come after ‘Clean home’ (which may seem like the same operation but is not – think what would happen if someone else moves ‘Finish blog post’ at the same time) we submit the following delta:

task_826721_position: ‘2.5′

This means that positions must now be floating point numbers. Try to apply those two deltas and see what gadget state we reach. Think what would happen if someone else moves ‘Finish blog post’ at the same time.

There are just two minor issues. First of all, two people might move two different items to position 2.5. This on its own is not necessarily an issue but then one wouldn’t be able to put something between those two positions (it would also need to be position 2.5 but then there is no way to ensure that it is between the two). In order to solve this, instead of using the average of the two positions you are placing between we use a random number in that range (with a certain predefined precision.) Secondly, Floating point numbers have bounded precision, which means that at some point if a list is reordered a lot, the scheme will stop working. In order to solve this, we used a library for arbitrary precision floating point numbers, which are encoded in base-64.

Epilogue

Thank you for joining us for this lengthy exploration. Please let me know what you think of this form of presentation, as it is my first attempt of this kind. As for the gadget state paradigm, I realize that this is way too difficult to be implemented for each gadget separately, and I would like for gadgets to work with perfect concurrency. Therefore, once I have some more free time, I’ll start working on a framework that hides all these technicalities from the gadget developer.

1. This is the first time I’ve seen a robot/gadget combo. The robot, when invited to a wave, automatically embeds a specialized gadget. This seems to be a simpler way to embed specialized content for a typical user than actually installing a gadget, and it allows the robot to place additional structure on the collaboration (for example, permissions). Robots can also access and modify the enclosing wave, whereas gadgets can only access the gadget state. I believe we will see many significant add-ons to Wave being bundled in this way.
2. We are starting to see how the underlying philosophy of Wave is affecting developers. In Wave, people can only access waves if they were invited to join them. This is in stark contrast to the current state of affairs in other social platforms on the Web (Facebook, Twitter). In Brian’s participant map, any participant must opt-in to share their location. We are gracefully entering the next phase of the Web – the phase of true participatory social networks, built on top of Wave.

On another note, Vidar’s attempt at an SVG editor has shown how hard it is to manage gadget state information when all you can use for values is strings (no arrays or objects). Vidar made an attempt to solve it in his gadget by encoding an array-like structure as keys of the form “svg_<participant-id>_<index>”. This is a good idea but it is still broken because:

1. You can be logged in as the same user on multiple machines
2. This paradigm would not allow to add elements in the beginning or middle of an array

I am starting to work on a gadget state management framework that should give you the feeling like you’re almost just working with plain Javascript objects and arrays when in fact is uses a crazy encoding scheme to make it work with Wave’s basic gadget state structure without breaking concurrency.

Each generated URL points to a specific gadget with a specific instance. You can share these URL’s on multiple machines (or windows if you are testing alone) and they will synchronize data.

http://wave.thewe.net/emulator

Code on GitHub: http://github.com/avital/google-wave-gadget-emulator/tree/master