Collaboration: Theory and Practice

This post has a follow-up.

We are all familiar with the following scenario – You write some code. Everything is working. You change your code just a little bit…

Incidental logging and step-by-step debugging are basically the same thing. You make your way through your code in a divide-and-conquer search for the cause of an unexpected result. You add a log here, you debug a function there, you step-into-outta-over-and-out. And then you finally find it – the function that returns the wrong value. And then, hopefully, the line that messes everything up. It took you almost forever and you know that with the next bug everything will happen all over again. But you always have a feeling that there is some reason guiding you through this weird process. And it’s true – Logging is clearly a job for a computer. So we asked ourselves – why not automatically log all arguments and return-values for any function call?

If you were to try implementing this in other languages you would find out you need to pursue one of two approaches, both fundamentally flawed. One would be to preprocess your code and inject logs. Another approach would be to implement some hideous reflection mechanism to inject the logs in the right places. The first approach would require heavy coding and either text-parsing or some use of a code analysis library, the second approach would require compiling code at runtime. Luckily for us, a preprocessed code-walking in Lisp is as simple as writing a function that iterates a list. Well, It is iterating a list!. Moreover, what would be the right places to inject logs at? Well, if you write in functional style, you really know for certain that you only need to log the arguments and return value of every function (or a sub-special-form of a function). In a non-functional programming language (any language that allows modifying variables) this certainty doesn’t exist. Therefore, any attempt to automatically generate all needed logs would require the logging of all variables after each and every single line of code.

In Lisp, a code-walker macro can be written that will iterate a function’s s-expression and “inject” log calls before every sub-expression call. We had some issues with handling Clojure’s special forms like “let” and “for”, but victory was eventually ours. It ended up looking like this.

The question still arises: How will we view such a monstrous amount of logs? We would really want to be able to view it as if its a simulation of the actual run of the program. So we wrote a small Javascript application to display the results of the logging in such a way. And that looks something like this:

With that said, we are aware that as our project will grow in complexity it may not be feasible to continue with this paradigm. However, thanks to Clojure, we believe that we will be able to find ways to maintain some parts of it by improving the logging mechanism to be less verbose and more precise.