Ryan’s Tech Blog

Someone on #lisp started talking about making line charts using vecto, so I went ahead and got setup a common-lisp.net project: http://common-lisp.net/project/adw-charting/

Now I’m somewhat flooded with digital paperwork:

• Get my darcs repository onto common-lisp.net
• Update the documentation to point to common-lisp.net (removed all the damn lies, so only vague untruths remain)
• Get access to my trac sorted out (I sent an email to the cl.net admin)
• Get my ssh keys sorted out
• Get my setup at work pulling from the common-lisp.net darcs repo
• Get my setup at home pulling from the common-lisp.net darcs repo
• Make a script to generate docs/examples and transfer to common-lisp.net
• Make a script to generate / sign a new release
• figure out what I should be doing with mailing lists

I’ve never run an open source project before, I foresee a lot of learning in the coming weeks.

Paperwork aside, I did spend a lot of time last night on actual functionality. I spent a good bit of time reading through The Visual Display of Quantitative Information, to get a better sense of what goes into a good chart, and try to learn the right words for some of these things, and then worked on the code a bit:

• converted macros to use &body instead of &rest, which gives Emacs some better ideas on how to indent code
• REVERSEd the list of data elements before display, so the legends on pie/line charts are ordered sensibly
• Improved the example to not create duplicate X labels
• Pulled apart a monolithic function into several smaller ones, introduced a class to help pass data between them
• Tried to generate more human-friendly axis labels. Instead of splitting the y-axis into N even parts and drawing a label for whatever value happens to be there (which generates odd, unfriendly labels in the examples: 1.52, 3.05, 4.57), I look at the spread data, and pick a suitable interval. So, if the lowest y is -10 and the biggest is 2, then we have a total spread of 12, which is just over 10, so we should try to draw a label every 1.0 units. If the spread was 120, I’d try to draw a label every 10.0 units. I added some math to prevent overlapping labels, and then I have decent labels at human readable increments. There are still lots of problems with this scheme, so I haven’t pushed it yet. Hopefully I’ll have time tonight to work out some bugs, and add some more examples. This style of doing it would be a baby step towards another #lisp request, logarithmic axes.

I wasn’t very happy with using a helper class to pass state around, so I’ll probably try out a few different methods and see what works best.

My new laptop came in on Thursday, a tiny asus eeepc.  The default OS is a customized Xandros, a debian spin-off, so it was pretty easy to add some apt sources and get all the tools needed to try out clbuild.  So far I’m very impressed, almost everything has just worked.

I installed darcs to download clbuild, and then it told me to install a few other tools: cvs, svn, and git.  I had one error later in the process, when git tried to use curl, but after installing curl things went pretty cleanly.

I was unable to build SBCL from source using clbuild, but I think that was a sourceforge problem, not a clbuild problem.  Every other installation went very smoothly.

I was particularly impressed by the ability to load emacs and slime.  I find setting up emacs to be the most laborious part of getting a lisp development environment going, and the “clbuild slime” command gets me halfway there.  Looking at the clbuild script, the way they do it is pretty simple, but its a nice time savings to not need to look that stuff up.

By automatically downloading a ton of code, clbuild also provided me with some nice reading, and I spent the evening hanging out with some neighbors and browsing source code, and saw some neat patterns.

When I got back home, I hacked a little on adw-charting,  changing awhen and aif usages to when-let and if-let calls, which is a macro I saw in several Edi Weitz packages.  when-let/if-let are a little longer than Marco Baringer’s awhen/aif, but I like the explicit names.  awhen/aif introduce meaningful names into the lexical environment, and that has always made me a little nervous.  Whenever names are introduced as side-effects I’m reminded of the mess that is ruby’s ActiveRecord, which has so much magical run-time craziness that your .rb file can seem totally unrelated to what actually gets executed.

I also got just about done on a nice documentation page for adw-charting, similar to Vecto’s.  So similar, one might think I copy/pasted the Vecto content and changed the relevant bits.  I ended up using this as an excuse to try out cl-who, which is a pretty nice library.  I found a bunch of adw-charting bugs in the process.  A few people have asked for source in blog comments, so my goal was to put out a 0.5 release today, but there are still some details to clear up:

1. need to add some license crap to each source file (which might also be strangely similar to Vecto’s license crap)
2. need to add readme / license files
3. need to add some documentation about how adw-charting uses fonts
4. need to sort out gpg signing / asdf-install chores
5. possibly sort out apache incantations to allow read-only anonymous darcs access and submit a clbuild patch

I’ll try for Sunday.

Some more progress on the charting library today:

• cleaned up the API a little bit, opting for some nicer make-foo function instead of requiring calls to make-instance:
  • make-series label data &key color
  • make-axis label &key control-string draw-gridlines-p label-formatter
    • control-string: a format-compatible control string, and supplying it sets the label-formatter for the axis
    • label-formatter: a function of 1 argument, returns a string used as an axis label
  • make-line-chart width height &key series y-axis x-axis background
• The lightened portion of the graph is now the average of the background color and pure white
• The labels on the axes are optional, and the graph adjusts to fill as big an area as possible
• The distance between x-axis labes is calculated based on actual string-widths, so hopefully they will space themselves out reasonably under most conditions

Some examples:

This demonstrates using a control-string and a custom function for axis labels:

  (render-chart
   (make-line-chart 400 300
		    :background '(.7 .7 .7)
		    :series (list (make-series "SeriesA"
					       '((-1 -2) (0 4) (1 5) (4 6) (5 -3)))
				  (make-series "SeriesB"
					       '((-1 4) (0 -2) (1 6) (5 -2) (6 5))
					       :color '(.3 .7 .9))
				  (make-series "SeriesC"
					       '((-1 0) (0 3) (1 1) (2 5) (4 -6))))
		    :y-axis (make-axis "widgets"
				       :control-string "~$")
		    :x-axis (make-axis "time"
				       :label-formatter #'months-from-now->mm/yy))
   "line-chart-with-axis-labels.png")

This one changes the background color, removes the axis labels, and disabled grid lines for the x-axis:

(render-chart
   (make-line-chart 400 300
		    :background '(.7 .5 .7)
		    :series (list (make-series "SeriesA"
					       '((-1 -2) (0 4) (1 5) (4 6) (5 -3)))
				  (make-series "SeriesB"
					       '((-1 4) (0 -2) (1 6) (5 -2) (6 5))
					       :color '(.3 .7 .9))
				  (make-series "SeriesC"
					       '((-1 0) (0 3) (1 1) (2 5) (4 -6))))
		    :y-axis (make-axis nil
				       :control-string "~$")
		    :x-axis (make-axis nil
				       :draw-gridlines-p nil
				       :label-formatter #'months-from-now->mm/yy))
   "line-chart-with-axis-labels.png")

The internals are still messy, but I’ve gone through and was able to simplify some things here and there, here’s a few of the change’s I’ve made to make my code shorter and more readable:

• replacing (destructuring-bind (x y) (foo) (bar x y) with (apply #’bar (foo))
• adding a macrolet to concentrate some very similar behavior
• using loop to count up and down at the same time (for drawing labels up/down the y axis and left/right on the x axis)
• using (dolist (item lst) …) instead of (mapc #’(lambda(item) …) lst)
• using loop‘s maximizing / minimizing features
• using multiple-value-list and nth to get specific values from some function results without compiler warnings

I’ve been trying to put my tongue on why exactly I like lisp so much more than other languages, and working through this made it pretty clear: I have much better tools for reducing accidental complexity than in other languages, and can get the code focusing on what I’m trying to do, not how my compiler needs me to do it. The next time I get a free afternoon, I’m going to make an imperative API, similar to vecto’s:

(with-line-chart (400 300 :background '(.7 .7 .7))
    (add-series "SeriesA"
		'((-1 -2) (0 4) (1 5) (4 6) (5 -3)))
    (add-series "SeriesB"
		'((-1 4) (0 -2) (1 6) (5 -2) (6 5))
		:color '(.3 .7 .9))
    (add-series "SeriesC"
		'((-1 0) (0 3) (1 1) (2 5) (4 -6)))
    (set-axis :y "widgets" :control-string "~$")
    (set-axis :x "time" :label-formatter #'months-from-now->mm/yy)
    (save-file "line-chart-with-axis-labels.png")

That’s pretty damn readable.

updated: fixed syntax error in the imperative sample, and got the imperative bit actually working.

Another productive afternoon:

This is the start of axes. The internals are getting very, very messy, but I really like the result. Here’s the code used to create this:

(defun line-chart-with-axis-labels ()
  "draws a simple line chart"
  (let* ((seriesA (make-instance 'series
				 :label "SeriesA"
					;data expressed as a list (x y) pairs
				 :data '((-1 -2) (0 4) (1 5) (4 6) (5 -3))))
	 (seriesB (make-instance 'series
				 :label "SeriesB"
				 :data '((-1 4) (0 -2) (1 6) (5 -2) (6 5))))
	 (y-axis (make-instance 'axis
				:label "widgets"
				:label-formatter #'(lambda (x-val)
						     (format nil "~$" x-val))))
	 (chart (make-instance 'line-chart
			       :width 400
			       :height 300
			       :background '(.7 .7 .7)
			       :series (list seriesA seriesB)
			       :y-axis y-axis)))
    (render-chart chart "line-chart-with-axis-labels.png")))

I predict adding X axes will be very similar to adding the Y axis, and then I’ll be looking more seriously at spending time to make good docs and releasing this back to the community. The API is getting a little dense, too, but that should be pretty easy to simplify, and I have a few ideas on how to do that.