Ryan's Tech Blog

I stole some time from my increasing non-technical workload to play with generating heat-maps of residential energy consumption in my http://gainesville-green.com project.  The initial results are promising:

There are a few neat things going on here.  I’ve got a url handler in my lisp that looks to the query string for lat-lng bounds, image size, and some other variables to generate a PNG file.   I pass that URL to a Google Maps API GGroundOverlay to put the image onto the map.  Add some javascript event glue and I can do cool things like automatically regenerate the heat map overlay when you zoom/pan the map around, and display an animated heat map showing consumption over the course of the year.  There’s still a lot of UI interaction to sort out, but I think it’s a nice approach.

The heat map itself is generated using Vecto, and I think I’m doing it wrong.  I jump through some hoops to map lat-lng to image pixel coordinates, pull from the database, and end up with a list of (x y weight) tuples, with the weight being a number between 0.0 and 1.0 representing the relative consumption of the home that should be at pixel x,y in the result image.  Then I start painting, which is where I think I should be doing more math.  For each point, I pick a color between green and red based on the weight, using the handy cl-colors library to interpolate:

(defun find-color (percent)
  (if (> .5 percent)
      (cl-colors:rgb-combination cl-colors:+green+ cl-colors:+yellow+ (* 2 percent))
      (cl-colors:rgb-combination cl-colors:+yellow+ cl-colors:+red+ (* (- percent .5) 2))))

I actually have to go from green->yellow, then yellow->red, with some goofy adjustments to the percent to make the interpolation work out.  Once I have that, then I have my color, and my pixel, so I can start drawing.  To get a smoother look, for each point I draw concentric circles with different radius and opacity, so each individual data point is rendered like this:

This is enlarged to show some of the blockiness, it ends up looking pretty nice when they are small.  Here’s the actual function:

(defun draw-point (x y color max-radius)
  (iterate (for r from max-radius downto 1 by (max 2 (round (/ max-radius 6))))
	   (for alpha = (/ 1 r))
	   (vecto:set-rgba-fill (cl-colors:red color)
				(cl-colors:green color)
				(cl-colors:blue color)
				alpha)
	   (vecto:centered-circle-path x y r)
	   (vecto:fill-path)))

Max-radius determines how large the largest circle is, and is calculated based on how many points I’m drawing.

There are a few drawbacks to this approach.  First, it’s slow.  Drawing operations aren’t exactly cheap, especially when messing with alpha channels.  It takes me around 5s for 578 data points, which is fine for offline tasks, but on a web-app it needs to be super zippy or you fickle internet folk will close the tab. I also want it to be easy to show animations, so generating a bunch of them quickly would be nice.  The time spent increases fairly linearly with data points, and I’d like to be able to render heat maps for large areas with tens of thousands of data points.  Profiling shows practically all of my time and bytes consed are spent in the draw-point function. UPDATE: after more profiling, vecto:fill-path is most of my time, which makes sense.

Second, I have to be really careful to draw these points from lowest weight to highest weight, because I want red dots to be painted on top of green dots.  It seems like I should decide what color each pixel should be, then draw it once, rather then accumulating the right color in the image canvas.  Right now there’s also some bug with drawing lots of data points, I just get a big green image, when I would expect some reds or yellows.

Another issue is for apartments I have coordinates for the apartment complex, but not each individual unit.  This makes some funny results, like the big orange blob on the right side of the screenshot above where I’ve painted a few dozen points on top of each other.

I did some googling on heat-map algorithms, and found some actionscript and java code, but the actionscript was using a similar approach and the java was incomprehensible.  I think I’ll try making a big array for the result image, and calculating an average weight for each pixel, then loop through that and draw once.  I’m also going to try calculating the weights using magnetic field strength or gravity math.  I think that approach will end up faster, look nicer, and should be a fun problem.