Archive for code snippet

lisp geocoding, more library possibilities

April 16, 2008 at 7:17 pm · Filed under code snippet, lisp

I’m finally working on a project that has a public component (besides a login screen), and the lisp has been flowing. I’ve been able to put in probably around 10 hours a week on this thing, and I can’t wait until its more presentable to show off. As part of this, we’re building up our work lisp codebase, and I’ve been keeping my eye out for opportunities to open source different library components, and trying to avoid creating a big ball of mud.

The first real candidate so far was adw-charting, but I think I found the next potentials: adw-yahoo and adw-google. They will be libraries for interfacing with various Yahoo! and Google services. So far I’ve only got code for one service apiece, but there’s plenty of growth potential.

Here’s a sample:

(defun geocode-fight ()
  (let ((adw-yahoo:*api-key* "YAHOO APPID")
	(adw-google:*api-key* "GOOGLE API-KEY")
	(address "5308 SW 75th ter, Gainesville FL, 32608"))
    (list
     (adw-yahoo:latlong address :cache-p nil)
     (adw-google:latlong address :cache-p nil))))

(geocode-fight)
=> (("address" 29.604265 -82.42349 "5308 SW 75th Ter")
(8 29.604923 -82.42338 "5308 SW 75th Terrace, Gainesville, FL 32608, USA"))

Yahoo and Google give back different some different data, but so far Yahoo’s coordinates seem more accurate. One thing I noticed was that Google’s geocoding service gives a different lat/long than what maps.google.com displays, which I thought was a little sneaky. I’ve also got some code to help construct google map widgets, using the homegrown html template system and parenscript, but I think much of that will end up getting stripped out, as it depends on the in-house libraries that are likely completely useless to anyone but us.

Still lots to do before either of those libs would be ready for a cl-net request, but it’s on the plate. After building so much on top of so many great free libraries, the need to contribute something back is very strong.

Comments

an x-axis plus API cleanup

January 14, 2008 at 7:22 pm · Filed under adw-charting, code snippet, lisp, vecto

Some more progress on the charting library today:

  • cleaned up the API a little bit, opting for some nicer make-foo function instead of requiring calls to make-instance:
    • make-series label data &key color
    • make-axis label &key control-string draw-gridlines-p label-formatter
      • control-string: a format-compatible control string, and supplying it sets the label-formatter for the axis
      • label-formatter: a function of 1 argument, returns a string used as an axis label
    • make-line-chart width height &key series y-axis x-axis background
  • The lightened portion of the graph is now the average of the background color and pure white
  • The labels on the axes are optional, and the graph adjusts to fill as big an area as possible
  • The distance between x-axis labes is calculated based on actual string-widths, so hopefully they will space themselves out reasonably under most conditions

Some examples:

This demonstrates using a control-string and a custom function for axis labels:

line-chart-with-axis-labels1.png

  (render-chart
   (make-line-chart 400 300
		    :background '(.7 .7 .7)
		    :series (list (make-series "SeriesA"
					       '((-1 -2) (0 4) (1 5) (4 6) (5 -3)))
				  (make-series "SeriesB"
					       '((-1 4) (0 -2) (1 6) (5 -2) (6 5))
					       :color '(.3 .7 .9))
				  (make-series "SeriesC"
					       '((-1 0) (0 3) (1 1) (2 5) (4 -6))))
		    :y-axis (make-axis "widgets"
				       :control-string "~$")
		    :x-axis (make-axis "time"
				       :label-formatter #'months-from-now->mm/yy))
   "line-chart-with-axis-labels.png")

This one changes the background color, removes the axis labels, and disabled grid lines for the x-axis:

line-chart-with-axis-labels2.png

(render-chart
   (make-line-chart 400 300
		    :background '(.7 .5 .7)
		    :series (list (make-series "SeriesA"
					       '((-1 -2) (0 4) (1 5) (4 6) (5 -3)))
				  (make-series "SeriesB"
					       '((-1 4) (0 -2) (1 6) (5 -2) (6 5))
					       :color '(.3 .7 .9))
				  (make-series "SeriesC"
					       '((-1 0) (0 3) (1 1) (2 5) (4 -6))))
		    :y-axis (make-axis nil
				       :control-string "~$")
		    :x-axis (make-axis nil
				       :draw-gridlines-p nil
				       :label-formatter #'months-from-now->mm/yy))
   "line-chart-with-axis-labels.png")

The internals are still messy, but I’ve gone through and was able to simplify some things here and there, here’s a few of the change’s I’ve made to make my code shorter and more readable:

  • replacing (destructuring-bind (x y) (foo) (bar x y) with (apply #’bar (foo))
  • adding a macrolet to concentrate some very similar behavior
  • using loop to count up and down at the same time (for drawing labels up/down the y axis and left/right on the x axis)
  • using (dolist (item lst) …) instead of (mapc #’(lambda(item) …) lst)
  • using loop’s maximizing / minimizing features
  • using multiple-value-list and nth to get specific values from some function results without compiler warnings

I’ve been trying to put my tongue on why exactly I like lisp so much more than other languages, and working through this made it pretty clear: I have much better tools for reducing accidental complexity than in other languages, and can get the code focusing on what I’m trying to do, not how my compiler needs me to do it. The next time I get a free afternoon, I’m going to make an imperative API, similar to vecto’s:

(with-line-chart (400 300 :background '(.7 .7 .7))
    (add-series "SeriesA"
		'((-1 -2) (0 4) (1 5) (4 6) (5 -3)))
    (add-series "SeriesB"
		'((-1 4) (0 -2) (1 6) (5 -2) (6 5))
		:color '(.3 .7 .9))
    (add-series "SeriesC"
		'((-1 0) (0 3) (1 1) (2 5) (4 -6)))
    (set-axis :y "widgets" :control-string "~$")
    (set-axis :x "time" :label-formatter #'months-from-now->mm/yy)
    (save-file "line-chart-with-axis-labels.png")

That’s pretty damn readable.

updated: fixed syntax error in the imperative sample, and got the imperative bit actually working.

Comments

charting library taking form

January 8, 2008 at 3:17 pm · Filed under code snippet, lisp, vecto

It now does line charts and better pie charts, and has had a lot of bugs removed since the last post. We did some code review on it at work, brought some of the spaghetti under control, and included the library in an intranet app.

Here’s a sample line graph:

line-chart-sample.png

The (somewhat verbose) code for this:

(defun line-chart-sample ()
  "draws a simple line chart"
  (let* ((seriesA (make-instance 'series
				 :label "SeriesA"
					;data expressed as a list (x y) pairs
				 :data '((-1 -2) (0 4) (1 5) (4 6) (5 -3))))
	 (seriesB (make-instance 'series
				 :label "SeriesB"
				 :data '((-1 4) (0 -2) (1 6) (5 -2) (6 5))))
	 (chart (make-instance 'line-chart
			       :width 400
			       :background '(.7 .7 .7)
			       :series (list seriesA seriesB))))
    (render-chart chart "line-chart-sample.png")))

I’ll probably make some easier syntax for creating series, similar to the one I made for pie slices, but for now the CLOS approach has been working decently. My data is coming from clsql, so the code ends up more concise than this example.

There are a few spots internally where it gets a little nasty, needing to pass many things around to many functions. I may end up changing the style to use “with-line-chart” functions with some global variables tracking the current chart, but I’m not sure how that would work in a multi-threaded environment. Hell, I’m not sure if my existing code will work in a multi-threaded environment. The load on this intranet app is very low, so I may be lucky enough to get away with it.

Comments (3)

Graphing my fuel efficiency with lisp and Vecto

October 7, 2007 at 10:01 pm · Filed under code snippet, lisp, open source, vecto

Gonna try to be quick about this, as there’s chicken on the grill. I took a break from Team Fortress 2 to play some more with Vecto, another fine library from Xach. I started another post about last weekend’s lisp playing, but this one got finished first.

I’ve been tracking my fuel efficiency for a few weeks, noting the date, gallons purchased, price per gallon, and miles since last refill. You can’t control what you don’t measure. I had a ruby script keeping up with these as YAML and then giving me some handy stats, but I’m trying to gain some levels in lisp, so I decided to fire up slime and make some graphs.

The graph isn’t pretty, but here’s my miles per gallon:

graph.png

As usual, I pulled up my favorite lisp tutorial and #lisp, and after a few hours, I had a graph. Given my logging fetish, I imagine I’ll be growing the graphing capabilities in the future.

Right now, the implementation is a little weird. I have an class for a fuel entry, and then convert the date to a universal-time using net.telent.date. Then in my graphing function I use the universal time for the x values, and any number of functions for the y values (defaulting to #’mpg-of). In order to get the graph to take up most of the canvas, I then normalize the x and y values, so the minimum is 0 and the maximum is 1, and then scale the number to fit the canvas size before drawing.

I still consider myself a lisp beginner, and there are some bits I’m sure could be more efficient, but it works for now. Those interested in the code can get it here: fuelman.tar.gz, and any feedback in comments is much appreciated.

Update: Played with it a lot more, simplified my normalization crap a lot using loop instead of mapcar, and pulled it into a couple of objects: graph, graph-series, and fuel-graph-series.  Also linked the source above as a proper .tar.gz file, after reading how to do that in wordpress.  The aesthetics are still terrible, but it’s getting there:

graph1.png

Comments

reading passwords from the console in C#

July 5, 2007 at 6:17 pm · Filed under C#, code snippet

I'm working on a simple command-line app, and have the need to collect a username and password. I don't want the password to be printed on the screen as they type, but System.Console.ReadLine doesn't seem to have any option to mask the input before it's echoed back to the console.
There are a couple ways I found to resolve this, the easy way, and the harder way with better UI.

Easy way:

PLAIN TEXT
C#:
  1. ConsoleColor oldFore = Console.ForegroundColor;           
  2. Console.ForegroundColor = Console.BackgroundColor;
  3. string password = Console.ReadLine();
  4. Console.ForegroundColor = oldFore;

This basically hides the echoed input by setting the text color to be the same as the background color. This is a little weird because your users can't see any indication that they've entered any information.

The following function catches the input, and then echoes "*" instead. There's some rudimentary handling of backspace, but no other special keys (end, delete, arrow keys, etc) are handled properly. For now it's good enough for my use:

PLAIN TEXT
C#:
  1. public static string ReadPassword() {
  2.   Stack<string> passbits = new Stack<string>();
  3.   //keep reading
  4.   for (ConsoleKeyInfo cki = Console.ReadKey(true); cki.Key != ConsoleKey.Enter; cki = Console.ReadKey(true)) {
  5.     if (cki.Key == ConsoleKey.Backspace) {
  6.       //rollback the cursor and write a space so it looks backspaced to the user
  7.       Console.SetCursorPosition(Console.CursorLeft - 1, Console.CursorTop);
  8.       Console.Write(" ");
  9.       Console.SetCursorPosition(Console.CursorLeft - 1, Console.CursorTop);
  10.       passbits.Pop();
  11.     }
  12.     else {
  13.       Console.Write("*");
  14.       passbits.Push(cki.KeyChar.ToString());
  15.     }
  16.   }
  17.   string[] pass = passbits.ToArray();
  18.   Array.Reverse(pass);
  19.   return string.Join(string.Empty, pass);
  20. }

A bit messy and bug-ridden, but for now it's good enough for my purposes. When C# 3 comes out, I might be able to add things like this to the Console class using extension methods. Maybe a community-driven FrameworkPlus library will get some momentum, and we can all ditch our home-grown Utils libraries and reap the benefits of each other's work.

Comments

EventHandlerList, key equality, and auto-boxing in C#

May 25, 2007 at 1:29 pm · Filed under C#, annoying, code snippet

I was recently implementing some custom events, and found a couple of good (if old) articles describing how to do this efficiently using EventHandlerList:

Those articles go into why it's nicer to deal with one EventHandlerList instead of many seperate EventHandlers, so read those for more information. For the lazy, here's some code showing how you're supposed to use these things:

PLAIN TEXT
C#:
  1. public class MyClass {
  2.     private EventHandlerList Events = new EventHandlerList();
  3.  
  4.     public event EventHandler MyEvent {
  5.         add { Events.AddHandler("MyEvent", value); }
  6.         remove { Events.RemoveHandler("MyEvent", value); }
  7.     }
  8.  
  9.     public event EventHandler MyOtherEvent {
  10.         add { Events.AddHandler("MyOtherEvent", value); }
  11.         remove { Events.RemoveHandler("MyOtherEvent", value); }
  12.     }
  13.  
  14.     protected void OnMyEvent(object sender, EventArgs e) {
  15.         EventHandler handler = (EventHandler) Events["MyEvent"];
  16.         if (handler != null) {
  17.             handler(sender, e);
  18.         }
  19.     }
  20.  
  21.     protected void OnMyOtherEvent(object sender, EventArgs e) {
  22.         EventHandler handler = (EventHandler) Events["MyOtherEvent"];
  23.         if (handler != null) {
  24.             handler(sender, e);
  25.         }
  26.     }
  27. }

Pretty straightforward stuff. When you add an event handler to the list, you associate it with a key, and then when its time to trigger the events, you look for any handlers under the same key. The other day I was putting together something similar, and ran into some unexpected behavior with the keys. I had started by refactoring the magic strings into an enum:

PLAIN TEXT
C#:
  1. protected enum MyEvents {
  2.         MyEvent,
  3.         MyOtherEvent
  4.     }

and replaced all the strings with members of that enum. I figured this would work just fine, but the change caused my unit test to fail. Upon debugging, the EventHandlerList was always returning null in my On*Event calls. After some more testing, the pattern became apparent: value types don't work as keys. This was somewhat unexpected, as I've used enums like this in Hashtables all over the place before. After doing a little Reflectoring, the actual search for the key comes down to traversing a linked list with a simple equality test, something like this:

PLAIN TEXT
C#:
  1. while (head != null)
  2.     {
  3.         if (head.key == key)
  4.         {
  5.             return head;
  6.         }
  7.         head = head.next;
  8.     }

The culprit ends up being C#'s auto-boxing. The key is stored as an object, so my value types are being boxed on the way in, and therefore == is comparing object identity, not the object values. If EventHandlerList used head.key.Equals(key), everything would have worked how I expected. The solution to rid myself of magic strings now becomes using static objects as my keys, so the object identities will match:

PLAIN TEXT
C#:
  1. private static readonly object MyEventKey = new object();
  2. private static readonly object MyOtherEventKey = new object();

That pattern reminds me a lot of enums in Java before it got a enum keyword, which came on the heels of C#'s nice solution to the enumerated type problem. It'd be nice if I could use enums for their intended purpose, but cases like this make me a bit wary. Where else in the .NET framework am I going to find object identity equality where I expect to find object value equality? Is there some rational explanation for this, or is this just a bug?

Comments

Pretty icons using CSS and Mark James’ Silk Icons

May 18, 2007 at 12:39 pm · Filed under code snippet, design, open source

I recently saw a post via reddit on Rediscovering the button element about making nice buttons with little icons, and was introduced to the Silk Icons available from Mark James. Mark has released these under a Creative Commons Attribution license, any my UIs will be better for it.

I went through last night and gave a little face-lift to one of my intranet projects, and it looks a lot nicer now. I added icons to buttons and links using CSS.

Here's the css I used:

PLAIN TEXT
CSS:
  1. .icon
  2. {
  3.     background-repeat:no-repeat;   
  4.     padding-left:20px;
  5. }
  6.  
  7. .delete-icon
  8. {
  9.     background-image:url('icons/cross.png');
  10. }
  11.  
  12. .edit-icon
  13. {
  14.     background-image:url('icons/pencil.png');
  15. }

and so forth, with a *-icon for the different icons I wanted. Then, I decorated my buttons, links, and what-have-you by adding a class="icon foo-icon" attribute. There are other ways accomplish that with CSS, but I liked this scheme the best. In the end, I have added nothing significant to the markup, but made everything else look a lot nicer:

Silk icon example 3Silk icon example 2Silk icon example

Thanks a lot, Mark!

Comments

Restart a windows service remotely

January 4, 2007 at 3:54 pm · Filed under code snippet, nant, windows

I've been working with Shibboleth, an Internet2 project to ease authentication and authorization across institutions. So far, its a mess of XML files, and its been a lot of guess and check on various magic strings. One annoyance in working with it was that I'd update the config files, and then need to remote into the server to restart the windows service using those files. Shibboleth will reread some of the config, but not all, so to be safe I just restart it.

After making a nant task to publish the latest config files for me, the need to click around to restart got very annoying. I wanted to get a way to restart it from the command line so I could restart it from my nant task. After a little googling and some futzing, I got it.

The solution

The program to use is sc.exe, a Service Controller. Unlike net.exe, sc lets you specify a server, so you can do things remotely.

Find the service name

You'll need the exact service name, which is stored in the registry. An easy way to find it is to enumerate all the services on the box, and then usually you can pick it out from there.

$ sc.exe \\\\server query | less

That will be a long list, so pipe it to less and page through it. That will list an entry for each service, like so:

SERVICE_NAME: shibd_Default
DISPLAY_NAME: Shibboleth 1.3 Daemon (Default)
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE,NOT_PAUSABLE,IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

This lets me know the service name I want to be using is "shibd_Default".

Stopping the service

Stopping is another simple command:

$ sc.exe \\\\server stop shibd_Default

SERVICE_NAME: shibd_Default
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 3  STOP_PENDING
                                (STOPPABLE,NOT_PAUSABLE,IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x2
        WAIT_HINT          : 0x0
$

Note that is doesn't wait to stop the service, it merely makes a request to stop, then prints out the current status. The status is predicably STOP_PENDING. It's important to note that sc.exe doesn't wait for the service to stop, and that the request to stop may fail.

Starting the service

Starting is another easy command:

$ sc.exe \\\\server start shibd_Default

SERVICE_NAME: shibd_Default
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 2  START_PENDING
                                (NOT_STOPPABLE,NOT_PAUSABLE,IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x1
        WAIT_HINT          : 0xbb8
        PID                : 2628
        FLAGS              :

This also only puts in the request for the service to start. As with stop, this request may fail.

Restarting using nant

To put it all together in nant, here's the plan:

  1. Send the stop request
  2. Sleep a little bit, to give the service a chance to stop
  3. Send the start request
  4. Sleep a little bit, to give the service a chance to start
  5. Send a query request, to check that it started ok

Here's a simplistic snippet of Nant xml for this:

PLAIN TEXT
XML:
  1. <!-- Send the stop request -->
  2. <exec program="sc.exe">
  3.   <arg line="\\server stop shibd_Default"/>
  4. </exec>
  5. <!-- Sleep a little bit, to give the service a chance to stop -->
  6. <sleep seconds="5"/>
  7. <!-- Send the start request -->
  8. <exec program="sc.exe">
  9.   <arg line="\\server start shibd_Default"/>
  10. </exec>
  11. <!-- Sleep a little bit, to give the service a chance to start -->
  12. <sleep seconds="5"/>
  13. <!-- Send a query request, to check that it started ok -->
  14. <exec program="sc.exe">
  15.   <arg line="\\server query shibd_Default"/>
  16. </exec>

The sleep time of 5 seconds probably needs to be adjusted for different services.

Conclusion

For once I'm not horribly disappointed in the tools microsoft has provided me. It would be nice if there was a "restart" command, or if you could tell sc.exe to wait until the service actually started or stopped, but there's at least something workable. Possible improvements would be to make this poll using sc \\server query shibd_Default to check for when the service actually stops or starts, I imagine 5 seconds isn't a universal upper bound.

If I get ambitious I might package up a nant task wrapping sc and send it to nantcontrib.

Comments (4)

C# generics and XML deserializing

December 29, 2006 at 12:45 pm · Filed under C#, code snippet

I've been playing with generics more, trying to ease the slog of C# development. The .NET framework allows you to do damn near everything, but its so freaking verbose. Almost anything I want to do with framework classes ends up taking 4-5 nearly identical lines, so I end up writing a lot of little wrapper functions to make one-liners.

The task I had at hand this time was to take a string of XML and deserialize is back into a C# object. I generated the C# object off of an .xsd file using xsd.exe (more info), so it has the spaghetti of attributes needed for the XmlSerializer.

public static T DeserializeFromXml<T>(string xml){
	T result;
	XmlSerializer ser = new XmlSerializer(typeof(T));
	using(TextReader tr = new StringReader(xml)) {
		result = (T) ser.Deserialize(tr);
	}
	return result;
}

With that utility function, now I can just:

MyClass obj = Utils.DeserializeFromXml<MyClass>(xml);

Working with these libraries, I always end up pondering why Microsoft didn't think to include shortcuts like this.

Comments