Colourblind

So what with my work on Oh Teh Noes I was wondering if I could co-opt it to do performance monitoring. As it happens, I couldn't figure out how to do it without doing some serious square-pegging, but it did get me thinking about how I would like to attack the problem. And as is so often that case that eventually turned into me actually attacking the problem. That happens a lot. I nerd snipe myself daily.

And thus Regarding Your Performance was born. It consists of a Windows service which sits on the host machine and farms basic performance data, and a HTML page which collects that data from a list of hosts via AJAX and drops it into some graphs. It became an excuse for me to learn some new things, so I thought I'd share the experience. This is a fairly high-speed tour so you might want to saunter over to Github and grab the source

InstallUtil Can Die in a Fire

I wanted the agent (the side of the system which resides on the target host and harvests delicious, delicious performance information) to run as a Windows service. But there's one particular element of Windows services that I find especially uncomfortable . . .

InstallUtil.exe can piss right off.

I've been packaging that dowdy little app with all of my Windows Services for years now and enough is enough. It's time to find a more elegant solution. Fire up Reflector and it quickly becomes apparent that InstallUtil doesn't actually do a lot. In essence it simply creates a ManagedInstaller, which itself is a thin wrapper around a TransactedInstaller. By taking that code and then adding on the contents of the usual Service designer files you can roll your own pretty easily. I ended up using command line arguments to install and uninstall the service, resulting in a Main method like this:

   1:  static void Main(string[] args)

   2:  {

   3:      if (args.Length > 0)

   4:      {

   5:          if (args[0] == "-i" || args[0] == "-u")

   6:          {

   7:              ServiceInstaller serviceInstaller = new ServiceInstaller();

   8:              serviceInstaller.ServiceName = "Performance.Agent.Service";

   9:              serviceInstaller.StartType = ServiceStartMode.Automatic;

  10:              serviceInstaller.DisplayName = "Colourblind Performance Agent";

  11:              serviceInstaller.Description = "Agent for the Colourblind performance monitor";

  12:   

  13:              ServiceProcessInstaller processInstaller = new ServiceProcessInstaller();

  14:              processInstaller.Account = ServiceAccount.LocalSystem;

  15:              processInstaller.Username = null;

  16:              processInstaller.Password = null;

  17:   

  18:              TransactedInstaller installer = new TransactedInstaller();

  19:              installer.Installers.Add(processInstaller);

  20:              installer.Installers.Add(serviceInstaller);

  21:              installer.Context = new InstallContext("install.log", null);

  22:              installer.Context.Parameters.Add("assemblypath", Assembly.GetCallingAssembly().Location);

  23:   

  24:              if (args[0] == "-i")

  25:                  installer.Install(new Hashtable());

  26:              else if (args[0] == "-u")

  27:                  installer.Uninstall(null);

  28:          }

  29:      }

  30:      else

  31:      {

  32:          ServiceBase[] ServicesToRun;

  33:          ServicesToRun = new ServiceBase[] 

  34:          { 

  35:              new Service() 

  36:          };

  37:          ServiceBase.Run(ServicesToRun);

  38:      }

  39:  }

Next up, how to poll our host system for performance information.

Real-time System Information Voyeurism

PerformanceCounter is the .NETified way of querying the Windows for performance metrics.

   1:  PerformanceCounter cpu = new PerformanceCounter("Processor", "% Processor Time", "_Total");

   2:  PerformanceCounter memory = new PerformanceCounter("Memory", "Available MBytes");

   3:  PerformanceCounter requestsPerSecond = new PerformanceCounter("ASP.NET Applications", "Requests/Sec", "__Total__");

The strings you use to instantiate PerformanceCounters come firmly in the realm of magic. They're not terribly well documented and in all honesty you're better off just opening up Windows' Performance Monitor (WinKey-R 'perfmon') and seeing which counters you can add through that. It'll even give you a convenient description of quite what some of the more esoteric ones mean. Failing that you can enumerate them yourself

It would make a lot of sense to create our performance counters in the constructor, but it turns out they can spend a lot of time sitting around doing nothing which then causes net start to report a timeout. This is not the case if we move the object instantiation to an OnStart event of the service though, so that's what I ended up doing. The last thing on the startup todo list is to get the total amount of physical memory for the host, and for this we take a brief forray into the nightmare realm of WMI.

   1:  ManagementObjectSearcher wmi = new ManagementObjectSearcher("select * from Win32_ComputerSystem");

   2:  foreach (ManagementObject o in wmi.Get())

   3:      TotalPhysicalMemory += Convert.ToSingle(o["TotalPhysicalMemory"]) / (1024 * 1024);

Now we simply poll the counters every second and store the results in a dictionary.

   1:  Status["CpuUsage"] = cpu.NextValue().ToString();

   2:  Status["MemoryUsage"] = ((TotalPhysicalMemory - memory.NextValue()) * 100 / TotalPhysicalMemory).ToString();

   3:  Status["RequestsPerSecond"] = requestsPerSecond.NextValue().ToString();

All well and good, but now our service has shut-in syndrome, and that frankly will not fly.

Bare Bones HTTP

At this point I had only vague ideas about the form that my client would take. With that in mind I wanted something flexible enough that my later decisions wouldn't be affected by my earlier design choices. With that in mind I decided to go for HTTP as my application protocol, and JSON as my information format.

.NET's HttpListener provides an itty-bitty HTTP server, capable of asynchronous responses and not a lot else. That said, it's perfect for small HTTP-enabled apps like this.

   1:  string prefix = String.Format("http://+:{0}/", ConfigurationManager.AppSettings["ServerPort"]);

   2:  HttpListener httpListener = new HttpListener();

   3:  httpListener.Prefixes.Add(prefix);

   4:  httpListener.Start();

   5:   

   6:  // 10 worker threads

   7:  for (int i = 0; i < 10; i ++)

   8:      httpListener.BeginGetContext(new AsyncCallback(RequestCallback), null);

Besides that, you're on your own. You handle each request as you see fit in your request handler, and once you've done what you need to you spool up another listener to replace the one you've just 'used', then wait for more calls to come rolling in.

   1:  private void RequestCallback(IAsyncResult result)

   2:  {

   3:      HttpListenerContext context = null;

   4:      try

   5:      {

   6:          context = HttpListener.EndGetContext(result);

   7:   

   8:          string output = "{";

   9:          bool foo = false;

  10:          foreach (string key in Status.Keys)

  11:          {

  12:              output += String.Format("{2}\n\t{0} : '{1}'", key, Status[key], foo ? "," : "");

  13:              foo = true;

  14:          }

  15:          output += "\n}";

  16:   

  17:          Encoding encoding = new UTF8Encoding(false); // Whenever I have UTF8 problems it's BOM's fault

  18:          byte[] outputBytes = encoding.GetBytes(output);

  19:   

  20:          context.Response.OutputStream.Write(outputBytes, 0, outputBytes.Length);

  21:      }

  22:      finally

  23:      {

  24:          if (context != null && context.Response != null)

  25:              context.Response.Close();

  26:          HttpListener.BeginGetContext(new AsyncCallback(RequestCallback), null);

  27:      }

  28:  }

Now we can consume the one-trick web service via AJAX requests spawned from an HTML page!

That's Not a Client - It's an HTML Page

Damn right it's an HTML page. Why bother with apps and servers and all that cruft when all you need is a few lines of Javascript? The 'client' is a simple HTML page containing a list of active agents which are polled every second and the results are used as the data for a set of sparklines graphs care of a jQuery plugin. The only gotcha here is the scale on the requests/sec graph. We can treat CPU and memory usage as percentages, but requests/sec is an absolute value so you may need to tweak the y scale accordingly.

Hmmm. Hang on a minute . . .

Oh crap, I forgot about the Same Origin Policy

Every time I think of something cool to do with a web client the Same Origin Policy comes and bites me on the arse. SOP states that an XmlHttpRequest can only be sent to the same domain as the Javascript file which initiates it, and therefore prevents the kind of cross-domain AJAX shenanigans we're attempting here.

JSONP is a cheeky little hack designed to circumvent this very restriction. Rather than being just Javascript object literal notation, as is the case with JSON, JSONP actually returns a javascript source file which is fetched by the host page using a script block, dodging the SOP. When making a call to a JSONP-aware service you provide the name of the function that is ready and waiting to process the response, and the service wraps it's JSON response in a call to that function so that when the file is fetched by the client the response handler is executed. Conceptually it feels weird as you're essentially pulling a pushing, or maybe pushing a pull, but getting down to the brass tacks this means we have to add two lines to our request handler:

   1:  string callback = context.Request.QueryString["callback"];

   2:  output = String.Format("{0}({1})", callback, output);

On the client side there is a handy jQuery JSONP plugin which makes for a similarly smooth transition.

~fin

Those are the bits that interested me, at least. Feel free to grab the code and have an explore. It's a testament to the .NET framework that it packs so much into so few lines of working code and made for a fun train-journey-home-from-work project over a few days.

"Could not load file or assembly 'OmgWeAreAllGoingToDie' or one of its dependencies. There is not enough space on the disk. (Exception from HRESULT: 0x80070070)"

We've all seen it. Alice was on holiday when it was her turn to clear out the IISLogs. Or Bob forgot to BACKUP LOG before he DBCC SHRINKFILEd. Or Chris didn't read that email about renewing the SSL certificate. Either way, the site's down and the clients are outside the office with pitchforks and burning brands and baying for your blood. Now I only have two readers, so if one of you is violently exsanguinated by a marauding mob of users that cuts my readership in half, which would be really bad. With that in mind Oh Teh Noes is a small, light, extensible little app that warns you when terrible, terrible things are about to happen to your server.

Is that right?

Yarr.

Oh Teh Noes is a console app that accepts an XML file as an argument. That XML file contains a list of tasks to be run, which run some fairly standard and roughly customisable checks, such as:

• Is the host machine running out of disk space?
• Is the site's SSL certificate about to expire?
• Is a daily updated file more than a daily old?

So how do I use it?

Oh Teh Noes is designed to be run as a scheduled task. If you wanted to run using the task definitions in the dailyTasks.xml file, the command line will simply look like this:

OhTehNoes dailyTasks.xml

The specified XML file contains a list of task definitions, thus:

<tasks logName="Test Logger">
    <task type="TaskTypeA" 
        name="Optional Name" 
        paramterA="100"
        parameterB="foo" />
    <task type="TaskTypeB" parameterA="bar" />
</tasks>

Each task definition lives in its own <task> element. The type is a required attribute, and must correspond to the type of the task (see below for the list of included tasks as of now). The name attribute is optional, but will be included with alerts so may help unravel things if you have the many tasks running on many boxen. Take a look below for the required attributes for the different kinds of task.

It informs you of impending doom using a slightly tweaked version of log4net that contains an extra appender which allows it to send unbuffered emails and add data into the email subject. I've included that as an assembly rather than source. As far as I know the log4net licence permits that, but if someone with a beard comes and punches me in the nuts then I guess I'll stand corrected. Until then: binaries it is. Besides that it's vanilla, so if you're familiar with log4net you'll be right at home. All of the log4net settings are tweakable in OhTehNoes.exe.config.

Tasks

The tasks themselves live in assemblies in the /plugins/ directory, and when run Oh Teh Noes will scan this directory for assemblies, and scan those assemblies for valid tasks. What follows is a list of the tasks provided out of the box, and attributes you need to configure them.

DiskSpace

• warningThreshold

Checks all of the physical drives in the machine and throws a warning if any of them have less MB remaining than is specified by warningThreshold.

SslCertificate

• certsToCheck
• warningThreshold

Reads the SSL certificates for the local machine, and then it finds one for a domain (or subdomain) in the comma-seperated list in certsToCheck it makes sure it has more than the amount of days specified in warningThreshold until it expires, or it throws a warning.

FileUpToDate

• filename
• thresholdInMinutes

If the file specified in filename hasn't been modified in the last thresholdInMinutes minutes, sirens sounds. We tend you use this one to make sure files we should be receiving daily from third parties actually arrive.

QueryReturnsRows

• connectionString
• sqlQuery

When this task it run, it connects to the SQL server specified by connectionString, runs the query in sqlQuery and spits its dummy if no rows are returned. We use this one on some of our clients' sites to make sure that we've had an order in the last hour during the normal working day, like so:

SELECT
    TOP 1 *
FROM
    [Orders]
WHERE
    [DateCreated] > DATEADD(hour, -1, GETDATE())
    OR DATEPART(hour, GETDATE()) < 10
    OR DATEPART(hour, GETDATE()) > 18

And it's open source, right?

Yup.

}

I think that about covers it. If you can make use of it, please do. If you find a bug, let me know. If you can think of more features or plugins, drop me a line or have a go at implementing yourself.

P.S. From a code perspective the main point of interest is the plugin system, and my next post will probably go into that in a bit of detail. Beyond that it's all fairly standard stuff, but it solves a problem we had and I can think of no better raison d'être for a piece of code.

Every now and then I dabble with making my own screensavers. I was one of the 5 people who were shaking their fists in the direction of Redmond when it became apparent that the version of ScrnSave.dll packaged with Visual Studio 2008 depended on Vista. As is so often the case the last round when belly-up when I got distracted, magpie-like, by some other glittering object. I had quite a decent piece of boilerplate code I could use to reduce the spool-up time and get straight to the fun bit. I was considering tidying it up and opening the source. But then I stumbled across Cinder. I spent a few hours prodding around it, then promptly threw out all of my code because it's just so, so bad by comparison.

Curse you Cinder, you Magnificent Bastard

Cinder sell itself as library for 'creative coding' and it delivers in spades. It provides code for window creation, management and tedium, input, sound, loaders for a healthy selection of image formats and a wrapper for OpenGL which goes a decent way to alleviating the, uh, idiosyncrasies of the library. And it makes writing screensavers a breeze. It's even cross-platform enough to work on a Mac if you're that way inclined.

TTIUWP

As such I was able to knock off the following fairly quickly.

FillrateNomNomNom - a simple particle system

Churn - avoid when drunk

TimeForBedSaidZebedee - hierarchically organised spinning circles

Cubesplosion - Cubes that subdivide and scatter

But yeah, I'm pretty happy with how it came out. They're all very much work in progress and I'm not finished yet. This kind of thing is like code doodling for me. I love it. Expect more in the future.

Yoink

Source on github
Download the binaries

Due to the aforementioned VS2008 ScrnSave.dll SNAFU, there are different binaries for XP and Vista/Win7. Likewise if you download the source you'll find several different build targets: standalone app, XP screensaver and Win7 screensaver, with debug and release for each. Build configurations galore.

And as a passing thought: I wonder how it would work as a game library? It removes a lot of the bullshit (windowing, input, sound, resource loading, message pump) that can be a hindrance when starting out, while still allowing you to get low-level with the fun bits (graphics, game logic, physics). I'd have a crack myself if I wasn't working on my own game library already (STOP TAKING MY LUNCH MONEY, CINDER).

So I made a database schema comparison tool for MSSQL 2005.

Compares Favourably

It started out as a hobby project since I saw SMO and assumed it would be simple to make something that would freak out if we tried to push a staging release to live without first getting the database structure the same. Turns out SMO is a whole bus-load of fail for churning out large amounts of DDL, so I started writing my own DDL generation code. Then I put a simple UI over the top of it because I was curious about WinForms, and it started growing from there.

You can download the source as well (static downloads, rather than repository since my build discipline is pants and half the time it doesn't even compile), which I'm releasing under the <shrug/> licence, also known as the Meh, Whatever PL. If you can find a use for it then you're more than welcome, but please let me know - I'm curious if the code ends up anywhere cool.

In other news best Christmas #1 ever.