Shog's Worklog - Tips

Fit acceptance testing and OpenWiki

nospam@example.com (Shog9) — Sun, 24 Jun 2007 17:16:27 -0700

NUnit now includes the Framework for Integrated Test (Fit) engine. I'm not entirely sure why (there seems to be some support for using it to run NUnit tests) but my curiosity upon seeing the new DLL led me to read more about Fit. And it couldn't have been more timely - the project I was working on had something like 47 test cases taken directly from sales orders, complete with supplier-provided "known good" outputs - a perfect ahem fit for Fit testing.

And what a beautiful thing it is. A few minutes work, and the Excel spreadsheet of test cases was a HTML document suitable for feeding into the .NET version of Fit - much preferable to the manual translation of each scenario into a NUnit test, or (shudder) manually entering each case into my interactive testing app. I'd been planning on creating a bulk-testing interface anyway, and this just seemed too good to be true - it was almost exactly what I'd been planning, and saved me the time and effort of writing and debugging a custom CSV parser + test harness.

The only thing lacking was integration with our wiki, which was the other part of my idle plan... Sure, there's FitNesse - but that just solves one problem with yet another wiki to maintain.

So I hacked together an ASP.NET runner for Fit, threw it on our build machine, and then built in a new processing instruction to let OpenWiki forward on a page to this service for processing.

Satisfaction!

In case anyone else has a use for it, here's the code required:

OpenWiki #FITTESTS processing instruction (modify mywiki.asp)

Function MyWikifyProcessingInstructions(pText)
       
   ' other custom processing instructions...
   
   ' Fit integration: detect Fit processing instruction, flag for later processing
   dim FitPI, FitPILen
   FitPI = "#FITTESTS"
   FitPILen = Len(FitPI)
   if m(pText, "\n" & FitPI & "\s+", False, False) Then
      dim lineStart, lineEnd
      lineStart = InStr(pText, FitPI)
      lineEnd = InStr(lineStart, pText, vbCR)
      If lineEnd > lineStart+FitPILen Then
         gFitTestPath = Trim(Mid(pText, lineStart+FitPILen, lineEnd - lineStart - FitPILen))
      End If
      pText = s(pText, "\n" & FitPI & "[^\r\n]*\r*\n", "", False, False)
   End If

   ' other custom processing instructions...
   
    MyWikifyProcessingInstructions = pText
End Function

Additional modifications to mywiki.asp - these actually call the testing web service after the page has been mostly built.

Function MyLastMinuteChanges(pText)

   ' other custom last-minute changes...

   ' Fit integration
   If (gAction = "view" and Len(gFitTestPath) > 0 ) Then
      dim TestResults
      TestResults = RunFitTests(pText, gFitTestPath)
      If ( err.number <> 0 ) then
         pText = "<i>Error running tests: " & err.description & "</i><br />" & pText
      Elseif ( Len(TestResults) = 0 ) then
         pText = "Unknown error running tests" & pText
      Else
         pText = TestResults
      End If
   End If

   MyLastMinuteChanges = pText
End Function

Function RunFitTests(pText, gFitTestPath)
   dim url
   dim verb
   dim form

   ' location of FIT service. Need not be local.
   url = "http://localhost:8088/FitService.ashx"
   verb = "POST"
   form = "ClassPath=" & Server.URLEncode(gFitTestPath) _
      & "&TestData=" & Server.URLEncode(pText)
      
   on error resume next  
   
   dim xmlhttp
   set xmlhttp = Server.CreateObject("Msxml2.ServerXMLHTTP.4.0")
   if ( xmlhttp is nothing or err.number <> 0 ) then
      err.clear
      set xmlhttp = Server.CreateObject("Msxml2.ServerXMLHTTP")
   end if
   
   dim xmlDoc
   if ( err.number = 0 ) then
      xmlhttp.open verb, url, false ' synchronous
   end if
   
   if ( err.number = 0 ) then
      xmlhttp.setRequestHeader "Content-Type", "application/x-www-form-urlencoded" 
      xmlhttp.send form
   end if
   
   ' try to figure out if it worked or not. And if not, why. VBS error handling... yuck.
   dim status
   if ( err.number = 0 ) then
      status = xmlhttp.status
   end if
   
   if (err.number <> 0) then
   elseif (status <> 200) then 
      Err.Raise 65000, "", xmlhttp.statusText & "(" & status & ")", "", 0
   else
      ' clean up Fit html
      RunFitTests = Replace(xmlhttp.ResponseText, "&nbsp;", " ")
      RunFitTests = Replace(RunFitTests, "<br>", "<br/>")
      RunFitTests = Replace(RunFitTests, "<hr>", "<hr/>")
      RunFitTests = Replace(RunFitTests, "<font size=-1", "<font size='-1'")
   end if  
   
   set xmlhttp = nothing  
End Function

FitService.ashx - quick'n'dirty Fit runner. You might want to add some error handling...

<%@ WebHandler Language="C#" Class="FitService" %>

using System;
using System.Web;
using fit;

public class FitService : IHttpHandler
{

   public void ProcessRequest(HttpContext context)
   {
      context.Response.ContentType = "text/html";
      context.Response.ContentEncoding = System.Text.Encoding.UTF8;

      Fixture.assemblyDirs = context.Request.Form["ClassPath"].Split(new char[]{';'});
      
      Fixture fixture = new Fixture();
      Parse tables = new Parse(context.Request.Form["TestData"]);
      fixture.doTables(tables);
      tables.print(context.Response.Output);
   }

   public bool IsReusable
   {
      get
      {
         return false;
      }
   }
}

With these in place, i can add a line to the beginning of any wiki entry:

#FITTESTS c:\MfgEngineInterface\Current build\

...followed by a table containing the tests, and the tests will be automatically run and their results provided upon page generation.

Dump calls

nospam@example.com (Shog9) — Thu, 22 Mar 2007 11:56:00 -0700

I'm working with a 3rd-party library that does some machine-specific calculations. It's called through COM interop, and takes a ton of parameters. When something doesn't come out right, it's nice to be able to quickly view what's being send in and returned without spending an excessive amount of time in the debugger... so I use this quick-and-dirty little wrapper to dump them to the output window:

using System;
using System.Collections.Generic;
using System.Text;
using System.Reflection;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Proxies;
using System.Runtime.Remoting.Messaging;
using System.Runtime.InteropServices;
using System.Diagnostics;

public class DebugTraceProxy
   : RealProxy
{
   readonly MarshalByRefObject target;

   public static T NewObject<T>()
   {
      return (T)(new DebugTraceProxy(typeof(T)).GetTransparentProxy());
   }

   private DebugTraceProxy(Type t)
      : base(t)
   {
      target = (MarshalByRefObject)Activator.CreateInstance(t);
   }
   
   public override IMessage Invoke(IMessage msg)
   {
      IMethodCallMessage call = msg as IMethodCallMessage;

      Debug.WriteLine(call.MethodName + ":");
      for (int i = 0; i < call.InArgCount; ++i)
      {
         string name = call.GetInArgName(i);
         object arg = call.GetInArg(i);
         Debug.WriteLine("   " + name + ": " + DumpOb(arg));
      }

      IMethodReturnMessage ret = RemotingServices.ExecuteMessage(target, (IMethodCallMessage)msg);

      Debug.WriteLine("Returned:");
      for (int i = 0; i < ret.OutArgCount; ++i)
      {
         string name = ret.GetOutArgName(i);
         object arg = ret.GetOutArg(i);
         Debug.WriteLine("   " + name + ": " + DumpOb(arg));
      }
            
      return ret;
   }

   private string DumpOb(object ob)
   {
      Type t = ob.GetType();
      if (t.IsPrimitive || t == typeof(string))
         return ob.ToString();

      string obDesc = "";
      FieldInfo[] infos = t.GetFields(BindingFlags.FlattenHierarchy | BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance );
      if (infos.Length == 0)
         return ob.ToString();
      foreach (FieldInfo field in infos)
      {
         if (obDesc != "")
            obDesc += ", ";
         obDesc += field.Name + ": " + field.GetValue(ob).ToString();
      }
      return obDesc;
   }
}

To use it, I create the object like so:

CrazyOb myCrazyOb = DebugTraceProxy.NewObject<crazyob>();

Reflection... handy.

VS2005 DataTips for MFC collection classes (CMap, CArray, etc.)

nospam@example.com (Shog9) — Sat, 10 Mar 2007 17:19:00 -0700

I spend a lot of time debugging old code. Tracing through libraries written years ago, before the VC++ STL had reached maturity. And one of the often-frustrating aspects of this is how difficult it is to examine the contents of the old, MFC collection classes. CDWordArray, CStringArray, CMapPtrToPtr... not to mention the various templatized versions of these classes. While sometimes these can be easily replaced with newer, better-supported STL equivalents, often the effort involved outweighs the benefit.

At least for the array classes, it's only a minor inconvenience: the array pointer, and the size of the array are both readily available, and with these it's easy enough to view the contents.

But for the CMap classes, things get ugly. Internally, these objects are hash tables, with an array of buckets and each bucket a linked list. Viewing the contents of one of these maps in the debugger requires manually tracing the list of each bucket, then backing up and trying another bucket, then another... until you've found the item, or just given up.

The std::map<> class, implemented as a red-black tree, should be just as much of an annoyance to view - however, it gets flattened into what looks like an array:

Handy, eh? So enough of tracing through buckets.
Open \Program Files\Microsoft Visual Studio 8\Common7\Packages\Debugger\autoexp.dat, and at the end of the [Visualizer] section, add:

;------------------------------------------------------------------------------
;  CArray
;------------------------------------------------------------------------------
CArray<*>|CDWordArray|CWordArray|CByteArray|CUIntArray|CPtrArray|CObArray|CStringArray{
   preview
   (
      #( "[", [$c.m_nSize], "](",
         #array 
         (
             expr : $c.m_pData[$i],
             size : $c.m_nSize
         ),
      ")")
   )
   
   children
   (
      #(
         #array 
         (
             expr : $c.m_pData[$i],
             size : $c.m_nSize
         )
      )   
   )
}

Start a debug session. You should now be able to quickly examine the various CArray (CDWordArray...) objects as easily as std::vector<...>s. Note that the preview (...) section controls what's viewed in the initial DataTip or watch line, while the children(...) section controls what's displayed when you expand it.

Now for the fun one. Start with:

;------------------------------------------------------------------------------
;  CMap
;------------------------------------------------------------------------------
CMap<*>|CMapPtrToPtr|CMapPtrToWord|CMapWordToPtr|CMapWordToOb|CMapStringToPtr|CMapStringToOb|CMapStringToString{
   preview
   (
      #( "[", [$c.m_nCount], "](",
         #array 
         (
             expr : $c.m_pHashTable[$i],
             size : $c.m_nHashTableSize
         ) : #list (
                 head : &$e,
                 next : pNext
             ) : $e,
      ")")
   )
   
   children
   (
      #(
         #array 
         (
             expr : $e.m_pHashTable[$i],
             size : $e.m_nHashTableSize
         ) : #list (
                 head : &$e,
                 next : pNext
             ) : $e
      )         
   )
}

Ah, now this one will save some serious time - it'll flatten the various CMap objects such that they look like arrays in the debugger. A CMap with 3000 items will show 3000 children, numbered 0-2999.

But it can still be better: each of those children is a key-value pair, specifically a CMap::CAssoc object. It'd be nice if we could get a preview of those keys and values as we scroll through those 3000 items, rather than having to expand each one...

CMap<*>::CAssoc|CMapPtrToPtr::CAssoc|CMapPtrToWord::CAssoc|CMapWordToPtr::CAssoc|CMapWordToOb::CAssoc|CMapStringToPtr::CAssoc|CMapStringToOb::CAssoc|CMapStringToString::CAssoc {
   preview
   (
      #( 
         "(", 
         $e.key, 
         ",", 
         $e.value, 
         ")"
      )
   )
   
   children
   (
     #(
         key: $c.key,
         value: $c.value
     )
   )
}

There! All happy.

Of course, you can write rules for your own datatypes too, if you want them to be immediately visible in the preview. For a good reference, see the excellent writeup over at virtualdub.org

Faster C++ compilation on VS2005

nospam@example.com (Shog9) — Thu, 02 Mar 2006 17:02:00 -0700

So after a couple years of putting up with very slow build times, i couldn't take it anymore. We're preparing to migrate to VS2005 for primary C++ development, and a 20x increase in the time needed for a full build just isn't gonna cut it. So i started going through the options, hoping there was an option or something, hoping that they hadn't made the compiler itself that much slower in the process of adding new features and increasing standards compliance.

Sure enough, there was a simple solution. Seems the "automatic" option for using pre-compiled headers isn't available anymore, and so during the conversion PCH was just turned off for all projects.

At this point, i'm going to admit that i've never given too much thought to PCH support in VC++. The only other time i've used PCH at all was with Borland's C++ compiler, where manually specifying what you wanted pre-compiled was the only option (at the time anyway). That It Just Worked on VC++ was enough for me - why mess with what works? But the downside was, i didn't have a very clear idea of what was actually going on. Put the system headers in STDAFX.H, rebuild all, check in and forget it. Upon realizing that this was no longer an option, i didn't really know what to do...

Then i found Bruce Dawson's excellent article on the subject. Going well beyond MSDN's dry descriptions of how, Bruce details the various methods for configuring, optimizing, and using PCH in VC++.

With this, i enabled PCH manually on our various projects, and brought build times back down to something acceptable (roughly 1.5x what it takes in VS6). VS2K5 is usable at last!

Now, if i could just do something about that annoying "Updating Intellisense..."

Painless NTLM authentication in Firefox

nospam@example.com (Shog9) — Wed, 09 Mar 2005 14:23:00 -0700

I've used Firefox heavily at work ever since they built NTLM authentication in, but have always been rather annoyed at how limited it is compared to the support built into IE. With IE there is no difference between accessing an intranet site that requires NTLM, and one that requires no authentication, assuming you're already logged into the network on your machine. Firefox on the other hand, would bring up its authentication dialog for each site, prompting for a username and password. Although allowing Firefox to save the password helped reduce the trouble to simply hitting Enter whenever such a box appeared, it was still annoying, and worse yet, slowed access to these servers.

After being notified by a server admin here that i was causing frequent authentication errors to show up in the logs, i decided it was time to investigate this problem further. Fortunately, a quick Google search brought me several hints at a solution, which was soon put in place.

The fix

In Firefox, type about:config into the address bar and hit enter. You should see a huge list of configuration properties.
Find the setting named network.automatic-ntlm-auth.trusted-uris (the easiest way to do this is to type that into the filter box at top).
Double-click this line, and enter the names of all servers for which NTLM is desired, separated by commas. Then press ‘OK’ to confirm.
Open the options dialog (Tools->Options menu), and on the General page press the Connection Settings button to get the proxy configuration dialog:
Make sure the correct proxy server is configured, and that the same list of servers are listed in the No Proxy for: entryfield as were set in step #3.
Done.

Result? No more slow access, no more authentication errors, no more clearing the list of saved passwords when i change my network password (or worse yet, having to call support and have my password reset because i forgot to do this...)