Archive for the ‘Tools’ Category

Getting “localhost.” to work with IIS7

March 9, 2008

I’m not 100% sure why, but with IIS7, I am unable to get connect to my local IIS instance using Fiddler’s magic “localhost.” hostname.  Normally this hostname allows localhost traffic to be monitored within Fiddler. Out of the box, IIS7 was returning me the following error on requests to “localhost.”:

Bad Request - Invalid Hostname

Trying to add a secondary binding with “localhost.” explicitly set as the host yeilds another error, this time from the IIS Management Console:

Value does not fall within the expect range.

I found that editing Fiddler’s CustomRules.js provides a quick workaround while still allowing you to use the exact “localhost.” name.  Simply add the following line to the “onBeforeRequest” function:

if (oSession.host.ToUpper() == “LOCALHOST.”) { oSession.host = “localhost”; }

This fix of course only works when Fiddler is running, so please comment if you know of a workaround within IIS7.  (Alternatively, please comment if you have no issue in IIS7 — that my box simply has a case of the Vistas.)

LINQ vs LINQ

December 19, 2007

I’m just getting into LINQ, and experimenting with what goes on behind the scenes.  LINQ uses the idea of a “DataContext” as an interface to the database, much like the older “DataSet” acts to encapsulate multiple table structures.   One notably lacking feature of LINQ’s database-derived classes is the ability to traverse many-to-many relationships without touching the intermediate cross-reference table. 

Assuming an existing “vsNetwork” entity, you can query a many-to-many relationship directly against the ”DataContext” ala:

from f in myDataContext.ScienceFocus
join vsf in vsDc.VitalSignFocus on f.FocusId equals vsf.FocusIdf
where vsf.VitalSignIdf == vsNetwork.VitalSignId
select f;

Again assuming an existing “vsNetwork” entity, you can also query by walking the database-derived class’s table-relations, ala:

vsNetwork.FXrefVitalSignFocusList.Select(vsf => vsf.FTluScienceFocus).ToList();

The magic of LINQ exists within all things that implement the IQueriable interface.  This allows the keywords “select”, “where”, “join”, etc. to be applied to a collection of objects.  Note, however, that there are differences in how “IQueriable” is implemented.  The examples above, indeed generate markedly different SQL.  Respectively,

SELECT [t0].* FROM [tlu_Science_Focus] AS [t0] INNER JOIN [xref_Vital_Sign_Focus] AS [t1] ON [t0].[Focus_ID] = [t1].[Focus_IDF] WHERE [t1].[Vital_Sign_IDF] = vsNetwork.Vital_Sign_ID

versus

SELECT [Focus_IDF] FROM [xref_Vital_Sign_Focus]  WHERE [Vital_Sign_IDF] = vsNetwork.Vital_Sign_ID

*For Each Focus_IDF…*

SELECT * FROM [tlu_Science_Focus] WHERE [Focus_ID] = Focus_IDF

We see that the latter runs multiple queries instead of using a SQL join.  The source of this inefficiency is described by The Wayward Weblog:

“[EntitySet's] ToQueryable() wraps your IEnumerable<T> in IQueryable<T> clothing, uses the Queryable infrastructure to let you build up your own expression tree queries, and then when you enumerate it, the expression is rebound to refer to your IEnumerable<T> directly”

In this case, “vsNetwork.FXrefVitalSignFocusList” is of type EntitySet, which does not implement IQueriable directly, but rather through IEnumerable.  So not only does this shorter syntax lose efficiency by issuing multiple and superfluous queries, it also loses efficiency by iterating through each xref table in memory.  In the case of large sets, using LINQ against IEnumerables can be an issue.  It has been enough of an issue that people are adding indexing for in-memory LINQ objects, rather than simply iterating collections.

In earlier Visual Studio 2008 CTP’s, LINQ EntitySets implemented the iQueriable interface (directly?).  I’m not sure if at that point, this was implemented to elegantly generate SQL, I simply know that O’Reilly’s C# in a Nutshell author Joseph Albahari made some fuss about the change. 

It appears that Mr. Albahari is behind a library called LINQKit designed to mitigate his qualms with the LINQ’s recently-neutered EntitySet classes.  While this library appears interesting, it critically doesn’t make EntitySets natively IQueriable.  Hence, I don’t see a way where LINQKit allows effortless, short syntax, efficient SQL queries to be built from existing database-derived classes.

Winners: Tautologically, LINQ totally won this battle.

Losers: LINQ EntitySets, for failing to combine short syntax and efficient SQL generation.

Python vs. IronPython : Round 2

June 19, 2007

Yesterday I let IronPython battle it out with TileCache, which was written for CPython and CGI.  IronPython was stumbling — not seeming like a drop-in replacement for CPython yet — but I didn’t quite call the fight.  I wanted to port TileCache’s CGI library calls to ASP.NET calls and see what I came up with.

After trivially porting TileCache’s CGI code, I discovered a new issue.  IronPython’s lacks support for ”os.access.”.  TileCache requires “os.access”.  I’m not sure where this method normally lives.  Its not defined in the “os” module of the Standard Library, but it works from CPython.  AFAIK, its magic. 

 With a bit more research, and I stumbled upon FePy.  “The FePy project aims to provide enhancements and add-ons for IronPython, an implementation of Python programming language.”  They’ve already addressed my “hashlib” problem, providing a custom .NET module for it and for “socket.”  They’ve also identified and patched 3 issues in IronPython, and 6 issues in the Python Standard Library when used with IronPython. 

FePy’s patches should fix my issue.  They’ve added “os.access” directly into the IronPython DLL.  However, FePy lists their patched IronPython DLL version as 1.1, while ASP.NET Futures seems to use a Version 2.0.  The thought ”with a little more effort, I can make this work” was beginning to feel familiar.  I’ll wait till FePy moves to 2.0.

Winners:  the FePy developers, notably Seo Sanghyeon, for relentlessly discovering and fixing IronPython bugs before MS can

Losers:  IronPython, for stumbling twice on a mere 44k of code (+400k of relevant, standard libraries)

Python vs. IronPython : TileCache

June 18, 2007

There is a maker’s serial number 9906947-XB71. Interesting. Not fish. Snake scale.”  Blade Runner has shown how tell a real snake from a replicant snake:  (Sys.Version).  Beyond the superficial, what happens when we take the animoid IronPython , and attempt to replace CPython in a real-world battlefield?  I chose MetaCarta’s TileCache.

For those who don’t know, IronPython is MS’s implementation of Python on top of .NET.  Its pre-release software in the context of running it from ASP.NET, and therefore against the pre-release license to benchmark.  I won’t be doing that; nor do I need to.  The purpose is to see if IronPython is a drop-in replacement for CPython.  To be fair, I know next to nothing about Python in either implementation.

It took me less than 30 minutes to set up TileCache using CPython and CGI on MS’s IIS.  The directions weren’t 100%, but anybody with a clue can fill in the blanks.  Hint:  IIS has no default CGI handler. 

IronPython right now is much trickier.  I put about two hours into it, and didn’t get TileCache running in that time.  I did finally get everything to compile, but it took some work.  The basic problem is that IronPython won’t import compiled C/C++ modules.  The IronPython developers are aware of this, and will supposedly replace these compiled modules with .NET equivalents as time permits. 

For now, IronPython doesn’t ship with standard library equivalents, and must reference CPython’s modules, many of which in turn reference compiled C/C++ modules.  TileCache references script modules that reference the “hashlib” compiled module, even though I’m pretty sure that TileCache doesn’t use these. 

After a long, uneducated process to track down where “hashlib” was being imported, it was easy to comment out.  After this, TileCache would run, but it didn’t do anything.  I was just getting back zero-byte responses, because TileCache’s CGI-specific code needs porting to the ASP.Net environment.  To a .NET developer, the nice part is that VisualStudio’s debugging tools made it simple to track down this problem. 

Winners:  CPython, because it already works just fine

Losers:  Impatient developers?  People who waste time evaluating beta products?

Best GIS Infrastructure?

June 1, 2007

If you could take any existing GIS products, and not have to worry about personnel / training / cost / platform considerations, what you would be your ideal GIS infrastructure?

 I think was thinking I’d like Oracle Spatial + ArcInfo + GeoNetworks + Mapserver +  MetaCarta’s TileCache + Openlayers.

Its funny – I rarely/never use these components.  Perhaps I chose them because I don’t know their faults.

CSW + GPL = Help!

May 9, 2007

I’ve been hunting decent Catalogue Service clients.  I want to use one in my web applications, and if I’m going to put any work into it, I’d like to be able to reuse it as an ArcMap plugin.  Now, ESRI’s got their own, closed-source Portal Toolbar.  However, my initial assessment says it has no soul; it relies on ESRI’s metadata shortcut nodes.  Not cool, not extensible, and not useful as a library for my web applications.

I did find a couple of FOSS CSW clients… one from Deegree (LGPL‘ed), and one from GeoNetwork Opensource (GPL‘ed).  To make a long story short, the Deegree one isn’t going to work.  So I’ve strapped on my scuba gear, and begun the descent into the world of the GPL.

Now, the GPL FAQ tells me that I can modify GeoNetwork’s code however I want, as long as I don’t share it.  I’m assuming that means I’m allowed to modify it for my web applications.  Dive much deeper than this, and the barotrauma starts to kick in.  I can’t distributed code that links closed-source libraries (ArcGIS’s) with GPL’ed libraries.  Okay… but where do you draw the line on what’s considered linking? 

Its entirely reasonable to use GeoNetwork’s GPL’ed CSW client to call a closed-source CSW server.  Can I interpret this to mean that an XML web service is enough to differentiate between the GPL’s concepts of bundled programs and combined modules?  So, what, I expose my server-side, modified, GPL’ed CSW client to ArcMap via SOAP?  That doesn’t exactly make me feel happy technology-wise, and violates the spirit of the GPL.

To the point, I’m hoping some OS-GIS people out there will help me do a very bad thing.  I’m hoping GeoNetwork’s CSW client can swim to more pleasant LGPL waters {minus some GPL’ed dependencies}.  I’ve previously made ESRI-kool-aid inspired comments that OS-GIS needs more ArcGIS plug-ins to subvert ESRI users.  I’m not sure GPL’ed clients are going to get us there.

Comments?  Any GeoNetwork people out there?

 UPDATE:

Paul Ramsey says fear not!  The GPL’s code-mixing considerations shouldn’t affect me, see the comments for details.

Register all the Web ADF Samples with IIS at Once

March 29, 2007

You know the drill… if you’re creating an ASP.Net web application, you need to configure its folder as an application within IIS. 

ArcGIS Server comes with 52 samples ADF web applications, 26 in C#, 26 in VB.Net.  Stick them into folders within your web root, and you’ll have 52 error messages telling you to configure your 52 folders as IIS applications.

Here’s a rough VBS script that walks your web root, wiring up an IIS application anywhere it sees web.config file:

FixAspxRoots.vbs

Dim IISroot
Dim objFSO
Set objFSO = CreateObject(”Scripting.FileSystemObject”)
Set IISroot = GetObject(”IIS://localhost/W3SVC/1/ROOT”)
objStartFolder = IISroot.path
WScript.Echo “Scanning for web.config files below ” + objStartFolder
ExamineSubfolders objFSO.GetFolder(objStartFolder)
IISroot.SetInfo()

Sub ExamineSubFolders(Folder)
 If (objFSO.FileExists(Folder.path + “\web.config”)) Then AddvirtualDir Folder.Path, Folder.name
 For Each Subfolder in Folder.SubFolders
  ExamineSubFolders Subfolder
 Next
End Sub
Sub AddVirtualDir(folderPath, folderName)

 Dim iispath
 Dim VirtualDir
 
 
 iisPath = replace(mid(folderPath, len(objStartFolder)+2), “\”, “/”)
 
 Set VirtualDir = Nothing
 On Error Resume Next
 Set VirtualDir = GetObject(”IIS://localhost/W3SVC/1/ROOT/” & iispath)
 On Error Goto 0
 If Not VirtualDir Is Nothing Then
  If (VirtualDir.class = “IISWebVirtualDir”) Then
   ’I'm assuming that this should be a webdir, not a vdir
   WScript.Echo “killing vdir ” & iispath
   Set IISParent = GetObject(VirtualDir.Parent)
   IISParent.Delete VirtualDir.class, VirtualDir.name
   IISParent.SetInfo()
   WScript.Echo “Creating IIsWebDirectory ” & iispath
   Set VirtualDir = IISroot.Create(”IIsWebDirectory”, iispath)
  Else
   WScript.Echo “updating  ” & VirtualDir.class & ” ” & iispath
  End If
 Else
  WScript.Echo “Creating IIsWebDirectory ” & iispath
  Set VirtualDir = IISroot.Create(”IIsWebDirectory”, iispath)
 End If
 ’I have no idea what half these settings mean
 VirtualDir.AppIsolated = 0
 VirtualDir.AppFriendlyName = folderName
 VirtualDir.AccessScript = true
 VirtualDir.AccessRead = true
 VirtualDir.SetInfo
 VirtualDir.AppCreate2 1
 VirtualDir.SetInfo
End Sub

Search compiled code with Reflector

March 28, 2007

I regularly use Lutz Roeder’s .NET Reflector for gaining insight into mysterious new codebases.  I was always a little disappointed in its search functionality though — it can only search through Types, Members, and Constants.  Today I discovered a little tool that will make my life 100% easier.  A Code Search Add-In for Reflector:

 http://www.codeplex.com/reflectoraddins

PsService Batch Files for ArcIMS Administration

March 27, 2007

Dave Bouwman had a post on using the “NET” command for service administration.  Hats off to Dave, but PsService fits much better into my workflow.

Check out http://www.microsoft.com/technet/sysinternals/ProcessesAndThreads/PsService.mspx

Now check out the code below, half stolen, half rewritten:

@echo off

echo **************************************
echo Stopping IMS Services
echo **************************************

psservice \\10.147.156.69 STOP “ArcIMS Tasker 9.2.0″

psservice \\10.147.156.69 STOP “ArcIMS Monitor 9.2.0″

psservice \\10.147.156.69 STOP “ArcIMS Application Server 9.2.0″

pause

cls
echo ***************************
echo Restarting IMS Services
echo ***************************
psservice \\10.147.156.69 start “ArcIMS Application Server 9.2.0″

psservice \\10.147.156.69 start “ArcIMS Monitor 9.2.0″

psservice \\10.147.156.69 start “ArcIMS Tasker 9.2.0″

pause