I know I shouldn’t be looking at these things before coffee.  Still, when a SQL Import task complained about my Geometry column type, I tried to fix this.

I editted my “C:\Program Files\Microsoft SQL Server\100\DTS\MappingFiles\MSSQLToSSIS10.XML” file, duplicating the entry for varbinary(max), but changing the data type name to “geometry.”

Bingo.  Everything now seems to copy fine.

I’m attending the ESRI Developers Conference for the first time ever, and was hunting for hotel suggestions.  There are comments on James Fee’s blog from last year that recommend Zoso over the Wyndham.  TripAdvisor skews towards the tiny romatic places… any recommandations on a cheaper place for the barely-under 30 crowd?

Earn ~$70k, 5 weeks vacation, 3 weeks sick, with awesome people, for a good cause (the National Parks Service).  Take my old job.  Deadline is the 22nd.

http://www.cemml.colostate.edu/Jobs/351.pdf

Please forward / link this post!!

The blogs are abuzz today with Google’s new cross-browser, cross-platform plugin that allows native x86 code to be run from a browser with minimal code tweaking.

Called Native Client, its complete with sandboxing, Javascript interop, a Quake demo and …  Yet Another Globe Application!!

This really escalates the Javascript / Flash / Silverlight wars…. x86?!

Hello OpenGL, goodbye vector drawing performance woes.

If you’re inserting SQL’08 Native Spatial Datatypes into a brand new table, favor SqlBulkCopy over Insert. Its ridiculously faster. Below is some reduced code to show the basic idea:

// use minimal transaction logging during upload
new SqlCommand("alter database [" + dbName + "] SET RECOVERY BULK_LOGGED", connection).ExecuteNonQuery();

// create and a datatable of points
DataTable dt = new DataTable();
dt.Columns.Add("Shape", typeof(SqlGeometry));

// populate your SqlGeometry
dt = magicFunction_fillMyDataTableWithGeometry();

// copy data, use batches and tablelock to ensure small minimal transaction log sizes
using (SqlBulkCopy sbc = new SqlBulkCopy(connectionString(dbName), SqlBulkCopyOptions.TableLock))
{
    sbc.DestinationTableName = tableName;
    sbc.BatchSize = 10000;
    sbs.ColumnMappings.Add("Shape", "Shape");
    sbc.WriteToServer(dt);
    sbc.Close();
}

// use normal transaction logging after upload
new SqlCommand("alter database [" + dbName + "] SET RECOVERY FULL", connection).ExecuteNonQuery();

My last post on wiring up JSON to ASP.NET MVC showed how to use an ActionFilter to deserialize JSON into .NET objects.  However, using a custom IModelBinder is a cleaner and in many ways the more correct solution:

using System;
using System.Web.Mvc;
using System.Text;
using System.Xml;
using System.Runtime.Serialization.Json;

namespace NRPC.Citation.Web.Controllers
{
    public class JsonBinderAttribute : CustomModelBinderAttribute
	{
		public override IModelBinder GetBinder()
		{
			return new JsonModelBinder();
		}

        public class JsonModelBinder : IModelBinder
        {
            public ModelBinderResult BindModel(ModelBindingContext bindingContext)
            {
                Encoding requestEncoding = bindingContext.HttpContext.Request.ContentEncoding;
                byte[] stringBytes = requestEncoding.GetBytes(bindingContext.HttpContext.Request.Form[bindingContext.ModelName]);
                XmlDictionaryReader rdr = JsonReaderWriterFactory.CreateJsonReader(stringBytes, XmlDictionaryReaderQuotas.Max);
                object o = new DataContractJsonSerializer(bindingContext.ModelType).ReadObject(rdr);
                return new ModelBinderResult(o);
            }
        }
	}
}

public ActionResult postDTO([JsonBinder]GridStateDTO colState){ ... }

    Ext.Ajax.request({
        url: '/myURL.mvc/postDTO',
        success: success,
        failure: failure,
        params: { 'colState': strDTO }
    });

Note how in this case I’m passing the JSON as a form value. This is notable in part because the code in my last post assumed the entire post body was JSON; while this code can co-exist with other form data.

In my previous post, I talked about using r-trees in JavaScript for fast client-side spatial indexing.  The idea is a mixed bag.  Its speedy enough to query tens-of-thousands of records in real time; the problem is that you can’t feed it tens-of-thousands of bounding-boxes quick enough.

The author of the original Java library, Robert Olofsson, was kind enough to host my C# port.  Either way, you can grab my port or visit the project page for more info.  Robert was also kind enough to let me leave the Json serialization code in.   :c)

In .NET you’d generate the Json r-tree something like this:

PRTree<NamedPoint> tree = new PRTree<NamedPoint>(new NamedPointConverter(), 4);
List<NamedPoint> points = getRandomPointData();
tree.load(points);

StringBuilder sb = new StringBuilder();
sb.Append("var myJsonRtree=");
tree.getRoot().toJSON(sb, new NamedPointConverter(), "{0:#.####}");
sb.Append(";");

And in JS you’d query the Json r-tree something like this:

var walkTree = function(node, searchArea, results) {
 if (node.i != undefined) {
  if (!(node.b[0] < searchArea[0] || node.b[0] > searchArea[2] || node.b[1] < searchArea[1] || node.b[1] > searchArea[3]))
   results.push(node.i);
 }else {
  if (!(node.b[2] < searchArea.minx || node.b[0] > searchArea.maxx || node.b[3] < searchArea.miny || node.b[1] > searchArea.maxy)) {
   var s = node.c.length - 1;
   for (var i = 0; i <= s; i++)
    walkTree(node.c[i], searchArea, results);
  }
 }
}

results = [];
var searchArea = [-180, -180, 180, 180];
walkTree(myJsonRtree, searchArea, results);

Special thanks also to Nate Irwin, for not treating me like I was crazy when I suggested client-side spatial indexing as a good idea.

I extended ESRI’s ArcWeb Flex API’s RasterTileLayer class which normally supports only geographic tile layouts, and made it work with Mercator.  Picture a lot of trial and error, and a few functioning yet not working tile-layouts that were plain silly. 

ESRI’s ArcWeb Flex API supports making web maps from ArcWeb data-sources and from some custom data-sources such as ArcIMS and ArcGIS Server.  A couple of code-samples infer that writing your own custom imagery provider is easy.  But I wouldn’t call it easy.  The ArcWeb Flex API is closed source, the documentation extends only to public methods/properties, and does not cover all classes in the API.

Luckily, there are ways of digging.  The Flex API ships as an SWC file — a compiled Flash/Flex library.  These files are well known to be ZIP files containing a CATALOG.XML file.  This catalog file lists all of the class names in the API along with dependency classes.  Its a more convenient way of getting to know the hidden API than blindly exploring the “import com.esri.awx…” intellisense code-hints available within Flex Builder.

Once you’ve found an interesting class, cross your fingers and pray that its implementation is obvious.  Again, code-hinting goes a long way.  Its trivial to instantiate a class and see what public methods and properties are available.  Sub-class using the ”extends” keyword and you’ll gain access to the internal and protected property and method names.  Within the Flex API, the class names, code-hinting, etc. are usually descriptive enough to get by.

When the path is less clear, you’ll need to use Flex’s debugging to find your way.  I eventually sub-classed and overrode every class method/getter/setter with a simple call to the super-class.  This allowed me to follow the logical order of method calls, and keep an eye on the Flex debug Variables at every step of the class’s lifecycle.

Did I mention trial-and-error?  I still don’t know how or why some of the code I wrote works.  To me, the math hacks are what makes this cool … we say the earth is 360×360 to make things square, but fudge the latitude to fix tile Y-positioning.

package com.esri.aws.awx.map.layers
{
	import com.esri.aws.awx.map.projection.*;
	import com.esri.aws.awx.map.layers.tiles.*;

	public class MercatorTileLayer extends RasterTileLayer
	{
		private var _mercLayout : ITileLayout;

		public function MercatorTileLayer()
		{
			m_scheme =  new ClientPowerOfTwoTilingSchemeImpl(true, 17, 256, 256, 1, 1, -180, -180, 180, 180, "");
			m_tileLayout = _mercLayout = new MercatorTileLayout(m_scheme);
		}

		override protected function commitProperties():void{
			m_schemes[dataSource] = m_scheme; // setting m_schemes[] seems to prevent m_scheme resets
			super.commitProperties(); // This still resets m_tileLayout, I can't figure a way around it
			m_tileLayout = _mercLayout; // So we'll fix m_tileLayout ourselves
        }

		override public function get supportedProjections():Array
		{
			return [ProjUtils.PROJECTION_STYLE_MERCATOR];
		}

		override protected function constructUrl(tileKey:TileKey, token:String):String{
			return "http://localhost./TileServer/" + dataSource + "/" + getZoomString( tileKey ) + ".ashx";
		}

	}
}

package com.esri.aws.awx.map.layers.tiles
{
	public class MercatorTileLayout extends TileLayoutImpl implements ITileLayout
	{
		internal static var toDegrees:Number = 180.0 / Math.PI;
		internal static var toRadians:Number = Math.PI / 180.0; 

		public function MercatorTileLayout(tilingScheme:ITilingScheme)
		{
			super(tilingScheme);
		}

		override public function determineTilesForView(list:TilesList, centerX:Number, centerY:Number, mapScale:Number, displayWidth:int, displayHeight:int):TilesList
		{
			if (centerY > 85.05113) centerY = 85.05113;
			if (centerY < -85.05113) centerY = -85.05113;

			centerY = centerY * toRadians;
			centerY = Math.log( Math.abs(Math.tan( (Math.PI/4) + (centerY/2) )) );
			centerY = centerY * toDegrees;

			return super.determineTilesForView(list, centerX, centerY, mapScale, displayWidth, displayHeight);
		}

	}
}

One final note is that this won’t get you too far using Google or Microsoft’s Mercator tiles. Flash famously has the ability to bypass cross-domain restrictions using CrossDomain.xml files. Lesser known is Flash’s lameness in using images across domains. You oddly cannot smooth/resize/alter an image loaded from another domain without a CrossDomain.xml file. The Flex API does image smoothing and resizing, so no-go for commercial imagery providers without crossdomain.xml files. Ridiculous!

Having the R stats package available to your web/GIS applications via DCOM sounds pretty sweet.  It does statistics, graphs, and has quite a few GIS related add-ons.

A co-worker was working with a solution which calls R by invoking RTerm.  I stumbled across R(D)COM and thought a DCOM implementation might provide a more reusable solution.  I tried to get him to evaluate/benchmark R(D)COM, but eventually took up the torch myself.

Unfortunately, it seems to be crap. 

 It takes 1 full second to get an R instance.  The least-chatty cross-process IO is barely faster than disk IO.  Cross-process IO hits a scaling wall well before the OS runs out of RAM.  Sending large datasets via the DCOM proxy eats more CPU time than the actual processing within R.  The crowning fault is that R(D)COM fails to destroy one of it’s processes about 5% of the time.  Props to Greg Hill for having way better intuition than me.

Today I start that short, violent ascent towards fame.

James Fee, here I come.