I just read the VerySpatial post linking to Builder.com Ten Commandments of Egoless Programming via TheSteve0.  VerySpatial likes Commandment #3: No matter how much “karate” you know, someone else will always know more. 

I like this one too, but for very different reasons.  This rule, to me, is a signpost showing where ego can be put to use:  disruptive technologies.  In essence, if no matter how good you are, someone will always be better: then change the game. 

My other favorite?: The only constant in the world is change.   Keep your ego.  Lead.

I have been writing a lot of Flex code recently.
Somehow, I’m most pleased with this *tiny* little class to load XML settings from the HTML container.

The joy here is that Flash/Flex can *only* load data asynchronously… this class allows settings to be pulled in sychronously, before anything else, and with very little effort.

package com.wordpress.mapwrecker
{
	import flash.utils.Proxy;
	import flash.utils.flash_proxy;
	import flash.external.ExternalInterface;

	dynamic public class Settings extends Proxy {
		private var data:XML;

		public function Settings() {
			//load settings from parent HTML document
			if(ExternalInterface.available)
			{
				 var getSettingsRequest:String = "function(){ return document.getElementById(\"settings\").innerHTML; }";
				 var response:String = ExternalInterface.call(getSettingsRequest);
				 data = XML(response);
			}
		}

		override flash_proxy function getProperty(name:*):* {
			return data.children().(localName().toUpperCase() == String(name).toUpperCase());
		}
	}

}

I extended ESRI’s ArcWeb Flex API’s RasterTileLayer class which normally supports only geographic tile layouts, and made it work with Mercator.  Picture a lot of trial and error, and a few functioning yet not working tile-layouts that were plain silly. 

ESRI’s ArcWeb Flex API supports making web maps from ArcWeb data-sources and from some custom data-sources such as ArcIMS and ArcGIS Server.  A couple of code-samples infer that writing your own custom imagery provider is easy.  But I wouldn’t call it easy.  The ArcWeb Flex API is closed source, the documentation extends only to public methods/properties, and does not cover all classes in the API.

Luckily, there are ways of digging.  The Flex API ships as an SWC file — a compiled Flash/Flex library.  These files are well known to be ZIP files containing a CATALOG.XML file.  This catalog file lists all of the class names in the API along with dependency classes.  Its a more convenient way of getting to know the hidden API than blindly exploring the “import com.esri.awx…” intellisense code-hints available within Flex Builder.

Once you’ve found an interesting class, cross your fingers and pray that its implementation is obvious.  Again, code-hinting goes a long way.  Its trivial to instantiate a class and see what public methods and properties are available.  Sub-class using the ”extends” keyword and you’ll gain access to the internal and protected property and method names.  Within the Flex API, the class names, code-hinting, etc. are usually descriptive enough to get by.

When the path is less clear, you’ll need to use Flex’s debugging to find your way.  I eventually sub-classed and overrode every class method/getter/setter with a simple call to the super-class.  This allowed me to follow the logical order of method calls, and keep an eye on the Flex debug Variables at every step of the class’s lifecycle.

Did I mention trial-and-error?  I still don’t know how or why some of the code I wrote works.  To me, the math hacks are what makes this cool … we say the earth is 360×360 to make things square, but fudge the latitude to fix tile Y-positioning.

package com.esri.aws.awx.map.layers
{
	import com.esri.aws.awx.map.projection.*;
	import com.esri.aws.awx.map.layers.tiles.*;

	public class MercatorTileLayer extends RasterTileLayer
	{
		private var _mercLayout : ITileLayout;

		public function MercatorTileLayer()
		{
			m_scheme =  new ClientPowerOfTwoTilingSchemeImpl(true, 17, 256, 256, 1, 1, -180, -180, 180, 180, "");
			m_tileLayout = _mercLayout = new MercatorTileLayout(m_scheme);
		}

		override protected function commitProperties():void{
			m_schemes[dataSource] = m_scheme; // setting m_schemes[] seems to prevent m_scheme resets
			super.commitProperties(); // This still resets m_tileLayout, I can't figure a way around it
			m_tileLayout = _mercLayout; // So we'll fix m_tileLayout ourselves
        }

		override public function get supportedProjections():Array
		{
			return [ProjUtils.PROJECTION_STYLE_MERCATOR];
		}

		override protected function constructUrl(tileKey:TileKey, token:String):String{
			return "http://localhost./TileServer/" + dataSource + "/" + getZoomString( tileKey ) + ".ashx";
		}

	}
}

package com.esri.aws.awx.map.layers.tiles
{
	public class MercatorTileLayout extends TileLayoutImpl implements ITileLayout
	{
		internal static var toDegrees:Number = 180.0 / Math.PI;
		internal static var toRadians:Number = Math.PI / 180.0; 

		public function MercatorTileLayout(tilingScheme:ITilingScheme)
		{
			super(tilingScheme);
		}

		override public function determineTilesForView(list:TilesList, centerX:Number, centerY:Number, mapScale:Number, displayWidth:int, displayHeight:int):TilesList
		{
			if (centerY > 85.05113) centerY = 85.05113;
			if (centerY < -85.05113) centerY = -85.05113;

			centerY = centerY * toRadians;
			centerY = Math.log( Math.abs(Math.tan( (Math.PI/4) + (centerY/2) )) );
			centerY = centerY * toDegrees;

			return super.determineTilesForView(list, centerX, centerY, mapScale, displayWidth, displayHeight);
		}

	}
}

One final note is that this won’t get you too far using Google or Microsoft’s Mercator tiles. Flash famously has the ability to bypass cross-domain restrictions using CrossDomain.xml files. Lesser known is Flash’s lameness in using images across domains. You oddly cannot smooth/resize/alter an image loaded from another domain without a CrossDomain.xml file. The Flex API does image smoothing and resizing, so no-go for commercial imagery providers without crossdomain.xml files. Ridiculous!

I’m not 100% sure why, but with IIS7, I am unable to get connect to my local IIS instance using Fiddler’s magic “localhost.” hostname.  Normally this hostname allows localhost traffic to be monitored within Fiddler. Out of the box, IIS7 was returning me the following error on requests to “localhost.”:

Bad Request - Invalid Hostname

Trying to add a secondary binding with “localhost.” explicitly set as the host yeilds another error, this time from the IIS Management Console:

Value does not fall within the expect range.

I found that editing Fiddler’s CustomRules.js provides a quick workaround while still allowing you to use the exact “localhost.” name.  Simply add the following line to the “onBeforeRequest” function:

if (oSession.host.ToUpper() == “LOCALHOST.”) { oSession.host = “localhost”; }

This fix of course only works when Fiddler is running, so please comment if you know of a workaround within IIS7.  (Alternatively, please comment if you have no issue in IIS7 — that my box simply has a case of the Vistas.)

Summarizing last time:  Google’s WKT includes SPHEROID[”…”, 6378137, 0], meaning the earth is a sphere with a radius of 6378137 meters. 6378137 * pi = 20037508, the half-circumference of the earth in meters, and the Mercator max range.  Solve the Mercator equations so that max X = max Y, and you’ll find that happens at +/- 85.051 degrees latitude.

Notice this 85.051 degrees is a “magic number” in Mercator; it dictates a perfect Mercator square for the moon as well as it does the Earth.  Our WKT says Google’s Mercator maps are based on meters.  Yet Wikipedia’s math imagines a world with a radius of 1.  Worse, Charlie rightly points out that Google’s APIs also imagines multiple Mercator worlds with various pixel dimensions!

In short, there is only one square Mercator projection of a sphere.  Datum is the question.  Your local WMS/IMS may need to know the earth is 6378137 meters in radius for conversion from other projections.  One the other hand, it’d be happy with 6378.137 km, 31795.5 furlongs, or 6.7418e-10 lightyears, as long as you can explain the relationship.  {You can tweak your WKT SPHEROID all day, as long as you equally modify your DATUM’s UNIT values.}

More to the point, no matter what your datum, there is only one defacto tile standard: 256×256 tiles, a 256×256 base map, n zoom levels, and a 2×2 zoom between levels.  With tiles, the earth is 128*2^n/pi pixels in radius, end of story.  Any Mercator datum can be turned into another simply by multiplying by the ratio of their diameters.  Making a 6378137 m radius earth fit into 256×256 pixels is one such transformation of datum.

The fun thing, then, is coming up with a better datum.  Google’s solution of pixel datums is certainly elegant.  In many circumstances, however, we are trying to fit tiles to geographic coordinates.  Here, a 360×360 unit Mercator Earth is a nice, clean option. 

Remember that the Mercator projection does not touch longitude except for datum conversion.  Hence, with a 360×360 Mercator Earth, longitude is equal to X already.  Mercator Y can be derived from latitude using the simple { Y = log( tan( lat ) + sec( lat ) ) } equation.

And that’s Mercator.   Fin.

These days, everybody uses mercator tilesets.  Charlie and Morten both put forth excellent explanation of Mercator for us.  These two are both smart, smart guys.  I will attempt to give an unsmart explanation.

If you were to take a globe and count its degrees, you’d see the earth is 360 degrees around, and 180 degrees from pole to pole.  Easy!!  A circle’s circumference is 360 degrees, and also its radius * 2 * pi.  By that logic, 180 degrees = pi * r, and the earth is pi*r tall, and 2*pi*r wide.

Knowing the earth’s radius, then, we can calculate the circumference of the earth (2*pi*r).  The trick is, not everybody is willing to say that the earth is a sphere.  People who know fancy words are more inclined to describe our earth as an oblate spheroid — a somewhat flattened sphere. 

Morten’s WKTs inform us that WGS84 uses a spheroid of SPHEROID["...",6378137, 298.257223563] while Google uses a spheroid of SPHEROID["...", 6378137, 0].  It’s that third parameter thats interesting:  its the flattening factor.  It means WGS84 knows the earth is an oblate spheroid, while Google treats the earth as a perfect sphere.  6378137 meters is the semi-major axes of WGS84’s oblate world.  In Google’s world view, this number is simply the radius of the earth.

Sweet.  We know the radius of the Google’s earth.  We know the half-circumference of the earth, too, pi*r:  20037508.342789 meters.  Check the validMercator range for the Google projection and you’ll see a pattern:  [-20037508.3427892, -20037508.3427892] to [20037508.3427892, 20037508.3427892].  Morten is simply telling us that Google uses the circumference of the earth in meters!!

 But WAIIIIT.   The earth is pi*r tall, and 2*pi*r wide.  Half of those numbers should be half that!!  And they would be, in an unprojected/plate carre world; but Google uses a Mercator projection.  Referring to Wikipedia and our valid range, we can see that Mercator is happy to leave longitude alone.  But also from Wikipedia, “A Mercator map can never fully show the polar areas, since linear scale becomes infinitely high at the poles.”

 WHAT?  Yes, Mercator project has an infinite Y range.  We can’t have infinite bounds!! So how does Google make it look like a square?  Simple:  they chop off the poles. 

For simplicity’s sake, put your mental trig calculator into degree mode, or pretend the world has a radius of 1.  Take Wikipedia’s Mercator projection  { Y = log( tan( lat ) + sec( lat ) ) },  and solve with Y = 180.  You’ll arrive at a value of 85.051 degrees … which should again be familiar from Morten’s blog.  85.051 degrees is a magic latitude that gives Google their square earth.

So there you go:  A rough explanation of Mercator, and an only slightly better explanation of some of the WKT magic number behind hit.  Stay tuned for part two.

Last June I noticed that most of my blog posts were comparisons, to which I attributed a sort of kitsch.

Kitsch, according to Milan Kundera’s book The Unbearable Lightness of Being, defined it as “the absolute denial of shit.” I hoped to poke some fun at the GIS programmers’ feuding over paradigms / languages / libraries by presenting a world-view like Kundera’s kitsch, where “all answers are given in advance and preclude any questions.”  

Hence I embraced a cloying battle meme, and elevated my level of distasteful sarcasm.  Today, I’ve finally grown tired of this faux personality, and have accumulated a few ideas I’d like to post in a more traditional format.  In other words, I’m going back to normal.  Wish me luck.

PS:  WordPress’s spell check doesn’t recognize the word “blog”. “Biog” and “bldg” are suggested alternatives.  Just a note for anyone hoping my sarcasm would disappear overnight.

Time to change the name Dave.

     

using System;
using System.Text;
using ArcDeveloper.TileServer.Interfaces;
using System.Net;
using System.IO;
using System.Xml;
using System.Drawing;
using System.Drawing.Imaging;   

namespace ArcDeveloper.TileServer.ArcIMS
{
    public class ArcImsTileProvider : ITileProvider
    {
        ///
        /// Private member containing the web service url
        ///
        private string _webServiceUrl = "";   

        ///
        /// Constructor that takes a web service Url
        ///
        ///
        public ArcImsTileProvider(string webServiceUrl)
        {
            _webServiceUrl = webServiceUrl;
        }   

        ///
        /// Get the tile image
        ///
        ///
        ///
        ///
        /// Tile data stream
        public System.IO.MemoryStream GetTile(ITileExtent extent, int tileHeight, int tileWidth)
        {
            MemoryStream returnStream = null;   

            try
            {
                string axl = getAXL(extent, tileHeight, tileWidth);
                Stream responseStream = PostData(_webServiceUrl, axl);
                XmlDocument xd = new XmlDocument();
                xd.Load(responseStream);
                string imgURL = xd.GetElementsByTagName("OUTPUT")[0].Attributes["url"].Value;
                responseStream.Close();   

                Bitmap bm = new Bitmap(new WebClient().OpenRead(imgURL));
                returnStream = new MemoryStream();
                bm.Save(returnStream, ImageFormat.Png);
            }
            catch(Exception e)
            {
                Bitmap bm = GetMessageTile(e.Message);
                returnStream = new MemoryStream();
                bm.Save(returnStream, ImageFormat.Png);
            }   

            return returnStream;
        }   

        ///
        /// Gets the ArcIMS GetImage AXL request.
        ///
        ///
        ///
        ///
        /// GetImage ArcIMS AXL request
        private string getAXL(ITileExtent extent, int tileHeight, int tileWidth)
        {
            StringBuilder sb = new StringBuilder();
            sb.Append("<?xml version=\"1.0\" encoding=\"UTF-8\" ?>");
            sb.Append("<ARCXML version=\"1.1\">");
            sb.Append("<REQUEST>");
            sb.Append("<GET_IMAGE>");
            sb.Append("<PROPERTIES>");
            sb.Append("<ENVELOPE minx=\"" + extent.lon2 + "\" miny=\"" + extent.lat2 + "\" maxx=\"" + extent.lon + "\" maxy=\"" + extent.lat + "\" />");
            sb.Append("<IMAGESIZE height=\"" + tileHeight + "\" width=\"" + tileWidth + "\" />");
            sb.Append("</PROPERTIES>");
            sb.Append("</GET_IMAGE>");
            sb.Append("</REQUEST>");
            sb.Append("</ARCXML>");
            return sb.ToString();
        }   

        ///
        /// Gets a tile with an error message burned into it.
        ///
        /// The message.
        /// An error message PNG
        private Bitmap GetMessageTile(string message)
        {
            Bitmap tile = new Bitmap(Properties.Resources.blankTile);
            Graphics graphics = Graphics.FromImage(tile);
            Font drawFont = new Font("Arial", 12);
            SolidBrush drawBrush = new SolidBrush(System.Drawing.Color.Black);
            SizeF len = graphics.MeasureString(message, drawFont);
            graphics.DrawString(message, drawFont, drawBrush, new PointF(10, 10));
            graphics.Dispose();
            return tile;
        }   

        ///
        /// Does an HTTP Post, returning a stream.
        ///
        /// The URL to post to
        /// The data to post
        /// The Http response stream
        private Stream PostData(string sURL, string postData)
        {
            HttpWebRequest webReq = null;
            HttpWebResponse response = null;
            Stream requestStream = null;
            StreamWriter writer = null;
            Stream responseStream = null;
            try
            {
                webReq = (HttpWebRequest)WebRequest.Create(sURL);
                webReq.Method = "POST";
                webReq.ContentType = "application/x-www-form-urlencoded";
                webReq.Timeout = 0xea60;
                webReq.AllowAutoRedirect = true;
                requestStream = webReq.GetRequestStream();
                writer = new StreamWriter(requestStream);
                writer.Write(postData);
                writer.Flush();
                writer.Close();
                requestStream.Close();
                response = (HttpWebResponse)webReq.GetResponse();
                responseStream = ((HttpWebResponse)webReq.GetResponse()).GetResponseStream();
            }
            finally
            {
                if (writer != null) writer.Close();
            }
            return responseStream;
        }   

    }
}   

Winners: Python-fearing .NET folks still using ArcIMS.  All six of you!

Losers: “ArcGIS Server Virtual Earth Tile Server” as a name.  Die.

It looks like the boys at Microsoft Research implemented my idea: tiled vectors.  Their demo makes one thing obvious, however.  Douglas-Peucker just doesn’t cut it for topologies.  This becomes a non-issue with a pixel-level of detail, but just looks bad in their demo.  Their implementation doesn’t seem to preserve existing points between zooms; it’s probably just as efficient this way.  Via Vish.  (Check out Idevio, while you’re at it.) 

Winners:  Me!  Virtual Earth stole my idea!!  (LOL… “my” idea)

Losers: Me!  Virtual Earth stole my idea!!

I’m just getting into LINQ, and experimenting with what goes on behind the scenes.  LINQ uses the idea of a “DataContext” as an interface to the database, much like the older “DataSet” acts to encapsulate multiple table structures.   One notably lacking feature of LINQ’s database-derived classes is the ability to traverse many-to-many relationships without touching the intermediate cross-reference table. 

Assuming an existing “vsNetwork” entity, you can query a many-to-many relationship directly against the ”DataContext” ala:

from f in myDataContext.ScienceFocus
join vsf in vsDc.VitalSignFocus on f.FocusId equals vsf.FocusIdf
where vsf.VitalSignIdf == vsNetwork.VitalSignId
select f;

Again assuming an existing “vsNetwork” entity, you can also query by walking the database-derived class’s table-relations, ala:

vsNetwork.FXrefVitalSignFocusList.Select(vsf => vsf.FTluScienceFocus).ToList();

The magic of LINQ exists within all things that implement the IQueriable interface.  This allows the keywords “select”, “where”, “join”, etc. to be applied to a collection of objects.  Note, however, that there are differences in how “IQueriable” is implemented.  The examples above, indeed generate markedly different SQL.  Respectively,

SELECT [t0].* FROM [tlu_Science_Focus] AS [t0] INNER JOIN [xref_Vital_Sign_Focus] AS [t1] ON [t0].[Focus_ID] = [t1].[Focus_IDF] WHERE [t1].[Vital_Sign_IDF] = vsNetwork.Vital_Sign_ID

versus

SELECT [Focus_IDF] FROM [xref_Vital_Sign_Focus]  WHERE [Vital_Sign_IDF] = vsNetwork.Vital_Sign_ID

*For Each Focus_IDF…*

SELECT * FROM [tlu_Science_Focus] WHERE [Focus_ID] = Focus_IDF

We see that the latter runs multiple queries instead of using a SQL join.  The source of this inefficiency is described by The Wayward Weblog:

“[EntitySet's] ToQueryable() wraps your IEnumerable<T> in IQueryable<T> clothing, uses the Queryable infrastructure to let you build up your own expression tree queries, and then when you enumerate it, the expression is rebound to refer to your IEnumerable<T> directly”

In this case, “vsNetwork.FXrefVitalSignFocusList” is of type EntitySet, which does not implement IQueriable directly, but rather through IEnumerable.  So not only does this shorter syntax lose efficiency by issuing multiple and superfluous queries, it also loses efficiency by iterating through each xref table in memory.  In the case of large sets, using LINQ against IEnumerables can be an issue.  It has been enough of an issue that people are adding indexing for in-memory LINQ objects, rather than simply iterating collections.

In earlier Visual Studio 2008 CTP’s, LINQ EntitySets implemented the iQueriable interface (directly?).  I’m not sure if at that point, this was implemented to elegantly generate SQL, I simply know that O’Reilly’s C# in a Nutshell author Joseph Albahari made some fuss about the change. 

It appears that Mr. Albahari is behind a library called LINQKit designed to mitigate his qualms with the LINQ’s recently-neutered EntitySet classes.  While this library appears interesting, it critically doesn’t make EntitySets natively IQueriable.  Hence, I don’t see a way where LINQKit allows effortless, short syntax, efficient SQL queries to be built from existing database-derived classes.

Winners: Tautologically, LINQ totally won this battle.

Losers: LINQ EntitySets, for failing to combine short syntax and efficient SQL generation.