Monthly Archives: March 2008

Frustrations of a non-Agile Developer

As I look across the landscape of GIS development, I see the tide turning to Agile Methodologies. Specifically, I see SCRUM taking hold and changing the way clients and implementors look at doing their work. Blogs like Chris’s make me fell all warm and fuzzy, as I dream of the day where massive, BigDesignUpFront (BDUF) fixed price RFPs are replaced with agile, iterative, pay-per-deliverable/sprint type requests. I am in constant conflict with the people I work with about this very subject. There are 2.4 billion reasons why I have to write this requirements document or that approach document. I push back with YAGNI type responses, begging for us to encourage the client to create user stories and dump the IEEE “shall” statements.

This scenario has played itself out since the beginning of the year several times. In the end, while my fellow developers agree with my plight, the program management and client just find me combative and difficult. I imagine they tense up everytime they have to ask me to write a new approach document (oh, you have no idea how I loathe approach documents) and recoil in expectation of my Agile Evangelical Sermon. I have taken the approach of writing user stories for the client as an example, then listing what I think the remaining stories are so they can finish them. After all, having he developer write user stories is akin to having the fox watch the henhouse. So, I get “This is great. We can use these.” and just when I think they are getting it someone asks “Have we mapped these back to the requirements?” and my hope is crumpled up and tossed toward the already full wastebasket of former agile dreams where it bounces to a rest on the floor.

Well, now that the self-pity party is over, I would love anyone to give me examples of getting stubborn, set-in-their-waterfall-ways Project Managers to listen to the merits of Agile. And I mean REALLY listen. I have sent articles of how to move to agile, I have authored e-mail novels on it, and I have discussed it over beers. Nothing takes hold. I’ve already been through the patronizing head-nodding, reassuring “uh-huh” discussions that I bet are eerily similar to what my wife gets when she tries to talk to me while I watch TV. They seem to agree, but their actions betray their nodding. Just today I was asked to validate a “high-level” estimate for a MASSIVE (roughly $3 million) project as they submit their RFP. It went like this:

Them: “Glenn our hours are X for the first iteration and Y for the second.”

Me: “2 iterations? Each about Y months long? Also, trying to estimate a project of this size and scope is ludicrous. (I hear the sigh on the other end of the IM) We should ask them if they are willing to spend a little money to go through an Iteration Zero, y’know? Create the Project Backlog, estimate number of sprints, then we can get a really good idea of cost and risk……” (I go on for a bit)

Them: “The RFP asked for a fixed price bid against the requirements that are in the RFP. We can’t just ignore that.”

Me:

I don’t know what to say. I see the problem and agree we can’t just ignore the RFP rules and submit whatever the hell we want. But estimating everything up front is just bad, bad mojo. It never works, and whoever wins the work will either 1) Stick with the BDUF approach and fail, or 2) Get the work, and then totally change the approach to something that has a snowball’s chance in hell of succeeding. Lather. Rinse. Repeat.

Anyway, if you find yourself reading this and have some suggestions beyond “Stop being some a whiny child,” I would love to hear them. Truth be told, most of what I know about Agile is “book knowledge”, as I have had precious few opportunities to apply the principles in my career.

Advertisements

ArcDeveloper REST: Windsor Brings the Party That Rocks the Body

NOTE: Dave has blogged about the REST API, including a demo of it in action! Check it out!

In part III of my series on the ArcDeveloper REST API, I want to focus on how we use Windsor to configure the service. The best way to to that is to take a step back and look at why Windsor exists. Windsor is an Inversion of Control (IoC) container that provides a robust Dependency Injection (DI) framework. Both of those phrases have been blogged about by just about every human in existence, so if the Fowler link doesn’t help, then hit Google.

Background

In a nutshell, when you have classes that depend on other classes (and, really, what project doesn’t have that? Answer: Maintenance nightmare projects) a good pattern to follow is to supply the dependencies as opposed to instantiating them from within the class. The consummate example is:

public class ClassA{

  private ClassB _classB;

  public ClassA()

  {

    _classB=new ClassB();  //This is pure evil

  }

}

The above code violates so many design patterns and principles that I had to have my 7-year old actually type it (he is a TOTAL hacker) b/c I couldn’t bring myself to do it. A better way to do it is:

public class ClassA{

  private ClassB _classB;

  public ClassA(ClassB injectedClassB)

  {

    _classB=injectedClassB;  //This is sunshine and puppy dogs

  }

}

That code is an example of constructor injection, b/c the dependency must be provided to the constructor or the class cannot be instantiated. There is also setter injection, where public properties are exposed and the dependencies are suppled there. Both approaches have their pros and cons, and I am of the opinion that, if you are simply using DI in either form, you are WAY ahead of the game.

Before I leave the background, I would be remiss if I didn’t point you to the Bitter Coder’s Wiki, where the best tutorials on Windsor live and party.

Great, So How’d Does the ArcDeveloper Rest Stuff Use Windsor?

Well, the Windsor container can use an external configuration section, say, you the web.config file.

<configSections>

<section name=castle type=Castle.Windsor.Configuration.AppDomain.CastleSectionHandler, Castle.Windsor/>

</configSections>

Once that is defined, you have three major configuration areas that you can play with: facilities, properties, and components (NOTE: You’ll find a separate .config file for each of these sections in the web project of our REST stuff.) Facilities are, basically, an extension to the Windsor container. There are many existing facilities, but the easiest to grok is probably the LoggingFacility. You register facilities on the container like so (from our facilities.config file):

<facilities>

<facility

id=logging

type=Castle.Facilities.Logging.LoggingFacility, Castle.Facilities.Logging

loggingApi=log4net

customLoggerFactory=Castle.Services.Logging.Log4netIntegration.Log4netLogger

/>

</facilities>

Here I have registered the Log4Net implemenation of the logging facility, which I could change to a different logging implementation by merely changing the “loggingApi” value (NOTE: I omit the logging.config that is required to configure Log4Net as it’s an, erm, implementation detail.) Once defined, any components registered on the container that have an ILogger public property will automagically get an instance of my Log4Net logger. That is just cool, man.
The rest of our time together will be spent working with components (we aren’t currently using properties right now, but they rock and the tutorials cover them well.) The ArcDeveloper REST API has services that provide the query functionality (like an ArcGIS Server provider) and formatters that take the results from the providers and transform them into the requested format. So, this means that we can register any provider and any formatter we might want to use with Windsor, and then all we have to do is tell the RESTServiceManager (our web service implementation class) about them. Here’s a picture (not quite 1000 words worth, but maybe a quick fin’s worth)

High Level Architecture of ArcDeveloper REST API

So, let’s quickly register the ArcGIS Service provider, which is implemented in the AGSMapService class.

<component id=ags.service lifestyle=pooled initialPoolSize=2 maxPoolSize=2

service=ArcDeveloper.REST.Core.Interfaces.IRESTService, ArcDeveloper.REST.Core

type=ArcDeveloper.REST.ArcGIS.AGSMapService, ArcDeveloper.REST.ArcGIS>

<parameters>

<name>TestService</name>

<description>Base map</description>

<connectionString>http://65.101.234.201/arcgis/services/gains/gains/MapServer</connectionString>

</parameters>

</component>

There you go. The AGSMapService class implements the IRESTService interface, and we provide the pertinent parameters. The “name” parameter (value=”TestService”) will be what is provided in the URI in order to specify that we want to use this service. The “connectionString” parameter is the URL of our ArcGIS Server Map Server object. Look closely at the attributes of the component, and you will see that we are pooling 2 instances of the service, which allows us to connect on first request and keep the object around for other requests. Since connecting to the server is a very expensive operation, we only have to live with it once. Windsor does this all for you! What do YOU do for Windsor? HMMM? That’s what I thought.

Anyway, lets show the formatter:

<component id=geojson.formatter lifestyle=transient

service=ArcDeveloper.REST.Core.Interfaces.IFormatter, ArcDeveloper.REST.Core

type=ArcDeveloper.REST.Core.Services.GeoJSONFormatter, ArcDeveloper.REST.Core>

<parameters>

<name>geoJSON</name>

</parameters>

</component>

So, our GeoJSONFormatter implements the IFormatter interface. We name this one “geoJSON” which, whenever we write a second formatter, will be how the URI will refer to it when requesting the GeoJSON format. Oh, and the lifestyle of this bad boy is “transient”, meaning it’s created on request and disposed after request. This is not an expensive item, so that’s how we roll.

Finally, let’s take a look at the RESTServiceManager:

<component id=rest.service lifestyle=singleton

service=ArcDeveloper.REST.Core.Interfaces.IRESTServiceManager, ArcDeveloper.REST.Core

type=ArcDeveloper.REST.Core.RESTServiceManager, ArcDeveloper.REST.Core>

<parameters>

<services>

<list>

<item>${ags.service}</item>

<!– <item>${other.service}</item>–>

</list>

</services>

<formatters>

<list>

<item>

${geojson.formatter}

</item>

</list>

</formatters>

</parameters>

</component>

You can see that our RESTServiceManager implements our IRESTServiceManager interface and we register our AGSMapService and GeoJSONFormatter with the service manager. The “services” and “formatters” properties of the service manager are lists, so we can add more services and more formatters simply by registering them on the container and adding them as an <item> to the list. Let me say it another way, just to drive the point home: If you wanted to register another ArcGIS Service map service with the RESTServiceManager, you would register with Windsor (so, copy the “ags.service”, give it a new id and change the <connectionString>) and then add an <item> to the <services> parameter with the component id (oh, you may have to restart your web app, as changes to the external config files do not kick off an app unload. If you don’t like it, pull all the config sections into web.config). After that, you can issue REST queries against the map service. It’s just that easy.

One final caveat. The IRESTServiceManager is not only our service manager, but it is also our WCF service contract. When you host a WCF service in IIS, it is created by the WCF Service Host factory, which means there is no way to register it with the Windsor container. That is bad, because if we can’t register the service manager, then we can register the services, and pretty much we have a web service that does bugger all. In order to get around this, the (man and legend) Ayende wrote a component that will allow you to register your WCF services with Windsor. The documentation for it is here, and is so eloquently written that I am stunned that I, er, I mean, the author didn’t get at least a Pulitzer nomination. The long and short of it is;

– Change the .svc file to use a custom ServiceFactory (from our rest.svc file)

<%@ ServiceHost Service="rest.service"
 Factory="Castle.Facilities.WcfIntegration.WindsorServiceHostFactory, Castle.Facilities.WcfIntegration" %> (GRRR....CopyAsHTML doesn't help me in .svc files)

– Instantiate the container at application start up, using the Global.Application_Start method (from our Global.asax.cs file)

protected void Application_Start(object sender, EventArgs e)

{

container = new WindsorContainer(new XmlInterpreter());

WindsorServiceHostFactory.RegisterContainer(container.Kernel);

_log = container[typeof(ILogger)] as ILogger;

_log.Info(“Services app started successfully.”);

}

That’s it. Now the WCF service will use the “rest.service” component we registered in the config file above.

So, this post got a bit wordy on me. I am young in my blogging ways, so I have trouble focusing. Also, I am easily distracted by shiny objects. In future posts, I plan to show how we leverage the Json.NET stuff for the GeoJSON formatter and maybe write a blog about something besides REST.


ArcDeveloper REST: U R I and I R U

As I promised in my (first and) previous post, I am gonna quickly go over the very simple architecture of the ArcDeveloper REST API. This post covers how we expose our REST endpoint(s).

At a high level, we wanted to expose a single REST endpoint that allowed the caller to specify the “service”, “layer”, and query parameters in order to get geographic features returned in a format, which is also specified by the caller. So, attempting to be as RESTy as possible, the URI looks like:

 http://<yourServer>/rest.svc/service/layer/3?format=geojson

It’s clear and easy and other URI endpoints are easy to deduce. Yup, we are RESTing we the best of them. Our main endpoint is the rest.svc, which is WCF’s way of exposing a service endpoint. The first question, then, we had to answer was how we would handle the URI routing. If the map services are registered dynamically (using Castle Windsor, which is arguably the greatest piece of code ever written. Ever. Not that I am prone to hyperbole. Oh, I’ll cover the configuration in the next post) then we have to dissect the various pieces of the URI to:

  1. Get a hold of the service requested.
  2. Query the right layer
  3. Use the right query parameters.
  4. Grab the right formatter.

Luckily for us, .NET 3.5 had recently been released, incorporating some cool changes in WCF, the coolest one being the URITemplate (good background here). The URITemplate allows us to define URI templates (duh) and map them to service methods (or OperationContracts, in WCF-speak) complete with the mapping of parameters of the method. Let’s see how we are using this now:

[OperationContract]

[WebGet(UriTemplate=“{serviceName}/{featureType}/{featureId}?g={geom}”,ResponseFormat=WebMessageFormat.Json)]

Stream GetSingleFeatureJSON(string serviceName, string featureType, string featureId, bool geom);

You can see how the dealies in the “{}” get mapped to the method parameters. Just to be complete, {serviceName} maps to the serviceName parameter, etc. The “geom” parameter specifies whether or not you want all the coordinates returned, because sometimes that can be a LOT of data. We also specify the response format here and the HTTP verb (WebGet== HTTP Get. I’ll give you 3 guesses how you specify HTTP POST. Wrong. It’s WebInvoke.)

So, I know what you are saying. “WTF? You have hard-coded the format INTO THE BLOODY METHOD NAME! I am not reading any more of this tripe.” Well, that’s your prerogative (Tangent: I had NO idea that was how to spell that word. No wonder Bobby Brown is on drugs) but know that I am adhering to the Last Responsible Minute (LRM) doctrine. We only need (geo)JSON right now, meaning, no one has written any other formatters. Once we have another format, we’ll put in a “format” parameter, remove the ResponseFormat and life will be all good. Weak excuse, you say? Maybe. Let’s, uh, move on.

The URIs the project currently supports are:

  http://yourserver/rest.svc/services

Will return information about all of the services that are available to be queried.

  http://yourserver/rest.svc/serviceName/layerName/OBJECTID?g=true

Will return the feature corresponding to the OBJECTID value from the layer specified by “layerName” in the service configured as “serviceName”.

  http://yourserver/rest.svc/serviceName/layerName?query=whereClause&bbox=xmin,ymin,xmax,ymax&g=true

Will query layer named “layerName” in service configured as “serviceName” using the value in “whereClause” (like height>100 or streetname=’Main’ — don’t forget to escape your querystrings….) inside the bounding box specified by xmin, ymin and xmax, ymax.

As always, g=true returns the full coordinates of each geometry. Also, the serviceName corresponds to the name given in the Windsor configuration, NOT the name of the ArcGIS Server service.
In the next post (hopefully) I’ll cover the service and its configuration, showing you how we leverage the total kick-assness of Windsor to make life easy.


ArcDeveloper ArcGIS Server REST API is Breathing!

So, the angst around the ArcGIS Server Web ADF is well known and I shan’t rehash it here (OK, just a little rehashing: the ADF is a bloated sack of vomit) but, rather, I’ll point you to the beginnings of an open source ArcGIS Server REST API (svn) at ArcDeveloper.NET. The project is very young (and always looking for contributors) and, in it’s current state, has the following capabilities:

  • You can query single features by id (must be OBJECTID, ugh)
  • You can query features with a where clause
  • You can query features with a bbox (AND a where clause, if you want)

In this post, I will walk through what it takes to set it up and point it at one of your ArcGIS Server services.

What You’ll Need

Visual Studio 2008, implying .NET 3.5. You will have to build the solution, as we haven’t made an official release yet.

Optionally, I would have Fiddler, Firebug, and JSONViewer.

Step 1: Get the source

Using TortoiseSVN (or the svn client of your choosing), perform a checkout of the trunk (http://svn2.assembla.com/svn/arcdeveloper/ArcDeveloper.REST/trunk/)

Step 2: Open the solution

In the “Product” directory, you’ll find the VS2008 solution file. Like, open it or something. It consists of 5 projects, 2 of which are test projects. The other projects are the core interfaces and services, with the last one being a web project to show how to publish the REST service with WCF and a demo web page.

Step 3: Build the solution

Uh, in VS2008, select “Build Solution…”

Step 4: Use Dave’s Stuff

So Dave Bouwman, who I think is just the cat’s pajamas, has the demo site using an ArcGIS Server map service that he has been generous enough to provide. If you look in the ArcDeveloper.REST.Web project, you’ll find a config directory. Opening up the components.config file will show where the endpoint of the map services is specified. Looks alot like:

<component id=ags.service lifestyle=pooled initialPoolSize=2 maxPoolSize=2

service=ArcDeveloper.REST.Core.Interfaces.IRESTService, ArcDeveloper.REST.Core

type=ArcDeveloper.REST.ArcGIS.AGSMapService, ArcDeveloper.REST.ArcGIS>

<parameters>

<name>TestService</name>

<description>Base map</description>

<connectionString>http://65.101.234.201/arcgis/services/gains/gains/MapServer</connectionString>

</parameters>

</component>

Do you see the cool <connectionString> parameter? That’s Daves AGS service (I told you he was cool.) Run the site (it uses the VS Dev server now) and click “Load Polygon”. You’ll see a small polygon drawn over Yemen (erm, at least I think that is Yemen). Change the ID and draw some more.

ArcDeveloper REST API Demo Page

Here’s what is happening:

A HTTP GET is issued to the local web service with a URI that looks alot like:

 http://localhost:xxxx/rest.svc/TestService/Flyways/10?g=true

Let’s break down the URI, shall we? Starting with rest.svc, that is the WCF endpoint for the REST service. “TestService” points to our configuration (from above) file and tells the service which map we want to use. “Flyways” is the name of a layer in Dave’s service. “10” is the id of the feature we want, and “g=true” tells the service to return the geometry (there are cases where you don’t want that, b/c it can make the size of the response baloon pretty quickly.) So, that is pretty RESTy, yes?

The response from that HTTP GET looks like:

{  "type": "Feature","geometry": {"type": "Polygon",

"coordinates": [

[

[ 44.3231, 14.2555], [43.2827,13.6294],[ 43.285, 13.6503],        (...lots more coords...)     ]

},

"properties": {

"Ssp": "maculosus",

"Species": "Burhinus capensis",

"Shape_Length": "4.59630121045653",

"TSN": "0",

"extent": "42.9447173339071,13.6294196034485,44.3230947030963,15.0950604545328",

"SpeciesCod": "BURCA",

"WISDOM_SpeciesID": "557",

"OBJECTID": "10",

"Code": " ",

"Shape.area": "1.28930099595786"

}

}

Which is valid GeoJson. Neat, eh?

I know what you are saying. “Glenn, why would I use this when ArcGIS Server 9.3 will have a REST API?? HMMM?” Well, that is a great question. The only answer I have is that you can use this now, contribute to it, and make the world a better place. Also, you can learn about a lot of stuff, like REST, WCF, GeoJSON, AJAX, the Castle Project stuff, and much, much more. Plus, there are some real brains on this project (I am not one of them) so you can sop up their wisdom as well. Also, I heard a nasty rumor that the AGS REST API wasn’t using GeoJSON, but a proprietary spatial JSON format, which is a bit disconcerting, if not totally expected.

In future posts I’ll break down the ArcDeveloper REST API architecture, showing how you can write your own providers and formatters. We still have a TON of work to do on it, but we’re on our way.