Image(ry) is Everything

This post covers obtaining a high-resolution image of post-earthquake Christchurch and creating a mosaic dataset to manage it.

It’s that time of year when the students in ERST310/607 are hard at work on their independent GIS analysis projects; this also means a bit of trouble shooting at GIS Central.  One student, who is looking at some habitat differences in Christchurch pre- and post-earthquakes, got in touch when trying to track down some decent (and recent) imagery for her project.  She was wanting to derive land cover polygons from the image using some remote sensing techniques.  A few options came to mind, firstly some imagery from ECan, a portion of which is shown below:

Gorgeous imagery, yes!  This image has a resolution (pixel size) of around 0.5 m by 0.5 m which is pretty awesome, but the image has a few issues.  The first relates to time – if you look closely you can satisfy yourself that this imagery is probably from the 2011; very possibly from 22 or 23 February given the lack of fences around the cathedral.  You can even see people standing around taking it all in.  So it’s probably from just after the 6.3 and doesn’t capture the changes in the urban fabric that may have displaced habitats (i.e. think about all those new housing subdivisions).  The other problem is that this image is available as a web service rather than having the actual image.  (We talked about web services in an earlier post and how they can be added to your maps.)  This image was added to the map from the ECan servers.  Unless the right kind of services are available, when a layer like this is added from the internet, we aren’t able to use it for analysis – at best it serves as a sort of basemap (not unlike the basemaps that we can add in ArcMap by clicking the wee arrow next to the Add Data button and going to “Add Basemaps…”).  So while this one looks great on the map, all it can really do is sit there passively in the background and make our map look nice.

So another approach was needed.  While we do hold a lot of imagery on the J: Drive, it’s mostly pre- and just after the 2011 earthquakes, so wasn’t of much use to the student.  Time to look at some data portals, and the LINZ Data Service in particular:

Well that was almost too easy – the main entry is for a 0.075 m aerial photo of Christchurch circa 2015-16.  Jackpot!  The 0.075m refers to the resolution – each pixel of the image is 0.075 m by 0.075 m in size.  That’s glorious!  Next I added that layer to the map at right (by clicking on the “+”) to get some details on the image:

Good extent.  Certainly can’t complain about the resolution.  And it’s pretty much cloud free.  The planets are aligning.  The only major problem here is shown in the upper right hand corner: the image is 41 Gb in size.  That’s a lotta samolians (not so much in dollars as in gigabytes)!  When I click on the Download, the problem becomes more apparent:

There’s a 3.5 Gb download limit.  Plan B – note the window above that says “Original Resolution”.  I can change the resolution which will reduce the file size.  My options are1/2, 1/4 or 1/8 the original resolution.  With a bit of trial and error, I used the 1/4 resolution as JP2 images to get the download below the 3.5 Gb limit.  This ends up with a resolution of around 300 cm.  The download gets packaged as a zip file which you then download but once I had that locally, another problem emerged.  This image gives you a sense of the problem:

Instead of having one whopping great image, I’ve actually got several thousand files, 27,973 to be exact.  Some of these are images (the *.jp2s – 6,993 of those) some are xml files that have metadata, and some are *.j2w files (these are small text files that tell ArcMap where to place the images).  This isn’t an uncommon approach with images that cover large areas – it is broken up into a collection of tiles that match up and fit together to cover the area.  Each file is prefixed with something like “BW23_500_XXXXXX”.  The “BW##” part relates to the 1:50,000 scale topo map sheet that the tile sits within (Christchurch is covered by BW23 & 24, BX23, 24 & 25, and BY25).  The 500 relates to the tiling scheme and the six digits that follow give each tile a unique identifier.

So, what to do, what to do.  This poor student can’t add 6,993 separate images and then do her remote sensing analysis on each one – she’ll either lose her sanity (which she might anyway, that’s GIS) or way overshoot the project deadline.  Enter Mosaic Datasets!  These data structures are perfect for this problem.  When finished, the mosaic dataset stitches together all the tiles into a seamless whole that can be added and draws much more quickly than a collection of tiles.  Plus, they are honoured by analysis tools so will significantly help this cause.

To create the dataset, we first need a geodatabase to hold it.  In the J:\Data\Christchurch\Imagery folder I created a new file geodatabase called “ChristchurchImage2015”.  Inside that, I created a new mosaic dataset by right-clicking on the GDB and going to New > Mosaic Dataset.  This I called “CHC2015”.  At this point, it’s an empty container, so to fill it up, I right-clicked on the dataset name and went to “Add Rasters…”

The tool that opens lets you set which images to add.  Happily, I can point it at a folder (workspace) rather than all the individual files and it will add all the images in that folder:

Click OK and off it goes.  When finished I can add this to a map:

The green boxes show the outline (the footprints) of each of the tiles, so we can see there is good coverage across urban Christchurch and a smattering across Banks Peninsula to the southeast.  To see the actual images we need to zoom in.  I’ll do that in two steps.  First we’ll zoom in far enough that we can see the labels for each tile:

And then we’ll turn on the image:

Now we’re talkin’!  With this image you can see how the central city has been transformed.  We haven’t lost a lot by working with a mere 300 cm resolution and, more importantly, the student can now get on with her analysis.

This mosaic dataset is available for your use.  Have a look in J:\Data\Christchurch\Imagery\ChristchurchImage2015\CHC2015 and add it to your map.  The mosaic dataset allows us to work seamlessly with a sizeable collection of individual image tiles as if it were one image.  It will load and draw faster this way and if we were interested in serving this image up on the GIS server, the mosaic dataset is optimised to do this efficiently.  And mosaic datasets aren’t just for imagery.  They can be used with any collection of raster grids, be it elevation, or aspect, or slope, or whatever.

One important thing to note is that the images aren’t actually “inside” the mosaic dataset.  On its own, the dataset is quite small.  It basically references all the raster by maintaining a list of the layer and where they are stored.  The footprints and boundary layers are a part of the dataset but not the images themselves.  All brought to you through the power of geodatabases.

C