{"id":1738,"date":"2017-05-17T21:22:20","date_gmt":"2017-05-17T21:22:20","guid":{"rendered":"http:\/\/blogs.lincoln.ac.nz\/gis\/?p=1738"},"modified":"2023-05-07T00:32:16","modified_gmt":"2023-05-07T00:32:16","slug":"keeping-your-feet-dry-with-mosaic-datasets","status":"publish","type":"post","link":"https:\/\/blogs.lincoln.ac.nz\/gis\/keeping-your-feet-dry-with-mosaic-datasets\/","title":{"rendered":"Keeping Your Feet Dry with Mosaic Datasets"},"content":{"rendered":"

Mosaic datasets can be used store and manage a collection of raster grids.\u00a0 We demonstrate this by downloading some high resolution DEMs and the stitch them together in a mosaic dataset.\u00a0 (Updated May 2019 to point out how to fix any problems from the start.)<\/em><\/p>\n

Even as we speak, students in ERST202 are engaged in a fairly big project looking at the impacts of potential sea level rise on the New Zealand coast.\u00a0 We’re modelling the effects of a 2 m rise in sea level and, as you might imagine, a key first step is trying to determine the areas that would be directly affected.\u00a0 For this we need some good elevation data, handily provided by some digital elevations models (DEMs).\u00a0 While we’ve got a national scale DEM with a grid cell size (resolution) of 25 m by 25 m, this is pretty coarse for our purposes.<\/p>\n

Happily, the good folk of 202 are getting to be well versed with finding data on the internet<\/a>.\u00a0 A bit of detective work took some of them to the LINZ Data Service<\/a>, and look what they found:<\/p>\n

\"\"<\/a><\/p>\n

Top of the list is a 1 m DEM for Christchurch and Selwyn derived from LiDAR data<\/a> – sweeeeeeet!\u00a0 All up, this DEM is 2.7 Gb in size:<\/p>\n

\"\"<\/a><\/p>\n

After signing in, they zoomed into their area of interest, which was Sumner\/Godley Head:<\/p>\n

\"\"<\/a><\/p>\n

The LINZ Data Service allows one to crop a smaller area out by clicking on the crop button, \"\"<\/a>, and drawing a box around the area to be cropped (I’m clipping out a smaller area than they needed just for demonstration purposes):<\/p>\n

\"\"<\/a><\/p>\n

This gets the file size down to 5.1 Mb, much more manageable.\u00a0 When the download button is clicked, you’ve got a choice of format and coordinate systems:<\/p>\n

\"\"<\/a><\/p>\n

We try and keep all of our data in New Zealand Transverse Mercator (NZTM) so that’s a good choice for the projection.\u00a0 For rasters we could download them as TIFFs or ASCII files – I’ll go with TIFFs for this but maybe we’ll talk about ASCII files another day.\u00a0 The data get packaged up in a zip file and once that’s been downloaded and unzipped, I can add the DEMs to a map:<\/p>\n

\"\"<\/a><\/p>\n

Well.\u00a0 That’s interesting.\u00a0 It’s certainly not what we’re used to when looking at DEMs – it’s clearly tiled.\u00a0 When downloading, the data service automatically tiles the DEM so in this case we’ve ended up with nine smaller grids.\u00a0 It would certainly be nice to somehow group these grids together rather than have to treat each grid separately.\u00a0 And this is where mosaic datasets<\/a> come in.<\/p>\n

These are data structures that allow you to store and manage collections of raster data layers.\u00a0 Once a dataset is set up, it’s as if the grids have been stitched together and treated as one whole.\u00a0 Mosaic datasets must reside in a geodatabase so we have to start there.\u00a0 To get this process going, I created a geodatabase:<\/p>\n

\"\"<\/a><\/p>\n

First I’ve got to create the mosaic dataset.\u00a0 This is done by right-clicking on the geodatabase and going to New > Mosaic Dataset:<\/p>\n

\"\"<\/a><\/p>\n

A dialogue box takes us through creating the dataset by giving it a name and setting its coordinate system:<\/p>\n

\"\"<\/a><\/p>\n

After clicking OK we’ve got a new, empty mosaic dataset.\u00a0 Next we need to add some rasters by right-clicking on the dataset name and going to Add Rasters…<\/p>\n

\"\"<\/a><\/p>\n

Since mosaic datasets are geared towards rasters, the options under “Raster Type” feature lots of satellite image formats.\u00a0 For our purposes, “Raster Dataset” will work fine.\u00a0 For the “Input Data” box, we can point the tool at a folder that holds all the rasters rather than having to add each one separately.\u00a0 Clicking OK adds the rasters to the dataset – here’s what it looks like when I add the mosaic dataset to my map:<\/p>\n

\"\"<\/a><\/p>\n

Wait…hang on…where are my data?\u00a0 Something’s weird: the maximum\/minimum values don’t make sense, ranging from 3.40282e+038 to -3.40282e+038.\u00a0 That’s just bizarre.\u00a0 But all is not lost. The dataset is actually fine and can be used for analysis straight away.\u00a0 It’s just not displaying correctly (and I admit to being stumped by this in lab only yesterday).\u00a0 To fix this, we need to build “overviews<\/a>“, which help the grid (now called an image in the mosaic dataset) redraw faster.\u00a0 We can do this by right-clicking on the dataset in Catalog and going to Optimize > Build Overviews:<\/p>\n

\"\"<\/a><\/p>\n

No need to change any settings in the tool.<\/p>\n

\"\"<\/a><\/p>\n

Click OK and off we go – here’s the result:<\/p>\n

\"\"<\/a><\/p>\n

Now that’s more like it!\u00a0 Notice how with the mosaic dataset we also have the green footprints of each input tile and a purple boundary line (these can be turned on and off) and the mosaic dataset will be honoured by all geoprocessing tools. So this grid is now good to go – here’s a slope layer derived from the dataset:<\/p>\n

\"\"<\/a><\/p>\n

(Finally, some colour…!)\u00a0 The grid can now also be used to idenitify all those areas affected by a 2 m rise in sea level (amongst other things).\u00a0 Doing this with nine tiles isn’t such a big deal, but it’s stitching them together has already simplified our analysis.\u00a0 Imagine if we were using the whole of the original grid, which could easily contain over 1,000 individual tiles (or more).\u00a0 And that’s where the real power of this data structure comes in – managing large amounts of raster data (DEMs, satellite images, aerial photos) as if they were one, fast-acting grid.\u00a0 It will also relieve that occasional heartburn (not really).\u00a0 And hopefully this particular one will help us keep our feet dry in the coming years.<\/p>\n

Getting it right at the start<\/strong><\/p>\n

What we’ve talked about above goes through how to fix the problem after you’ve created the mosaic dataset.\u00a0 It’s better to get it fixed from the start and here’s how.\u00a0 When you’re at the point of adding the rasters using the Add Rasters to Mosaic Dataset, scroll down to the bottom of the window and expand the Raster Processing section.\u00a0 Tick the Calculate Statistics and Build Raster Pyramids boxes.\u00a0 Under Mosaic Post-processing, tick Build Thumbnails and Update Overviews as shown below.\u00a0 This should allow your new dataset to display nicely.<\/p>\n

\"\"<\/a><\/p>\n

C<\/p>\n","protected":false},"excerpt":{"rendered":"

Mosaic datasets can be used store and manage a collection of raster grids.\u00a0 We demonstrate this by downloading some high resolution DEMs and the stitch them together in a mosaic dataset.\u00a0 (Updated May 2019 to point out how to fix any problems from the start.) Even as we speak, students in ERST202 are engaged in […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1738","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/1738","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/comments?post=1738"}],"version-history":[{"count":1,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/1738\/revisions"}],"predecessor-version":[{"id":4001,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/1738\/revisions\/4001"}],"wp:attachment":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/media?parent=1738"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/categories?post=1738"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/tags?post=1738"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}