{"id":3050,"date":"2021-05-20T15:58:17","date_gmt":"2021-05-20T03:58:17","guid":{"rendered":"http:\/\/blogs.lincoln.ac.nz\/gis\/?p=3050"},"modified":"2023-05-07T03:02:46","modified_gmt":"2023-05-07T03:02:46","slug":"the-best-laid-plans","status":"publish","type":"post","link":"https:\/\/blogs.lincoln.ac.nz\/gis\/the-best-laid-plans\/","title":{"rendered":"The Best Laid Plans"},"content":{"rendered":"<p><em>In this post we look at different strategies\u00a0to approach\u00a0an analysis plan.\u00a0 (Updated 26 May with additional comments from a colleague.)<\/em><\/p>\n<p>We talk a lot about analysis on the GIS Blog and there have been a lot of examples with specific tools and how a particular analyses werecarried out.\u00a0 This is all (hopefully) good after-the-fact stuff, bearing in mind that a lot often goes into a workflow before we get to those final layers. I like to think about GIS as a problem solving tool but that works on two levels &#8211; first is how we approach the spatial aspects of a problem and how GIS can help and second is the GIS workflows themselves.\u00a0 How does one know what to use when?<\/p>\n<p>Along those lines I had a student ask a very good question the other day in class.\u00a0 We were talking through various analysis options for a class project, when, with a bit of noticeable frustration, he said something along the lines of (and I&#8217;m paraphrasing), I know how to use these tools, but how do I figure out how to put it all together?<\/p>\n<p>Fair question.\u00a0 It&#8217;s kind of like learning a language, in many ways.\u00a0 Vocabulary (tools) is one thing, but grammar (workflows) can be quite another &#8211; knowing all the words in the world makes no difference unless you can put them together in\u00a0a meaningful way.\u00a0 On top of that, there&#8217;s what just <em>sounds<\/em> right, for lack of a better word, and that&#8217;s the sort of thing that only comes with experience.<\/p>\n<p>So in this post\u00a0I&#8217;ll do my best to posit some strategies.\u00a0 Here are the main headings:<\/p>\n<ul>\n<li>Begin with the end in mind and work backwards<\/li>\n<li>Raster or vector?<\/li>\n<li>Use flow diagrams<\/li>\n<li>Analyst, know they data<\/li>\n<li>&#8230;And thy tools<\/li>\n<li>Check, check and double-check<\/li>\n<li>Know when to walk away<\/li>\n<\/ul>\n<p>So, on with it then.\u00a0 I&#8217;ll apologise at the outset that this is a long post with no pretty pictures.<\/p>\n<p><strong>Begin with the end in mind and work backwards<\/strong><\/p>\n<p>One really effective strategy I find myself using is starting at the end and working backwards.\u00a0 This means knowing as clearly as you can what the end result should look like. Be as clear as you can &#8211; if it&#8217;s not clear, ask questions of your client\/colleague\/supervisor\/lecturer (sometimes, <em>you&#8217;re<\/em> the client!)\u00a0 For instance, we&#8217;ve recently started some work looking at identifying the best areas to plant hops in, so one end result would be a map showing the best areas.\u00a0 Working backwards, I can surmise that I&#8217;m going to need to know what conditions hops do best in, which is likely to involve things like soil properties, rainfall, maybe slopes, probably temperatures and levels of solar radiation.\u00a0 I&#8217;m no expert on hops, so I&#8217;m just making some (hopefully) educated guesses here in lieu of anything more specific.\u00a0 I&#8217;d like to have some spatial criteria that allow me to narrow things down more effectively and I&#8217;ll need to rely on someone with some expert knowledge for that.\u00a0 But assuming I can get that, going further backwards allows me to think about data.\u00a0 Soils &#8211; we&#8217;ve got that in a variety of forms: <a href=\"https:\/\/lris.scinfo.org.nz\/search\/?q=smap\" target=\"_blank\" rel=\"noopener noreferrer\">S-Map<\/a> and the<a href=\"https:\/\/lris.scinfo.org.nz\/search\/?q=fsl\" target=\"_blank\" rel=\"noopener noreferrer\"> Fundamental Soil Layer<\/a> being the prime suspects, both vector polygon layers.\u00a0 Slope?\u00a0 Our 25 m DEM gives us coverage across the whole country, so tick.\u00a0 There are several climate raster layers from NIWA that should give us most of what we need in that realm.<\/p>\n<p>In between input layers and output layers I can see that I&#8217;ll need to somehow manipulate the inputs\u00a0to match whatever criteria I&#8217;m given &#8211; so tool-wise, I&#8217;m anticipating I&#8217;ll be reclassifying my raster layers and perhaps adding new attributes to the vector layers and then at some point I&#8217;ll need to combine all my layers into one output layer, which could be either some map algebra or spatial joins.\u00a0 Depending on my criteria, I&#8217;ll need to think carefully about how I combine the layers (and how they are coded) to get a meaningful result, but this gives me a very rough picture of my workflow, though there&#8217;s still a lot of detail to be filled in.<\/p>\n<p>So what I&#8217;m trying to illustrate here is that by starting with my endpoint, I think I&#8217;ve given myself a way to think though\u00a0how I&#8217;ll get to that endpoint.\u00a0 And already I&#8217;ve had to start thinking\u00a0through my next strategy.<\/p>\n<p><strong>Raster or Vector?<\/strong><\/p>\n<p>As we saw above, I&#8217;m likely to have a mixture of vector and raster inputs.\u00a0 This is quite common to most GIS projects.\u00a0 At some point\u00a0though, we&#8217;re needing to join layers together to get that final output, in which all my layers need to be either raster or vector, but not both.\u00a0 This is a massive generalisation as some project don&#8217;t fit this mold, but I&#8217;d venture to say that a majority do.\u00a0 It&#8217;s not like they&#8217;re matter and anti-matter that annihilate upon contact, but for most cases, vector and raster don&#8217;t mix.\u00a0 So a decision has to get made at some point: am I doing a raster analysis or a vector analysis?<\/p>\n<p>This is a big question, worthy of its own post, but I&#8217;ll hit the highlights here.\u00a0 Sometimes the answer is very clear:<\/p>\n<ul>\n<li>If measuring distance or area is critical, then vector is usually the way to go.\u00a0 Even with high-resolution (i.e. 1 m resolution) raster data, we still can&#8217;t measure distances and areas as precisely as we can with vector features;<\/li>\n<li>If modelling things like best routes through a linear network (e.g. roads, rivers, fiber optic cables), then hands down it&#8217;s vector and <a href=\"http:\/\/blogs.lincoln.ac.nz\/gis\/a-smokin-script\/\" target=\"_blank\" rel=\"noopener noreferrer\">network analysis<\/a>;<\/li>\n<li>Working with terrain and topography (slope, aspect, <a href=\"http:\/\/blogs.lincoln.ac.nz\/gis\/a-shady-topic-hillshade-layers\/\" target=\"_blank\" rel=\"noopener noreferrer\">hillshades,<\/a> <a href=\"http:\/\/blogs.lincoln.ac.nz\/gis\/data-with-a-view-2\/\" target=\"_blank\" rel=\"noopener noreferrer\">viewsheds<\/a>)\u00a0cries out for raster.\u00a0 It&#8217;s vastly superior for handling these as well as continuous data like rainfall, temperatures, air pollution levels, i.e. things we can measure everywhere.<\/li>\n<\/ul>\n<p>This also ties in with knowing what you&#8217;re endpoints are a bit.\u00a0 There are a few other things to consider on this decision point.\u00a0 In many cases, a given analysis can be done in either vector or raster; the two approaches tend to differ on the level of complexity (i.e. number of steps) and processing time so, often, it depends (something my students tire of hearing me say &#8211; it&#8217;s my stock response).<\/p>\n<p>In the hops example, I&#8217;m anticipating that most of my input data are raster in their raw form.\u00a0 It&#8217;s feasible to convert data between the two models &#8211; noting that we&#8217;re almost always sacrificing something, be it resolution, or continuity.\u00a0 There are suites of tools for converting vector to raster and vice-versa.\u00a0 One direction is fairly straight-forward, the other has potential fish hooks so approach carefully.\u00a0 With the hops, most of my data at this early stage are raster, so I&#8217;m already leaning towards that as my medium, knowing that it will mean converting my polygons to rasters.<\/p>\n<p>Another related consideration is analysis extent and its effect on processing time.\u00a0 When comparing the two data models, rasters are very simple compared to vector (polygons especially).\u00a0 In another post we&#8217;ll see how rasters are basically matrices and are easily represented as simple text files.\u00a0 If we look at, say, a vector polygon layer, my data have to keep track of all the points that make up the corners of the polygons, and all the lines that connect them as well as all the attributes.\u00a0 When we do spatial joins, we usually end up creating a lot of new polygons.\u00a0 That takes processing time and more computational grunt than a raster calculation.\u00a0 As you might imagine, as the extent of the analysis increases, the respective processing times also increase, but they increase <em>much<\/em> more for vector that for raster.\u00a0 As a <strong>very<\/strong> general statement, raster analysis is generally faster than vector (but it depends&#8230;)\u00a0hence the\u00a0old, hackneyed statement: <a href=\"https:\/\/osgis.org\/2020\/09\/raster-vs-vector-maps-whats-the-difference-which-are-best\/\" target=\"_blank\" rel=\"noopener noreferrer\">raster is faster but vector is correcter<\/a> (sic).\u00a0 (<a href=\"https:\/\/geozoneblog.wordpress.com\/2013\/04\/19\/raster-vs-vector\/\" target=\"_blank\" rel=\"noopener noreferrer\">Here too<\/a>).\u00a0 \u00a0As the extent of the analysis gets larger (and assuming I could use either model), I lean more towards raster for this reason.\u00a0 Doing analysis at a national scale is often a good reason to go with raster (but not alwasy).\u00a0 As processing power becomes faster and faster, this becomes less of an issue, but it&#8217;s still relevant for most of us.<\/p>\n<p>So with the hops, I&#8217;m thinking raster. (<em>Ed. couldn&#8217;t you have just said that earlier?\u00a0 Geez&#8230;<\/em>)<\/p>\n<p><strong>Use Flow Diagrams<\/strong><\/p>\n<p>Now that I&#8217;ve got a slightly better picture of what I&#8217;m doing, I might use <a href=\"http:\/\/blogs.lincoln.ac.nz\/gis\/flow-diagrams-that-work\/\" target=\"_blank\" rel=\"noopener noreferrer\">flow diagrams<\/a> to help me think through the overall workflow.\u00a0 I might use ModelBuilder in Pro, or could just as easily use the back of a napkin.\u00a0 The important thing is trying to think through everything into an organised, coherent workflow.\u00a0 Using something graphical helps me see potential pitfalls or may suggest ways of doing things more efficiently.\u00a0 This works for me personally, but may not be effective for everyone.\u00a0 Flow diagrams work well on at least two levels &#8211; one they can be a nice planning tool a t the start of a project and then, at the end of the analysis, they can serve as a nice, graphical summary of what you actually did.<\/p>\n<p>Which brings up an important point about flow diagrams &#8211; the first one seldom matches the final one.\u00a0 And that&#8217;s okay.\u00a0 More often than not, my original plan changes as I get to know the data better and the possible tools better.\u00a0 Here&#8217;s a <a href=\"http:\/\/blogs.lincoln.ac.nz\/gis\/deleting-smokers-part-2\/\" target=\"_blank\" rel=\"noopener noreferrer\">good example<\/a> of typing out loud to figure out the best tool to use in one case.\u00a0 Flow diagrams are like essay outlines; they tend to change as you get deeper into the task, which is usually a good thing.\u00a0 At the early stages its helpful to give yourself a place to start, knowing that it will probably change.\u00a0 How many well planned trips have you taken that ended up being completely different from what you started with?\u00a0 (And were they better for it?)<\/p>\n<p><strong>Analyst, know thy data<\/strong><\/p>\n<p>Have you ever went to put petrol in the car (assuming you&#8217;ve got one of those carbon spewing internal combustion engines &#8211; heathen) and realised you were holding the diesel hose?\u00a0 Happens to the best of us.\u00a0 Data are fuel that make the GIS engine go, so it pays to spend time making sure you&#8217;ve got the right data.\u00a0 And that means being conscious of the spatial features (are they in the right place?\u00a0 Are they are the right scale for your analysis?) and the attributes (Do the attributes\u00a0give you the information you need?\u00a0 If not, can you some how add it in?).\u00a0 Without good quality data going into your analysis, you can only get poor quality outputs.\u00a0 The Red Cross screens all its blood donations &#8211; you should screen your data.\u00a0 Be aware of <a href=\"http:\/\/blogs.lincoln.ac.nz\/gis\/data-data-everywhere-but-what-about-what-i-need\/\" target=\"_blank\" rel=\"noopener noreferrer\">data on the J: drive<\/a> as well as the <a href=\"http:\/\/blogs.lincoln.ac.nz\/gis\/the-data-i-need-arent-on-the-j-drive\/\" target=\"_blank\" rel=\"noopener noreferrer\">online data portals<\/a>\u00a0or data you may have <a href=\"http:\/\/blogs.lincoln.ac.nz\/gis\/dont-let-that-dragonfly-get-away-offline-mapping-in-vanuatu\/\" target=\"_blank\" rel=\"noopener noreferrer\">collected yourself<\/a>.<\/p>\n<p><strong>&#8230;And thy tools<\/strong><\/p>\n<p>You could easily think of GIS as a craftsperson&#8217;s toolkit.\u00a0 It&#8217;s just jam packed full of things, 90% of which you might never need.\u00a0 But quite often, there&#8217;s a tool for the job.\u00a0 Finding it can sometimes be a challenge.\u00a0 Keyword searches in both the Geoprocessing pane in Pro as well as every analyst&#8217;s best friend, Google, can often put you on track to potential tools to use.\u00a0 But it&#8217;s got to be the right tool for the job.\u00a0 Trawl through the tools and toolsets built into the Geoprocessing Toolboxes and see what you find.\u00a0 <a href=\"https:\/\/idioms.thefreedictionary.com\/chance+favors+the+prepared+mind\" target=\"_blank\" rel=\"noopener noreferrer\">Chance favours the prepared mind<\/a>.<\/p>\n<p>I find myself often referring to GIS as a <a href=\"https:\/\/www.merriam-webster.com\/dictionary\/craft\" target=\"_blank\" rel=\"noopener noreferrer\">craft<\/a>.\u00a0 Effective craftspeople know their tools and they know the material they work with intimately, along with the limitations and the opportunities they open up.\u00a0 Of course, time and experience all contribute to crafting, so give yourself a break now and then.\u00a0 It just takes time to get familiar and comfortable with all the options.<\/p>\n<p><strong>Check, check and double-check<\/strong><\/p>\n<p>So we&#8217;ve talked a lot about planning, filling in all the gaps between input data and outputs.\u00a0 The doing comes next.\u00a0 One of the most effective strategies I&#8217;ve developed over the years is to constantly be checking and double-checking my outputs at all stages.<\/p>\n<p>Sometimes this is anticipating the results before I run a tool and then checking if the output&#8217;s what I was expecting.\u00a0 If not, well, either I was wrong, or something wasn&#8217;t set properly (usually it&#8217;s the former&#8230;).\u00a0 Either way, I may pick up an error before it propagates through the rest of my analysis.\u00a0 There&#8217;s nothing worse than standing in front of the PowerPoint slide with your output map at the conference and someone saying, &#8220;ummm&#8230;that&#8217;s wrong&#8230;.&#8221;.\u00a0 Give yourself opportunities to check the outputs and ensure that 1) they make sense, and 2) they&#8217;re correct.\u00a0 We don&#8217;t always know what the correct answer is, but the more you can check things as you&#8217;re going, the more likely it is that they are correct.<\/p>\n<p>My last strategy:<\/p>\n<p><strong>Know When to Walk Away<\/strong><\/p>\n<p>Anyone who&#8217;s worked with GIS knows how frustrating it can be.\u00a0 Sometimes, it&#8217;s best to close log off the computer and just walk away, go outside, gets some air, think about something else.\u00a0 I have long maintained a dysfunctional love\/hate relationship with GIS.\u00a0 And it&#8217;s often quite frustrating to realise that more often than not, it&#8217;s done exactly what I told it to do and the mistake was mine and my expectation.\u00a0 (That&#8217;s <em><strong>especially<\/strong> <\/em>true of coding.)\u00a0 Sometimes banging your head against that digital wall does more harm than good.\u00a0 GIS is frustrating at the best of times but OneDrive has certainly complicated things recently.\u00a0 I am not a big fan.<\/p>\n<p>So, that&#8217;s a bit of a brain dump about doing GIS of some of the strategies that seem to work for me.\u00a0 But I&#8217;m still learning, so I wouldn&#8217;t call it the definitive list.\u00a0 Have you got something that works particularly well?\u00a0 Drop in a comment and share it.<\/p>\n<p>I&#8217;m presuming I&#8217;ve lost all my readers by this point, happily sleeping away, so if you&#8217;ve persevered to the very last full stop, let me know and I&#8217;ll shout you a Moro bar <span style=\"font-size: 10pt\"><em>(Terms and conditions apply.\u00a0 On road charges not included.\u00a0 Only open to New Zealand citizens whose surnames begin with Q.\u00a0 And if you order by midnight tonite&#8230;!)<\/em><\/span><\/p>\n<p>C<\/p>\n<p>Postscript: I ran this post by my colleague from Manaaki Whenua, James Barringer, from whom I&#8217;ve learned a lot about GIS.\u00a0 He had some good comments that I thought would be worthwhile, so below are his lightly edited thoughts, used with his permission.<\/p>\n<p><em>You missed one of my favourites for problem solving \u2013 \u201cDivide and Conquer\u201d \u2013 sort of comes under flow diagrams \u2013 but break complicated analyses down into more manageable sub-tasks.<\/em><\/p>\n<p><em>Don\u2019t entirely agree about raster and vector \u2013 I think you can work with both very easily now along the way &#8211; but your final product needs to follow your advice about picking one or the other depending on needs.<\/em><\/p>\n<p><em>The end in mind bit \u2013 absolutely \u2013 call it what you like but you need to know where you\u2019re trying to get to, in order to get there \u2013 look up what the Cheshire Cat said to Alice in Wonderland when she asked him the way.<\/em><\/p>\n<p style=\"padding-left: 30px\"><span style=\"font-size: 10pt\"><em>For the record:<\/em><\/span><\/p>\n<p style=\"padding-left: 30px\"><span style=\"font-size: 10pt\"><em>\u201cAlice asked the Cheshire Cat, who was sitting in a tree, \u201cWhat road do I take?\u201d<\/em><\/span><br \/>\n<span style=\"font-size: 10pt\"><em> The cat asked, \u201cWhere do you want to go?\u201d<\/em><\/span><br \/>\n<span style=\"font-size: 10pt\"><em> \u201cI don\u2019t know,\u201d Alice answered.<\/em><\/span><br \/>\n<span style=\"font-size: 10pt\"><em> \u201cThen,\u201d said the cat, \u201cit really doesn\u2019t matter, does it?\u201d<\/em><\/span><\/p>\n<p style=\"padding-left: 30px\"><span style=\"font-size: 10pt\"><em>I usually paraphrase the last line to something like &#8211; \u201cif you don\u2019t know where you\u2019re going you\u2019re hardly likely to get there\u201d.<\/em><\/span><\/p>\n<p><em>In that section you also say you\u2019re \u201cno expert on hops\u201d but to me the real skill required here is knowing\/learning enough domain knowledge to have a useful conversation with your client and come away knowing\/understanding what they want.\u00a0 If you can\u2019t do that you are a GIS tech\/programmer who needs detailed technical instructions about every step.\u00a0 Basically a whizz at taking a workflow and implementing it, but unable to know if the desired outcome was achieved.\u00a0 Very hard for you to check results because you won\u2019t have a gut feeling for what the outcome should look like.\u00a0 If you can do this you can think of yourself as a GIS Analyst who can solve real problems.<\/em><\/p>\n<p><em>Same thinking as you say applies to data and tools \u2013 critical that data and tools are fit-for-purpose which means understanding both the task and the data properly.\u00a0 Garbage in \u2013 Garbage out applies.\u00a0 But with the added dimension that you can have good data put through an inappropriate analysis, or inappropriate data put through an appropriate analysis.\u00a0 An understanding of scale, resolution, accuracy, precision, units of measure (nominal, classified, interval, ratio) and what these all mean in the context of your analysis is critical.\u00a0 A lifetime of learning to be had in there!\u00a0<\/em><\/p>\n<p><em>Cheers,\u00a0 JB<\/em><\/p>\n<p>Thanks James!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this post we look at different strategies\u00a0to approach\u00a0an analysis plan.\u00a0 (Updated 26 May with additional comments from a colleague.) We talk a lot about analysis on the GIS Blog and there have been a lot of examples with specific tools and how a particular analyses werecarried out.\u00a0 This is all (hopefully) good after-the-fact stuff, [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3050","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/3050","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/comments?post=3050"}],"version-history":[{"count":1,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/3050\/revisions"}],"predecessor-version":[{"id":4081,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/3050\/revisions\/4081"}],"wp:attachment":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/media?parent=3050"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/categories?post=3050"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/tags?post=3050"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}