{"id":3747,"date":"2022-09-16T15:00:25","date_gmt":"2022-09-16T03:00:25","guid":{"rendered":"http:\/\/blogs.lincoln.ac.nz\/gis\/?p=3747"},"modified":"2023-05-07T02:42:40","modified_gmt":"2023-05-07T02:42:40","slug":"sometimes-simple-is-best","status":"publish","type":"post","link":"https:\/\/blogs.lincoln.ac.nz\/gis\/sometimes-simple-is-best\/","title":{"rendered":"Sometimes Simple Is Best"},"content":{"rendered":"

When faced with a potentially challenging task, a simple tool made it all possible.<\/em><\/p>\n

Sometimes using a simple tool can make all the difference.\u00a0 While working on a recent project<\/a>, one of my colleagues needed some data from the River Environment Classification<\/a>\u00a0(REC) dataset, developed by our good friends at NIWA.\u00a0 This is a really useful surface water layer with a number of different uses.\u00a0 Its original aim was to map all of NZ’s river reaches while also classifying them in a hierarchical schema which took into account six factors: climate, source of flow, geology, land cover, network position and valley landform.<\/p>\n

\"\"<\/p>\n

By being able to group river reaches at a level of your choice, the results can allow you to identify reaches that have common characteristics, regardless of where they are. In other words, rivers may be geographically separated but still share common environmental factors.\u00a0 This may help with, say, finding the reaches that have characteristics beneficial for Canterbury mudfish populations or for the design of monitoring programmes.\u00a0 For many, just having a national scale layer of rivers is useful in and of itself, but there are times when the extra depth those attributes provide comes in very handy.<\/p>\n

Case in point: in this\u00a0project we’ve been looking at some river characteristics to support the design water quality monitoring programmes at specific sites (and oh, the (GIS) stories I could tell you…another time).\u00a0 Of particular interest for my colleague were the Climate and Source of Flow attributes in the REC data.\u00a0 And here’s where the problem arose.<\/p>\n

There are currently two version of REC, oddly enough known as version 1 and version 2.\u00a0 Even odder, there are some subtle differences between the two, not least is that REC1 has 576,688 reaches while REC2 has 593,517.\u00a0 My colleague has been using REC2 reaches for his modelling and we quickly saw that the Climate and Source of Flow attributes weren’t included in REC2 – data he needed for some subsequent analysis.\u00a0 At first I though there would be no big problems here\u00a0– I knew that each reach had a unique identifier and surely<\/em> it’s the same in both versions, right?\u00a0 Alas, no.\u00a0 The IDs are close but are nonetheless different, so my initial thought of a simple table join<\/a> wasn’t going to work.\u00a0 To make matters worse, the river lines aren’t in the same place between the two versions – in the image below, REC1 is in green and REC2 is in blue:<\/p>\n

\"\"<\/p>\n

(Ed. Hmmmm…what’s with the jagged nature of those lines?\u00a0 And why don’t they match the basemap very well?\u00a0 C. I’ll get to that.)<\/em><\/p>\n

So, how am I going to transfer the attributes from V1 to V2 without there being a unique ID between the two?\u00a0 This was starting to feel like it was going to be a big job (and of course they needed these data today!)\u00a0 After I gave it some more thought, though, a very simple solution presented itself; the image above helps to illustrate – I’ll zoom in a bit:<\/p>\n

\"\"<\/p>\n

Each reach has been labelled with its unique ID – you can see they’re not the same and that they are in different places, even though they are representing the same reach of the Avon.\u00a0 What I’ve got going for me is that they are close<\/em> to each other, so some proximity tools might help my cause.\u00a0 Prime candidate: Near<\/a>.\u00a0 This handy tool does a very simple job – it calculates the straight line distance between two features.<\/p>\n

\"\"
https:\/\/pro.arcgis.com\/en\/pro-app\/2.8\/tool-reference\/analysis\/near.htm<\/em><\/span><\/figcaption><\/figure>\n

While the distance is\u00a0often very useful<\/a>, what I’m really counting on is something that comes along for the ride.\u00a0 When\u00a0the the Near tool runs, it adds\u00a0two new fields to the input layer, NEAR_ID and NEAR_DIST.\u00a0 (NB: this is one of the few tools that makes changes to the input layer rather than creating a new output layer.)<\/em>\u00a0 As you might imagine, the calculated distance is in NEAR_DIST but to solve this problem, the more valuable thing is the NEAR_ID – the ID of that nearest feature.\u00a0 If I know which feature from REC1 is nearest to its REC2 counterpart, I should be able to join the two together with a table join, et voila!<\/p>\n

The theory sounds good – let’s try it.\u00a0 The next few images will need some explanation.\u00a0 You’ll see both versions – REC1 is in green and REC2 is in blue, as above.\u00a0 I’ve already run the Near tool on the REC2 layer so you’ll see the NEAR_FID value which has the FID (Feature ID) of the nearest REC1 reach.\u00a0 For this first example, it seems to have worked well:<\/p>\n

\"\"<\/p>\n

No problem here.\u00a0 I’m doing a bit of spot checking to give me confidence that the output is doing what I need to do.\u00a0 On checking one of the other ones, it didn’t look so hot:<\/p>\n

\"\"<\/p>\n

This one hasn’t worked so well – Near has identified a feature that’s nearby, but not the right one.\u00a0 I’ve got almost 600,000 reaches here and this makes me\u00a0think I can’t rely on this working for every single reach.\u00a0 I was assuming (hoping) that the tool works off the centre<\/em> of each feature but based on this\u00a0I think it’s working off the first point that got created on these features.\u00a0 Quite likely this won’t work extensively, so back to the drawing board.<\/p>\n

That idea of the feature centre got me thinking.\u00a0 If I convert my reach lines to points, maybe I could use those<\/em>\u00a0features with the Near tool.\u00a0 It’s an easy thing to do with the Feature to Point<\/a> tool – and the point is at the geometric centre of the feature (which sometimes that means it’s not on the line).\u00a0 For the map extent above, my points look like this:<\/p>\n

\"\"<\/p>\n

It’s not perfect, but I think I’ll get some more robust results out of this.\u00a0 So, rerunning the Near tool gives me the linkage between REC2 and REC1 on the point layers:<\/p>\n

\"\"<\/p>\n

Next to last thing to do is a table join to bring the REC1 Climate and Src_of_Flow attributes together with the REC2 reaches (right-click on the REC2_points layer > Joins and Relates > Add Join):<\/p>\n

\"\"<\/p>\n

(Don’t worry about the warning triangles – everything’s under control.\u00a0 Ed. Yeah, right<\/em>)<\/p>\n

It looks like we’ve now brought the REC1 attributes into the REC2 table – which should make my colleague very happy:<\/p>\n

\"\"<\/p>\n

Last step.\u00a0 My colleague is not a GIS user.\u00a0 He’s much more of an R guy, but we still get along well.\u00a0 I know that he’d like to have a CSV (comma separated values) file to work with – easy peasy.<\/p>\n

With the table open, click on the three horizontal bars at upper right and choose Export:<\/p>\n

\"\"<\/p>\n

Then save the output to the project folder (not the geodatabase) with a “.csv” on the end:<\/p>\n

\"\"<\/p>\n

Which gives me this to email off to him (with a bit of simplifying) and then sit back and bask in the glory:<\/p>\n

\"\"<\/p>\n

Job done.\u00a0 I have to say that when I first got wind of\u00a0this job, I thought I was in for a whole lot of pain, perhaps even needing a Python script.\u00a0 Happily, a simple tool ended up doing the job quite nicely, or at the very least, making it much easier to do.\u00a0 Sometimes, simple is best.\u00a0 I’d venture to go a step further and say that almost all<\/em> of the time, simple is best.\u00a0 Or at least as simple as possible.<\/p>\n

C<\/p>\n","protected":false},"excerpt":{"rendered":"

When faced with a potentially challenging task, a simple tool made it all possible. Sometimes using a simple tool can make all the difference.\u00a0 While working on a recent project, one of my colleagues needed some data from the River Environment Classification\u00a0(REC) dataset, developed by our good friends at NIWA.\u00a0 This is a really useful […]<\/p>\n","protected":false},"author":3,"featured_media":3766,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3747","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/3747","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/comments?post=3747"}],"version-history":[{"count":2,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/3747\/revisions"}],"predecessor-version":[{"id":4053,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/3747\/revisions\/4053"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/media\/3766"}],"wp:attachment":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/media?parent=3747"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/categories?post=3747"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/tags?post=3747"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}