{"id":1261,"date":"2016-03-29T21:37:17","date_gmt":"2016-03-29T21:37:17","guid":{"rendered":"http:\/\/blogs.lincoln.ac.nz\/gis\/?p=1261"},"modified":"2023-05-07T04:05:58","modified_gmt":"2023-05-07T04:05:58","slug":"python-scripting-with-the-field-calculator","status":"publish","type":"post","link":"https:\/\/blogs.lincoln.ac.nz\/gis\/python-scripting-with-the-field-calculator\/","title":{"rendered":"Python Scripting with the Field Calculator"},"content":{"rendered":"<p><em>This post details how you can use some Python scripting to do if\/else statements in the Field Calculator.<\/em><\/p>\n<p>If the title of this post and the intro haven&#8217;t put you to sleep already, read on! \u00a0Perhaps they&#8217;ve even piqued your interest! \u00a0<em>(There are support groups available for this condition.)<\/em><\/p>\n<p>I was recently contacted by a keen GISer (who shall remain nameless) with a problem they were having. \u00a0They were wanting to do some work around the Land Cover Database (LCDB) &#8211; a vector polygon layer, with up to 33 classes of land cover across the whole country. \u00a0To cover the whole country it uses a set of 479,353\u00a0polygons! \u00a0Previous versions were released for 1996, 2001, 2008 and this latest version has 2012 data, thus providing views\u00a0of New Zealand&#8217;s landcover across those years, like snapshots in a photo album. \u00a0The<a href=\"https:\/\/lris.scinfo.org.nz\/layer\/423-lcdb-v41-land-cover-database-version-41-mainland-new-zealand\/\" target=\"_blank\" rel=\"noopener noreferrer\"> lastest version (4.1)<\/a> includes data from the three previous versions so, in theory, one could track changes in land cover from 1996\u00a0\u00a0to 2012. \u00a0Here&#8217;s a shot of a portion of the table, focusing on the class names:<\/p>\n<p><a href=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/LCDBTable.jpg\" rel=\"attachment wp-att-1265\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1265\" src=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/LCDBTable.jpg\" alt=\"LCDBTable\" width=\"808\" height=\"417\" srcset=\"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/LCDBTable.jpg 1351w, https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/LCDBTable-300x155.jpg 300w, https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/LCDBTable-1024x529.jpg 1024w, https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/LCDBTable-768x397.jpg 768w\" sizes=\"auto, (max-width: 808px) 100vw, 808px\" \/><\/a><\/p>\n<p>So, this person was interested in looking at landcover\u00a0changes at particular study sites from one year to the next, with an eye towards quantifying changes based on % of total area in the study areas. \u00a0In the table above, most of these polygons haven&#8217;t changed across the years, but several have. \u00a0Some have gone from High Producing Exotic Grassland to Exotic Forest &#8211; and they wanted to quantify changes like this. \u00a0But how to do so?<\/p>\n<p>There&#8217;s a <a href=\"http:\/\/desktop.arcgis.com\/en\/arcmap\/10.3\/tools\/data-management-toolbox\/feature-compare.htm\" target=\"_blank\" rel=\"noopener noreferrer\">Feature Compare<\/a> tool (sort of like Compare in Word) that lets you look at differences between two layers, but we were\u00a0looking for changes <em>within<\/em> the table of a single layer; a different proposition. \u00a0My first thought was \u00a0to give yourself a new attribute that would signal if there had been changes from one year to another (a slightly simplified version of what they were after), which got me thinking about &#8220;if\/then&#8221; conditional statements. \u00a0And this takes us down the road of some simple scripting &#8211; abandon hope, ye who enter here&#8230;<\/p>\n<figure id=\"attachment_1272\" aria-describedby=\"caption-attachment-1272\" style=\"width: 461px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Inferno.jpg\" rel=\"attachment wp-att-1272\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-1272\" src=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Inferno.jpg\" alt=\"Abandon Hope\" width=\"461\" height=\"650\" srcset=\"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Inferno.jpg 625w, https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Inferno-213x300.jpg 213w\" sizes=\"auto, (max-width: 461px) 100vw, 461px\" \/><\/a><figcaption id=\"caption-attachment-1272\" class=\"wp-caption-text\">Source: http:\/\/viralyster.com\/a-helpful-illustrated-guide-to-dantes-inferno\/<\/figcaption><\/figure>\n<p>Python is a useful scripting language and is the preferred one for ArcGIS. \u00a0We&#8217;ve covered some\u00a0<a href=\"http:\/\/blogs.lincoln.ac.nz\/gis\/sticking-to-the-python-script\/\" target=\"_blank\" rel=\"noopener noreferrer\">scripting previously<\/a>, and here I was interested in using some very simple Python to get this done within the table. \u00a0After creating a new integer field in the table to store an integer (let&#8217;s call it Comparison), the basic idea was to:<\/p>\n<ol>\n<li>compare the landcover class names between two different years, and;\n<ul>\n<li>if the values for a\u00a0polygon are the same (no change) put a 0 in the Comparison field;<\/li>\n<li>if the values for\u00a0a polygon are different (a change) put a 1 in the Comparison field;<\/li>\n<\/ul>\n<\/li>\n<li>Do this for all the polygon records in the table.<\/li>\n<\/ol>\n<p>In scripting terms, we can do with an &#8220;if&#8221; statement, which reads something like this:<\/p>\n<pre style=\"padding-left: 60px\">if (some condition is true):<\/pre>\n<pre style=\"padding-left: 90px\">do this<\/pre>\n<pre style=\"padding-left: 60px\">if (some other condition is true):<\/pre>\n<pre style=\"padding-left: 90px\">do that<\/pre>\n<pre style=\"padding-left: 60px\">else:<\/pre>\n<pre style=\"padding-left: 90px\">do the other thing<\/pre>\n<p>Often, we might write a stand alone script, in a separate file that does this, but I wanted to try using the Field Calculator instead. \u00a0Using a smaller copy of the LCDB with only 160 records, called FieldCalculationTest, I added\u00a0a new integer field called &#8220;Comparison&#8221;. \u00a0The\u00a0Field Calculator can be opened directly from the table by right-clicking on the name of the field you want to put the result in, as shown below:<\/p>\n<p><a href=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/FieldCalculator.jpg\" rel=\"attachment wp-att-1266\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-1266\" src=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/FieldCalculator.jpg\" alt=\"FieldCalculator\" width=\"467\" height=\"565\" srcset=\"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/FieldCalculator.jpg 467w, https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/FieldCalculator-248x300.jpg 248w\" sizes=\"auto, (max-width: 467px) 100vw, 467px\" \/><\/a><\/p>\n<p>Amongst other things, I&#8217;ve got all my Fields in the upper lefthand box, some functions to choose from at right, and a window at the bottom where I can set up my calculations. \u00a0We can use this to put some Python script in by setting the Parser button to Python and ticking the &#8220;Show Codeblock&#8221; box:<\/p>\n<p><a href=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Parser.jpg\" rel=\"attachment wp-att-1267\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-1267\" src=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Parser.jpg\" alt=\"Parser\" width=\"469\" height=\"563\" srcset=\"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Parser.jpg 469w, https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Parser-250x300.jpg 250w\" sizes=\"auto, (max-width: 469px) 100vw, 469px\" \/><\/a><\/p>\n<p>I can now type in my script elements into the &#8220;Pre-Logic Script Code:&#8221; box and then call\u00a0it from the &#8220;Comparison =&#8221; box. \u00a0In the first, I need to set up a generic sort of code and in the second I can call it using specific fields from the table. \u00a0To keep it generic I&#8217;ll use two simple names (variables actually) that will stand in for my true field names: class1 and class2. \u00a0This means I could then run it using any two of my fields, so long are they&#8217;re of the same type. \u00a0the code&#8217;s pretty simple: compare two field values in a record; if there&#8217;s the same, put a 0\u00a0in the field, if they&#8217;re not, put a\u00a01. \u00a0When writing code, it&#8217;s often better practice to start with the negative\u00a0case first. \u00a0So a simple version would be:<\/p>\n<pre>if class1 != class2:<\/pre>\n<pre style=\"padding-left: 30px\">return 1<\/pre>\n<pre>else:<\/pre>\n<pre style=\"padding-left: 30px\">return 0<\/pre>\n<p>The &#8220;if class1 != class2:&#8221; translates to: if class1 is not equal to class2, await an action. \u00a0In this case the action is place a 1 in that record&#8217;s Comparison field. \u00a0For any other conditions, place a 0.<\/p>\n<p>So how does this work in practice? \u00a0We&#8217;ve first got to wrap the code in a function definition, give it a name, and specify any inputs that the function may take. \u00a0I can do this with the following text:<\/p>\n<pre>def compare(class1, class2)<\/pre>\n<p>The function&#8217;s name is &#8220;compare&#8221; and it takes two input variables, &#8220;class1&#8221; and &#8220;class 2&#8221;. \u00a0Those are generic names and you&#8217;ll see how they get used\u00a0below.<\/p>\n<p>Next, I can use the &#8220;Comparison =&#8221; box to specifically run this function for every record in the table (or only those that are selected if there are any). \u00a0The calculator knows to run it on the Comparison field because that&#8217;s where I started it from. \u00a0In the &#8220;Comparison =&#8221; box I can invoke (great word there) the function and tell it to use specific fields as the input variables. \u00a0To demonstrate this I&#8217;ll use the &#8220;Name_2001&#8221; and &#8220;Name_2008&#8221; fields. \u00a0The image below pulls all this together and should do the trick.<\/p>\n<p><em><a href=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/WorkingCode.jpg\" rel=\"attachment wp-att-1262\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-1262\" src=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/WorkingCode.jpg\" alt=\"WorkingCode\" width=\"468\" height=\"566\" srcset=\"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/WorkingCode.jpg 468w, https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/WorkingCode-248x300.jpg 248w\" sizes=\"auto, (max-width: 468px) 100vw, 468px\" \/><\/a><\/em><\/p>\n<p>When I click OK, here&#8217;s a sample of the output I get:<\/p>\n<p><a href=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Output.jpg\" rel=\"attachment wp-att-1269\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-1269\" src=\"https:\/\/d-blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Output.jpg\" alt=\"Output\" width=\"866\" height=\"453\" srcset=\"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Output.jpg 866w, https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Output-300x157.jpg 300w, https:\/\/blogs.lincoln.ac.nz\/gis\/wp-content\/uploads\/sites\/3\/2016\/03\/Output-768x402.jpg 768w\" sizes=\"auto, (max-width: 866px) 100vw, 866px\" \/><\/a><\/p>\n<p>(I&#8217;ve simplified the table to only show the relevant fields by going to the layer Properties &gt; Fields and only turning on the fields I want to display.) \u00a0So any of the records with a 1 should have different values between Name_2001 and Name_2008.<\/p>\n<p>So this is a pretty simple example, but does demonstrate how you can do simple scripting tasks from within the Field Calculator. \u00a0In this one we&#8217;ve just compared two fields but there&#8217;s no reason we couldn&#8217;t consider more comparisons than that. \u00a0With scripting we can make things as complex as we like, so long as things don&#8217;t become too complex and unwieldy. \u00a0Given the generic nature of this function, I can save it\u00a0and rerun it on other fields and can easily use different input fields by making small changes. \u00a0Since the layer contains area measurements, a next step would be to summarise the areas of those polygons that changed, divide by the total area and there&#8217;s your measure of change. \u00a0In all honesty, that&#8217;s something that might be easier to do in Excel, and frankly, the whole thing may have been easier to do in Excel from the start, but I do like a challenge. \u00a0Especially one like this.<\/p>\n<p>What I haven&#8217;t included above is all the trial and error work we went through to get this to work. \u00a0I&#8217;m not a proficient enough user of Python to simply get this right the first time so it&#8217;s <em>always<\/em> trial and error for me. \u00a0ArcGIS attribute tables in and of themselves aren&#8217;t that powerful and not nearly as powerful as an Excel spreadsheet, but hopefully this helps you see that some Python scripting does allow you to wring some additional information out of your tables.<\/p>\n<p>C<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This post details how you can use some Python scripting to do if\/else statements in the Field Calculator. If the title of this post and the intro haven&#8217;t put you to sleep already, read on! \u00a0Perhaps they&#8217;ve even piqued your interest! \u00a0(There are support groups available for this condition.) I was recently contacted by a [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1261","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/1261","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/comments?post=1261"}],"version-history":[{"count":1,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/1261\/revisions"}],"predecessor-version":[{"id":4147,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/posts\/1261\/revisions\/4147"}],"wp:attachment":[{"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/media?parent=1261"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/categories?post=1261"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.lincoln.ac.nz\/gis\/wp-json\/wp\/v2\/tags?post=1261"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}