Degrees of Difficulty
To work through a mapping project a bit of detective work was needed to get data in the right place.
GIS Central has recently been involved with some global mapping as part of an Our Land and Water project. This has involved collating a wide range of environmental data covering soils, hydrology, vegetation, terrain and a range of others (20 variables in all) on a global scale. I’ll admit that this hasn’t been easy in many respects – there are lots of data available but they’re not always in an easy format to work with, or are difficult and time-consuming to get on to a map (Ed. Oh no – does that mean more blog posts about this…?). Perhaps the easiest part of this is that my colleague provided me with a set of points (n = 13,353 from around 20 different countries) to use with this – data from the global datasets are to be attached to each of these points and then used in some statistical modelling. So I thought that part was the least of my worries. Turned out not to be entirely true, especially when it came to the US.
Data arrived in my inbox in the form of an Excel spreadsheet and looked a bit like this:

Oh, what fun…! For mapping purposes you can see I’ve got “Lat” and “Long” columns, coordinates for latitude and longitude that allow each point to be mapped, so these are absolutely critical. (Notice, too, that each has a Country and a Reference entry.) First task is to get these on the map.
Just like any other layer of data, I can add the specific sheet I need from the spreadsheet using the Add Data button. It appears in my table of contents at the bottom of the stack (“Filtered data$”).

(From Add Data you could also use XY Point Data.)
From here I can right-click the table name and go to Display X Y Data:

- For this to work I need to know which fields have my X (longitude) and Y coordinates (latitude)
- I also need to know what coordinate system that data are in
- When using Latitude and Longitude, the units need to be in decimal degrees rather than degrees minutes and seconds
That last comment needs a bit more detail as it will become more important later. Latitude and Longitude are often shown in the form of degrees, minutes and seconds (D:M:S) as you might see on your GPS receiver. For instance, Mrs O’s has lat/long coordinates that are approximately:
- Latitude: 430 38′ 35″ S
- Longitude: 1720 28′ 00″ E
For latitude, that would be read as 42 degrees 38 minutes 35 seconds South (of the equator), based on the idea that degrees can be broken down into minutes (60 in a degree) and minutes into seconds (60 in a minute). All well and good but most software can’t easily read D:M:S. A much more straightforward way for computers is decimal degrees. For our latitude we can get from D:M:S to decimal degrees in the following way:
42 + 38/60 + 35/3600 (there being 3600 seconds in a degree) = 42.64305.
Since we’re south of the equator, we put a negative sign in front. In decimal degrees, Mrs O’s location can also be shown decimal degrees as:
- Latitude: -43.64305
- Longitude: 172.46680 (no negative sign since we’re east of the Prime Meridian)
This isn’t quite the same thing as being in two places at once, but it’s close.
For the coordinate system, I know I’m working with data from around the globe so a good choice is WGS84, the system that GPS uses. It’s a good, safe bet but bear in mind that some countries (e.g. the US) have their own geographic coordinate systems (e.g. NAD83) that work nicely in their part of the world – the same coordinates may actually end up in different places, but probably not by much.
So, setting up the tool and clicking OK gets me all the points, cobbled together from various data sources:

So far so good…until…doing a bit of quality assurance, I found some issues. This QA process mainly involved looking at all the points and ensuring that they were in the country they said they were from and not out at sea or had migrated across national borders.
Most countries looked fine but I noticed some issues in the US (Ed. is that any surprise these days?), specifically, points at sea, and a fair number of them:

This is a problem. Have the data been somehow shifted too far east? Are they in a different coordinate system? Shifting over to the west coast, no points off shore but they do seem to be shifted inshore possibly?

Canada and Mexico look to be well behaved but I remain suspicious of these US data. But at this point I’ve really got nothing to go on. A bit more digging is called for.
As we saw above, my colleague has conscientiously included a Reference column for each data point, so when I selected all the US points, I saw that they all come from the same source – a 2022 publication with an associated dataset. Digging even further, I found that some of the data in that dataset originated from the US Geological Survey, particularly a 2018 report that had some maps in it that I could compare against the mapped points. Looking at my data versus the report maps (on the left below), there’s very clearly an issue and it’s not the wrong coordinate system (I had already tried that). My points are just in the wrong place:

(The maps look slightly different due to different projections.) By the looks of it, I’ve got a national scale problem and If I want to get things right, I’ve really got no option but to go back to the raw data and see where things may have gone wrong.
What we’re dealing with here is data quality, and locational accuracy in particular. If I hadn’t gone in to this level of detail, there are several thousands of points that may get the wrong data attached to them, which will flow through to the final analysis. In all good conscience, I can’t let that happen if I know it’s a problem.
But that’s a job for another day and another post – bet you can’t wait to see how this gets resolved (Ed. Be still my beating heart…I can barely contain myself…[yawn]).
C
That’s quite the dramatic cliff-hanger ending! Good luck getting this resolved.
Are you planning a trilogy here or aiming for an open ended franchise? This is like Rogue One where our hero struggles and it ends with a grim situation. I’m hoping the next post will be Crile Doscher’s Data Wars: A New Hope. 😉
I promise a satisfying ending, Jon, perhaps a bit better than for Jyn and Cassian. With GIS there’s always hope – one just has to be ready when The ESRI Strikes Back.
Very nice blog post. I definitely love this site. Stick with it!
This was beautiful Admin. Thank you for your reflections.
Good post! We will be linking to this particularly great post on our site. Keep up the great writing
I very delighted to find this internet site on bing just what I was searching for as well saved to fav Watch bbc farsi
I m often to blogging and i really appreciate your content. The article has actually peaks my interest.
Degrees of Difficulty – The GIS Blog
azvyxkrdliq
[url=http://www.g642b9h4i877ce0350p1ghf546jjxcocs.org/]uzvyxkrdliq[/url]
zvyxkrdliq http://www.g642b9h4i877ce0350p1ghf546jjxcocs.org/
Butyl Acetate
5 Acetic Acid
http://www.adentech.com.tr
Stainless Steel Hexagon Socket Head Half Thread Cap Screws with Reduced Shank
Cross Hexagon Head Machine Screws
Methyl Acetate
M6 M8 M10 Stainless Steel Machine Lag Eye Screws Eye Hooks
Hot Dipped Galvanized Unistrut Stainless Steel Strut Channel 3 C Channel Steel
borisevo.myjino.ru
Solar Power Mounting Galvanised Channel C Shaped Channel
Ev Home Charger Manufacturers
Ev Fast Charger Manufacturers
Galvanized U Beam Steel Z Channel Structural Steel T Channel Solar C Shaped Channel
Ev Charging Station Suppliers
For the reason that the admin of this site is working no uncertainty very quickly it will be renowned due to its quality contents.
Retro Polygon Sunglasses
Drawstring Bags 15 Liter Waterproof Interior with Double Layered Rope Pocket
Industrial Grade Sodium Carbonate Dense Soda Ash
curved glass protector
http://www.bwheater.co.kr
Working Principle and Application Scenarios of Marine Cranes
Womens metal frame sunglasses
How to Select a LED Warning Lights
http://www.viktoriamebel.by
High speed beef jerky packaging machine
What are the advantages of Ball Joint
Peacock Color Diamond Cat Eye Nail Gel Polish
Beef jerky packaging equipment.
Quality multihead weigher packaging machine
H340LAD+ZF Color Coated Galvanized Steel Coil
mck-web.co.jp
H260PD+Z Color Coated Galvanized Steel Coil
Air Flow Dryer
Pellets For Pellet Stove
Biomass Pellet Machine
BUFDE+Z Color Coated Galvanized Steel Coil
Flyknit Breathable Running Shoes
Cheap Electric Golf Carts
Running Shoes
Zinc Alloy Straight Microduct Connector
Fiber Optic Adapter Round Microduct Straight Connectors
http://www.kotech.co.jp
HDPE Microduct Straight Optic Connector
E5061B ENA Vector Network Analyzers
High-Speed Power Bank 10000mAh 22.5W
setsatian.ac.th
Power Bank 10000mAh 22.5W With Touch Switch
USB C Mobile Phone Charger
E5080B ENA Vector Network Analyzers
E5063A ENA Vector Network Analyzers
Also I ve shared your site in my social networks!
Flatware Set
20 Pieces Flatware For 4
mxixray.com
Cake Spoons
Solar Bracket Mounting System for Tiled Roof Solar Panel Mounts
Solar Panel Mounts for Metal Roof Solar Mounting System
Solar Panel Roof Mounts Accessories for Tile Roof Solar Mount System
http://www.market.hexaxis.ru
Car Air Diffuser
Glass Bottle Cosmetic
Car Vent Diffuser
Custom Lotion Bottle
Hand Cream Tube
Car Aroma Diffuser
Lathe Machining Parts
Sand Casting Gray Iron Foundry
What are metal mesh belt made of
What is a label sticker
Cotton Hiking Socks
Stainless Steel Coupling Casting
dblink.co.th
Swing Solid MSQB Hydraulic Table Cylinder
Laser Tattoo Removal Gun
Ultrasonic Cavitation Handheld Device
gaucbc.org
Low Profile Cylinders Air Gripper Slide Table Cylinder
Solenoid Valve Airtac DC 24 two Position three Way Air Pneumatic
Co2 Fractional Laser Medical Machine
Contact Ball Bearing
Sheet Metal Laser Cutting Parts
Zinc Die Casting Components
http://www.mcityband.ru
Combined radial/axial bearings
Zinc Alloy Die Casting Spare Parts
1622-2rs Bearing
Small Laser Marking Machine Price
Graphite Pump Packing
http://www.modan1.app
50w Fibre Laser
Braided Graphite Packing
Laser Cut Head
Graphite Valve Packing
Granite Sculpture
dblink.co.th
Marble Figure
Roman Soldier Statue
White Anti-slip Glow Tape
Waterproof Anti-slip PEVA/PU Tape
Heavy Duty Anti-slip Glow Tape
Neodymium Magnets Arc
White Double Sided Carpet Cloth Duct Tape
3inch Waterproof Double Sided Carpet Cloth Duct Tape
Flat Neodymium Magnets
Multicolor Double Sided Carpet Cloth Duct Tape
Cylinder Neodymium Magnets
http://www.nunotani.co.jp