A while back, particularly with the introduction of Microsoft MapPoint, it was commonly believed that the need for GIS specialists would diminish, and a GIS package would be used just like an office product. Pre-configured datasets would be readily available, and products would come integrated with mapping and basic spatial analysis. In a way, that has happened to a certain extent, but the need for specialist analysts has not waned as was feared. In fact, this insurgence of easy to use GIS products has actually had the opposite effect, and put GIS into the hands of more people. Is this happening too fast.....or are new GIS users not being fully prepared for the task at hand?

Over the past 12 months, with the startup of our new company, I have had the fortune of "getting dirty" with GIS again, and do some serious spatial analysis (something that had been sidelined for the past few years). Working with and analysing data you immediately identify inconsistencies and flaws in the data structure, and the data I have been working with recently has not been in the best state. I am also not convinced this is an entirely South African problem, as I have seen similar problems in other datasets from around the globe. Nowadays, most data has been vectorised into a useable GIS dataset, however, there will always be certain datasets that require continuous maintenance - Cadastral Parcels, Roads, Zoning etc. Are the people responsible for maintaining these datasets qualified to adequately maintain them? Additionally, I have noticed that planning organisations are utilising GIS more and more, and moving away from their traditional CAD background. With this comes the delineation of development corridors and new town planning schemes. Are these also being demarcated by qualified professionals? In most cases not, GIS is simply being used to satisfy the needs of a map in a report without considering the need for analysis on this data in the future In the South African context, we have been involved extensively in the establishment and training of GIS units in municipalities across the country. I can reliably say that the GIS users appointed to these positions do not have the skill to adequately manage such a dataset (and any others - i.e. water and sewer reticulation).

So why is this happening? I believe the fundamental problem, is that many new GIS users are simply picking up a package, much like they would learn Excel or word - through trial and error. This is in some ways a commendation to the GIS vendors, in providing packages that are nowadays intuitive and easy to learn. But, what is the problem with this? For starters these users are doing GIS with no real background into what GIS is, thereby not getting to grips with these critical items:

1. The importance of topology and understanding vector data
2. Spatial accuracy and appropriate digitisation scale
3. Metadata
4. Understanding of projections

Projections

Understanding of projections is probably the most important factor that is resulting in dirty data. GIS packages today place less emphasis on the importance of understanding the different projections available. What this does, is enable users to work with data without really understanding the data projection. This however negatively impacts on the resultant datasets, and we see more and more datasets with "data shifts". This is directly related to situations where tow separate datasets with differing projections being combined. In the South African context, this is particularly true, since we have recently (+- 5 years ago) moved away from the traditional Cape datum to a modified WGS84 datum called "hartebeesthoek94". Look at the example below:

Notice how the red polygon has clearly been captured using a different projection to the rest of the data, resulting in a noticeable data shift.

Topology and understanding of vector data

Understanding the makeup of a vector feature is also critical in the provision of reliable GIS data. What does this mean? Using shared boundaries on polygon features, avoiding dangling nodes on line features (remember drawe node dangle !!), segmentation of line features, closing polygon features...and so the list goes on. For all of these items, maps can continue to be produced to satisfy a report, but, as soon as you need to go further than that, problems arise. Another example shows the cadastral parcels for a large city in South Africa (one of the capital cities). Here you can clearly see how polygons are left open, resulting in errant parcels. In this example, I have selected one polygon which bleeds out across multiple parcels and road reserves (I have not indicated the full extent of this selected polygon in this example either!)

In the next example, notice how the selected (in yellow) road segment encompasses multiple segments, again indicating poor topology. This is a dataset from a provincial (same as state) transport department:

Other problems I have encountered include the creation of multiple single shapefiles to define different criteria, which could as easily be defined in the attribute data of a single shapefile. This makes manageability more difficult, and highlights a lack of understanding of GIS concepts.

Spatial accuracy and Metadata

I do not have any examples immediately at hand, but the point I was trying to stress is that in many cases data is collected at the wrong scale and then used for analysis at a much smaller scale. I have seen real world examples of this, indicating again a lack of understanding of basic GIS concepts. Metadata itself is virtually non-existent today, making using data fro analysis extremely difficult.

Conclusion

Is this a problem? Of course it is - government departments and municipalities are making "informed" decisions using this data without even knowing the underlying problems exist. What makes it worse is that in many situations we have raised these problems with the data, only to be frowned upon for criticising the data! Well...at least it keeps us in a job as someone has to sort the data out sometime!

I am also interested to know if this is a world wide phenomena - as I said I am guessing it is based upon my experience with other datasets, and the fact that I have been able to download tools to do things such as segmentise a road, shift polygons etc