The challenge of ephemeral data

summary

Many of the resources in the BTAA Geoportal are from sites that continually update their datasets. As a result, we need to regularly re-harvest the metadata.

Government agencies now issue most geospatial information as digital data instead of as physical maps. However, many academic libraries have not yet expanded their collection scopes to include publicly available digital data, and are therefore no longer systematically capturing and storing the changing geospatial landscape for future researchers.

The BTAA Geoportal partially fills a gap in the geospatial data ecosystem by cataloging metadata records for current publicly available state, county, and municipal geospatial resources. The value of this data is high, as researchers routinely use it to form the base layers for web maps or geographic analysis. However, the the mandates and policies for providing this data varies considerably from state to state and from county to county. The lack of consistent policies at this level of government means that this data can be considered ephemeral, as providers regularly migrate, update, delete, and re-publish data without saving previous versions and without notification to the public.

The lack of standard policies at this level of government means that this data can be considered ephemeral. It may be updated, removed, or replaced without notification to the public. The rate at which datasets change or disappear is variable, but is often high.

This continual turnover creates a difficult environment for researchers to properly source data and replicate results. It also requires a great deal of dedicated labor to maintain the correct access links in the geoportal. As the geoportal’s collection grows, the labor required to maintain it grows as well.