Enhance automation in administrative workflows for data and services

From GRDI2020

This is a GRDI recommendation; return to Main Page with all the challenges and recommendations This recommendation is cited by Data Curation and Preservation.

Context and Challenges

Data management involves administrative workflows <ref>a reference model to such workflows is provided by the OAIS ISO standard. Within the data infrastructure, data infrastructure managers have an inherent self-interest for ensuring efficiency in these administrative workflows through, for example, structural adaptations and increasing automation. However, at the interface between the data infrastructure and application domains, administrative workflows run the risk of becoming inefficient because roles and responsibilities are not clearly assigned. At the same time, the efficiency of both ingest and access workflows are important for (a) user acceptance, (b) quality of the data ingested or disseminated, and (c) the reliability of the overall system.

In the context of the OAIS there have been various analyses of ingest workflows (e.g. by nestor), and tools are being created to facilitate format validation and metadata extraction. However, there is often still a lack of integration with user environments; for example, automatic metadata creation from the context of user environments or adaptation of the dissemination format to cater to the requirements of user environments are often still lacking.

Recommendation

Enhance automation in administrative workflows for data and services. This applies to all OAIS workflows, and particularly to ingest and access workflows, which include multiple spheres of responsibilities.

This may include

• (semi-)automatic extraction of metadata from source data
• using external reference works (e.g. dictionaries, thesauri) to enrich content
• enable automatic conversion paths for converting to preservation formats as well as embedding formats into application environments (cf. Apache Cocoon, Fedora disseminators)
• adaptivity of administrative archive workflows to distinct application environments

Stakeholders

This recommendation is closely related to the simple API recommendation, yet aims to enhance integration between the application environment and the infrastructure beyond the interface. As such, it is relevant to Data Curation and Preservation for its opportunities in enhancing quality and reliability in application/archive communication, as well as to "Data Use - Virtual Research Environments" for embedding into application environments.

The challenges described here require some technical engineering with regard to automation, but equally require constant adaptation to distinct application environments. As such, the Translator Role may be required to make this work.