Tag Archives: records management

How it could work…

The following workflows show how the open source tools and preservation techniques that the project has tested and explored could fit into a wider process for the effective management of electronic records.

The approach relies on:

  • adapting the available functionality of a well-managed file servers system
  • training and relationship building with relevant staff by the records manager
  • the identification of sets or classes of electronic records held on shared drives that need to be captured and managed in a structured way
  • a commitment to transform use of the ‘shared drive’ from a long-term, unstructured collection of documents of varying relevance and quality to a temporary, collaborative area with strict controls on size and data retention

 

 

Click on the image above to see it full size.

I’ve included the full document here, which has several other ‘scenarios’ for managing electronic records using these tools: Records management workflows

‘The DROID we are looking for?’ – Using the open source tools

One of the objectives of this project is to examine how useful and practical the Open Source preservation tools are.

I thought that it would be worth recording my experience in installing and using the software in order to carry out the tests for the project. I’ve heard so many people over the years, from my mother to system administrators, use the phrase ‘I’m not very technical or IT-savvy’ so I won’t say that. I will be honest and confess I am just ‘IT-savvy’ enough to embark on something like this but not savvy enough to master it correctly. I think many of us working in records management occupy this sort of space but it shows that none of the following should be attempted without the input and guidance of your IT department.

Deciding to use open source tools to support records management is a different approach for a records manager. We are used to stages such as building a business case, procuring a system and implementing it. Open source tools are a lot simpler. Find it on SourceForge and – if your IT department says it’s OK – click download and there it is on your desktop.

DROID

According to its website DROID (Digital Record Object Identification) is “a software tool developed by The National Archives to perform automated batch identification of file formats”.

I found this downloading this tool very simple to do and it is very intuitive to use.

The functionality of DROID that stood out for me was the analysis and reporting that it could produce. Fundamental to my conception of the business case for electronic records management is the amount spent on storage and back-up of large, unstructured shared drives. Whilst I had used simple ‘search’ analysis in Windows Explorer to assess shared drives, the DROID tool did this reporting quicker and was able to produce it as a CSV download that allowed for analysis. The possibility for producing metrics to support a business case is therefore much enhanced with DROID.

This also provided me with the idea of using DROID to run regular reports on a structured, organised File Servers repository. One of the challenges of a ‘file servers/existing tools’ approach to electronic records management is the analysis and reporting that can be produced. DROID allows a simple manual process of taking a few minutes at most that could provide a regular ‘snapshot’ to map the development, growth and reduction of an electronic repository over time. I have demonstrated how this might work in scenarios 2 and 3 of my project output ‘Records management workflows’.

DROID therefore was an immediate help and has the potential to be a valuable tool for records managers.

XENA
According to its website, http://xena.sourceforge.net/index.php XENA (Xml Electronic Normalising for Archives) is a tool intended to “aid in the long term preservation of digital records”. The tool was developed by the National Archives of Australia.

XENA needed a few other things installed on my PC before I could get it up and running. Therefore I had to enlist the help of my IT support people to install OpenOffice and upgrade my JAVA settings accordingly. 

Once it was installed, XENA was quite easy to use. It follows the OAIS model by converting the document into a preservable digital object. You can view some of the content through a ‘XENA viewer’, but to make it properly available you can publish or ‘normalise’ the document, usually as an Open Office document or PDF.  (Further details on the outcome of our testing are available elsewhere on the project blog).

The concept of a safely preserved set of documents of which a copy can be created when required is quite a departure from the idea of a EDRMS master repository of documents which a user can search. Could a user – a member of staff or a member of the public – search a catalogue of document titles and keywords rather than the documents themselves? This might be impractical for ‘short retention /regularly accessed’ types of records. But it could be ideal for permanent retention documents (such as governance minutes, regulations, strategies) that cover both bases as valuable administrative records and documents of archival or public interest.

Challenges / Risks

1) There is an assumption that XENA and DROID will continue to be supported and relevant in years to come. That they are products of National Archives programmes does suggest a longevity that means the software is worth using for long-term preservation.  Would the Open Office output of XENA run more of a risk of obsolescence than the XENA objects?

2) The use of these tools requires a lot of manual work for the records manager – running reports, converting documents etc. – which can be automated by the use of an EDRMS system. However, you could argue that this sort of work was a key part of past – and still persisting – paper records management processes. Additionally, an EDRMS brings, alongside significant costs, a host of system administrator and training overheads which would be sidestepped by this sort of approach.

Conclusion

Both these open source tools, which are free to download and took little longer than 30 minutes to get up and running, provide some really interesting and valuable functionality for records managers. Converting them from a desktop ‘toy’ to a scalable working component in a records management system or process is a much more difficult step, but my testing of DROID and XENA has certainly given me plenty to think about.

DIPs from a Xena object – descriptive metadata

In OAIS terms, the Xena digital object is our Archival Information Package (AIP). This is the object that would be stored and preserved in UoL’s archival storage system (if we had one).

For the purposes of this project, the records manager needs to be assured we can render and deliver a readable, authentic version of the record from the Xena object. In OAIS terms, this could be seen as a Dissemination Information Package (DIP) derived from an AIP. Among other things, the DIP package ought to include sufficient descriptive metadata.

We’ve defined a minimal set of what we would need for records management purposes (names, dates for authenticity) and for information retrieval, and criteria for assessing overall success of the transformations – such as legibility, presentation, look and feel, and basic functionality. See our previous post for the documentation of how we arrived at our criteria.

For most of the examples below we are looking at standard Xena normalisations, which can produce a DIP by the following methods:

  • Export the AIP to nearest Open Office equivalent
  • Export the AIP to native format
  • Export the AIP to a format of our choice (an option we haven’t tried as yet)

Emails are slightly more complex – see below for more detail, and previous post on emails.

Documents, spreadsheets and powerpoints

For these MS Office documents, Xena normalises by converting them to their nearest Open Office equivalent. We can view this Open Office version via the Xena Viewer simply by clicking on Show in OpenOffice.org.

In the case of a sample docx, this shows us the document with its original formatting intact. In Open Office, we can now look at File, Properties, and confirm the following descriptive metadata are intact (some of these we will revisit when we look at significant properties):

  • Title
  • Subject
  • Keywords
  • Comments
  • Company Name
  • Number of pages
  • Number of tables
  • Number of graphics
  • Number of OLE objects
  • Number of paragraphs
  • Number of words
  • Number of characters
  • Number of lines
  • Size
  • Date of creation / created by who
  • Date of modification / modified by who

One missing property is a discrete ‘Author’ field in the OO version.

PDFs

Xena Viewer has its own Adobe-like viewer for PDFs. When we look at the Document Properties for a normalised PDF, we get the following descriptive metadata:

  • File size
  • Page count
  • Title
  • Author
  • Creator (i.e. software that created the file)
  • Producer (i.e. software that created the file)
  • Date of creation
  • Date of modification

Images

We may not need much descriptive metadata for individual images. We looked at the converted images using a free EXIF viewer tool and the date values (creation and modification) are intact in the normalised object.

Emails

For emails, the AIP will either be a standard Xena normalised object or a binary Xena object. Broadly, the former can export to an XML version and the latter can export back to its native format (.msg file). We’re always guaranteed to get a minimum of descriptive metadata whichever transformation we enact. This would be:

  • Title
  • Author
  • Recipient
  • Date

Results – progress so far

Below is a PDF of my table of analysis of descriptive metadata (and significant properties).

We also have Kit’s assessment of the transformations, including his comments on whether a reasonable amount of metadata is intact.

Adapting OAIS for records management

In planning this project we’re obviously referencing the OAIS standard for a digital archive.

To quote from Wikipedia, ‘Open Archival Information System (or OAIS) is an archive, consisting of an organization of people and systems, that has accepted the responsibility to preserve information and make it available for a Designated Community’.

The OAIS approach reminds me of the old style paper records store method for managing records. We (the records manager) receive the closed files and store them safely and securely. When a record is required we are asked by the department for a specific box or file. We then make it available.

This is being replicated in several ways in current records management approaches. At one conference I heard of one organisation moving its paper files to offsite storage, then providing a ‘scan on demand’ approach when a file was requested. The offsite store would provide a scanned copy and email it to the requesting department. I thought this was a nice way of sidestepping a massive digitisation project – a digital ‘copy’ is provided when required, the mass of original paper retained in a low-cost storage environment.

In-house electronic document management solutions, from a simple intranet to a full blown EDRMS, have often relied on the information being available instantly via a search.  That obviously relies on the quality of the stored records and speed / accuracy of the search engine.

Could the OAIS approach work in an internal business context? Can the electronic records be submitted or ‘declared’ to the records management team and stored. The records with long term retention converted into preservable formats. On demand, the records can be made available through a print to PDF facility. The audit trail of access is inherent in the very record produced.

There are several visual depictions of the standard on produced on page 4-1 on http://public.ccsds.org/publications/archive/650x0b1.pdf, to which I add my ‘scrawled on the back of a napkin’ version to explain how this approach might work:

The benefit, you’d hope, would be ‘future proofed’ electronic documents managed and made available at a reasonable cost over decades. The drawbacks would, perhaps be in the user experience where people are used to the instant search return of a Google query. Now the user would effectively be searching a catalogue of record references and requesting access to the one they want.

The challenge is: would the approach actually work? That is one of the things we are trying to explore with this project.

What and why…

Most of the information now created in the University is ‘born digital’. Staff create documents in file formats that have much shorter life spans than paper. We need to mitigate the risk that these files become unreadable.

Buying an EDRMS is a huge cost for an organisation in terms of licences, implementation and training. Is there a way that electronic records can be effectively managed and preserved without this cost?

We thought: “Can we use our existing infrastructure, with some open source tools, to build a practical, cost-effective solution to the long-term management of our key electronic records?”

This project will build a test environment for converting files into preservable formats. Will they be fit-for-purpose as university records?

The aim of the project is to identify the opportunities and challenges to this approach to electronic records management. Is it a viable alternative? Is it practical for records managers?

Kicking off…

My name is Kit Good, University Records Manager and Freedom of Information Officer at the University of London. I’m lucky enough to work in an institution where the technology department, the University of London Computer Centre (ULCC), has a dedicated Digital Archiving team with a lot of experience in delivering preservation projects around electronic records.

 I’d met with Ed Pinsent, Digital Archivist, several times since I started last year to discuss the challenges around electronic records management. It was back at the start of September when Ed approached me about submitting a bid to the JISC 12/11 Digital Infrastructure Programme.

Ed and Kit

Our bid proposed testing the concept of ‘a simple toolkit of services and software that can plug into a network drive and create preservation copies of core business documents that require permanent preservation’. We were delighted to find out earlier this month that our bid was successful. 

We are hoping that the outcomes of this project will be useful to records managers, the digital archivist community and the Higher Education Sector as a whole. This blog will track the development of the project from its November kick-off to its February close.

More posts to follow…