Posts Tagged ‘digital archives’

Presentations from the successful open consultation day held at TNA on 12 November on digital preservation for local authority archivists are now available on the DPC website – including my report on my Churchill Fellowship research in the US and Australia.  Also featured were colleagues from other local authority services already active in practical digital preservation initiatives – Heather Needham on ingest work at Hampshire, Viv Cothey reporting on his GAIP tool developed for Gloucestershire Archives, and Kevin Bolton on web archiving work at Manchester City. 

Heather and I also reported back on the results of the digital preservation survey of local authorities and a copy of the interim report is also now available on the DPC site.   A paper incorporating the discussion arising from the survey, from the afternoon sessions of the consultation event, will be published in Ariadne in January 2009.

Read Full Post »

I’m quite pleased with the article in the Yorkshire Post today – even the things I didn’t say aren’t too far wrong!  Unfortunately they’ve mixed up the picture on the online version with the wrong caption, although one of my colleagues is captivated by the idea that when you open up your computer you’ll find a cardboard box inside it.  Another tells me that if it wasn’t for Barack Obama, I’d be on the front page… Interestingly, elsewhere, the press is reporting a surge in newspaper sales following Obama’s victory – perhaps we underestimate the public interest in the preservability of a keepsake.

Read Full Post »

Visiting Arizona was a useful way of pulling together many of the strands of what I’ve learnt so far. I was particularly interested in the Persistent Digital Archives and Library System (PeDALS) project, which aims to create an automated workflow for processing digital collections, but also to keep costs as low as possible in an effort to reduce the barriers to addressing the challenges of digital preservation.

The automation aim is of course shared with another of the State Government NDIIPP projects at Washington State Digital Archives, and there are indeed some conceptual similarities in the workflow. However, PeDALS also makes use of a LOCKSS (Lots of Copies Keeps Stuff Safe) private network to provide inexpensive storage with plenty of redundancy and automatic error detection and correction. Having visited the LOCKSS team earlier in my Fellowship, I was curious to see how this system (originally designed to enable libraries to collect and preserve locally materials published on the internet) could be implemented in an archival context.

The envisaged workflow for PeDALS works best when there are clear series of records – in other words, it should work pretty well for government record series, but less well for miscellaneous private and personal accessions.  This is because the system is based upon the application to systematic ‘business rules’ to process large sets of similar records in the most efficient way possible.  This programming work could only be justified where there are sufficient records of a similar type, being created as the result of a routine process.  As has become something of a theme in most of the operational digital archives I have visited, the PeDALS team originally intended to focus on born-digital records but has found that many routine processes are still embedded in a paper system, and hence is currently working primarily with digital records.

The current phase of the collaborative, inter-State NDIIPP PeDALS project is looking at writing these business rules and setting up the PeDALS workflow and storage systems.  Without going into all of this in a tremendous amount of detail (I’d suggest a look at the PeDALS website for further details), the basic idea is to write the rules once and then allow individual participants in the network to tweak them to suit their local circumstances.

Whilst very much in the early stages of building the system, the project is definitely work colleagues in the UK local archives network keeping an eye on – not least because of the emphasis on keeping costs down.  As well as the main project website, there is an update log at https://pedals.updatelog.com/login (you need to register for a username and password).

Read Full Post »

This was my most challenging (in a thought-provoking way) visit so far. The Washington State (upper left hand corner of the US, for those whose geography is as hazy as mine was!!) Digital Archives doesn’t seem to be terribly well known in the UK, and I’d certainly recommend colleagues have a look at their website, particularly some of the background documents in the About Us section. The Center for Technology in Government’s case study on the public value of returns to government resulting from the Digital Archives investment (available at http://www.ctg.albany.edu/publications/reports/proi_case_washington?chapter=1) is also well worth reading.

Why was the visit challenging? Well, essentially because this digital repository has been largely conceived and operated as an IT development project, and more recently as a business service and disaster recovery facility for creating agencies and departments within state and local government in Washington (and in this sense has certain parallels with TNA’s Digital Continuity Project). Microsoft, being in Washington’s backyard in Seattle, also have a not inconsiderable influence, and the Digital Archives staff have a strong working relationship and level of support from Microsoft.

Quite a contrast, then, from the Australian operational repositories, whose workflows are firmly rooted in archival and recordkeeping paradigms, often with a strong commitment to the use of open source software and XML open standards.

Initially, I found this approach very difficult to grasp, and indeed the staff at the State Archives freely admit that future developments of the Digital Archives will require a greater degree of partnership between archival staff and technologists. As I learnt more about the detail of the Digital Archives operation, however, I began to see both parallels with other digital archive operations (for instance, in maintaining authenticity and safe transfer of custody of files by means of sealed hard drives and secure FTP transfer) and ways in which the greater level of IT input into this Digital Archives has enabled extremely high levels of automation and efficiency in processing and searching.

The current run rate for ingest of single page TIFF images is over a million a day; use of the website (boosted by the decision to concentrate initially on the ingest of digitised birth, marriage and death records) runs at a level of around fifty to sixty thousand uses a day.

I still struggle, from a conceptual archival point of view, from the way in which different record series are merged together for access, and would hope to see a greater degree of contextual information in series descriptions into the future, although I can understand the processing efficiencies gained through only having to manage the one large database. The approach really makes you think about which of your archival assumptions are vital theoretical foundations for facilitating secondary use of archival resources, and which are merely legacies of a paper world.

Washington will also be of interest to those colleagues who would like to see regional partnerships of digital archives develop in the UK. Washington is leading one of the current round of NDIIPP projects to develop a centralized multi-state digital preservation consortium so that other States in the US can benefit from the expertise and workflows developed in Washington. Further details are available from the project website.

Read Full Post »

Operating the Digital Archive

As previously posted, the operation of the PROV Digital Archive is well integrated into the wider organisation, with the same team responsible for transfers of both paper and digital records. This team also creates the disposal authorities (more commonly known as ‘retention schedules’ in the UK – is the different terminology significant??!) for all Agencies within the State of Victoria.

Digital records are only accepted into the Archive if they are VERS compliant, and the Agency’s recordkeeping system can produce VEOS according to the standard mandated under the Victorian (as in ‘State of…’) Public Records Act.

This is obviously a strong advantage for PROV, and not a requirement which can easily be translated into the UK local authority archives context. However it is worth noting that despite the relative strength of their archival legislation, PROV staff still commit considerable effort into consulting with Agencies and carrying out pilot transfers. The team at PROV have noticed that it is harder to encourage deposit in a digital world, whereas historically a lack of physical space for keeping records often triggered transfers to the archives. Whereas traditionally the transfer process was client driven, commencing with an Agency request, PROV are now trying to move towards a programmed transfer timetable for both paper and digital records. PROV are trying to sell this to the Agencies as being cheaper and easier than ad hoc clear-outs of records.

There are in any case many similarities in dealing with transfers of records to the archives whatever the format of the records. PROV needs to maintain intellectual control over the records series, and descriptive lists need to be produced. Background information on provenance and access arrangements or restrictions is gathered prior to transfer by PROV staff through site visits or, increasingly, formalised documentation. The Agency staff are responsible for producing a ‘manifest’ listing the records being transferred. PROV provides advice and training on the process of preparing digital records for transfer, and transfer guidelines are published on the PROV website. Digital archives may be transferred on CD, hard drive or copied remotely into the Digital Archive inbox (though few Agencies have yet taken advantage of this method of transfer, preferring to follow the paper paradigm and copy records onto CD much as they would package paper records into boxes).

The system of intellectual control (assigning of unique identifiers etc.) for digital archives follows much the same pattern as for paper records. My feeling is that Australian practice in the use of consignments and the series system makes this simpler to implement than with the UK practice using accession numbers and hierarchical cataloguing, although clearly we in the UK need to take some time, as did PROV with the revision of their Archival Control Model, to consider how to integrate digital archives into key archival processes.

Where do PROV themselves hope to see improvements? Dealing with digital has highlighted an internal need for improved written procedures for dealing with transfers, whether in paper or digital formats. New staff need to be trained to operate the Digital Archives interface (a heavily customised version of Documentum). Improved guidelines are also needed to help Agencies, and in particular Agency IT staff who are most likely not familiar with archival practices and terminology. One of the technical support staff at PROV pointed out that ‘file’ in IT terms has potentially a completely different meaning to the archival ‘file’. Language needs to be translated into terms which Agency staff are familiar.

Once the digital records arrive at PROV, the manifest is loaded into the Digital Archive system and checked against the records actually received. The records are checked to ensure that they are valid VEOs and that they are virus-free. Various errors can be picked up at this stage – duplicate records, extra records received or too few, problems with the digital signature etc. Simple errors can be fixed by PROV staff, but in general it has been found best to request the Agency to resubmit the whole transfer. The records remain in ‘quarantine’ for seven days, before the checking process is re-run. If successful, the transfer can be finalised and the records become viewable through the PROV online catalogue.

The first pilot transfers to the Digital Archive took place in 2005. The largest accession so far has in fact been digital surrogates from PROV’s own digitisation programme, although another major and ongoing project is the archives of the Melbourne 2006 Commonwealth Games. This has brought its own unique challenges in working with a project organisation in the process of being wound down (for example, password protected records which cannot be processed into VEOs have had to be ignored).

Read Full Post »

Archival Support Programme

There are an estimated six to seven hundred places in the State of Victoria which hold archive collections, about 120 of which are recognised as Places of Deposit by PROV. PODs in this Australian context are “community facilities that meet the storage standards required by PROV to preserve records of significance to local communities”.

The Archival Support Programme started around ten years ago, originally as a small grants programme for archival supplies, and is run in collaboration with the Australian Society of Archivists and the National Archives of Australia. The programme takes the form of a travelling roadshow, with around four seminar topics presented each year.

This year’s programme included a roadshow seminar on “Computers and Small Archives”. This covered the basics of digitisation, designing online exhibitions, and using a computer to catalogue archival records, all focused on the kinds of practical situations likely to arise in a community archive setting. The seminar also included a session on digital preservation issues. This outlines the preservation issues of obsolescence and poor management, and encourages communities to adopt good practice in selecting appropriate long-term preservation formats, to copy media regularly, and to take care with storage conditions and handling, to take periodic backups, and to ensure documentation about the archives themselves, if maintained on a computer, can itself be exported and preserved over time. The central message is the need actively to manage digital information to ensure its continued accessibility.

The messages conveyed in this digital preservation talk are similar to those I incorporate in a WYAS presentation aimed at local Family History Societies. However, the emphasis in the PROV session on the various simple, yet effective, solutions which might be employed is striking, and is something which I will incorportate in future versions of the WYAS presentation.

Read Full Post »

« Newer Posts