Author Archives: oliveiradaniel

European Commission provisions on Electronic Documents

By Alfonso Casanueva, from Spain, former Policy Officer at the European Commission’s Secretariat-General, Document Management Policy unit, Belgium

The European Commission started a records management project called e-Domec (electronic document management in European Commission) in 2002, with the aim to rationalise the production and management of records, and to adapt it to the electronic environment, in the line of similar initiatives of e-government carried out in some European countries.

In this framework, several provisions were adopted, setting out the basic rules that all Commission officials should respect when dealing with their records. One of this provisions is the Commission Decision 2004/563/CE, EURATOM, on electronic and digitised documents, which was completed with the Implementing Rules SEC (2005) 1578.

With these provisions, the Commission aims to set the conditions for the validity and the legal value of electronic records, whether they are born digitally (Word documents, e-mails…) or scanned, and also ensure the authenticity, integrity and readability thereof, and the preservation of the records and their metadata during the whole lifecycle of the document. The link between these two texts and the rest of the provisions on records management written for the e-Domec project is obvious, as it is in these other texts where the lifecycle of the document is defined and where the metadata describing the document are set. It is important to mention that according to Commission Provisions, lifecycle and metadata for records are independent of the support of the record, paper or electronic, therefore the Decision 2004/563 does not cover these questions.

We will see how this text deals with the management of electronic records, not being very audacious in pushing towards the electronic environment, but taking, on the contrary, a very safe approach.

The provisions are organised, as usually, in two different texts, each of one of a different level of importance. On the top, we have the Commission Decision 2004/563, which sets the basic principles to be followed and it was approved by the College of Commissioners. Then, we have the Implementing Rules SEC (2005) 1578 that details those principles. The second one is a text approved by the Secretary General of the Commission. Both texts cover four basic questions regarding electronic documents:

Their validity

Their transmission

Their preservation

Their security.

Validity of electronic documents:

Regarding this question, the Commission distinguishes on one side, the validity of the document itself, and on the other side, the validity of an electronic process that may lead to the production of an electronic document (in other words: an electronic workflow)

For the first question, the legal text looks into the question of the unambiguous identification of the author. A precondition is established: if there is, somewhere, any regulation, any legal basis that requests a hand written signature on the (paper) document for its validity, like it is the case of many contracts, for instance then an electronic document would be valid only if it is born digital and signed with an advanced electronic signature. If there is not a legal text demanding for a handwritten signature as a condition for the validity of a document, scanned versions or electronic born documents are considered legal provided that their author is identified unambiguously.

Regarding the validity of electronic workflows, the texts request that every actor and the steps they perform are duly identified by the workflow system. Then, a difference is made among those workflows completely internal to the Commission and those where external actors may take part; for the internal ones, it is just said that the Commission Directorate for IT will ensure that the systems fulfil that conditions; for the second type, the workflow system shall be agreed among all parts performing any task in that given system.

Transmission of electronic documents:

The texts are pretty weak in this point. They just list the different existing forms of transmission at the moment (e-mail, fax…) and suggest that any form is accepted, as long as there is no any legal basis establishing a particular form of transmission. We notice again that the attitude of the text is “when nothing else is said, electronic transmission is accepted”

Preservation of electronic documents:

The provisions describe in detail the conditions for the preservation of electronic records, covering the question of format, time, and inalterability of the content.

Regarding the time and format, the provisions indicate that the electronic documents must be preserved during their complete lifecycle in the original format (i.e., the format in which they were created or scanned) and in a second format that facilitates its preservation for the long term. The text suggest (but does not establish) some of those so called “preservation formats”: PDF/A or Tiff. Together with the document, the provisions indicate that their metadata must be also preserved, and so with the electronic signature and the information regarding its transmission and production in a workflow (if any).

Concerning the inalterability of the content, the Commission is requested to have an electronic repository for its electronic records and keep an electronic “hash” code for each of the records in the repository. This hash code is an algorithm that will allow knowing if the document has been changed since its inclusion in the repository. The repository itself must include some functionalities about the safety of the records hold (back ups, etc.)

Security of electronic documents:

These provisions just indicate that the electronic records must comply with the general provisions on security (applicable also for paper records) and just add that any new electronic program must have the agreement of the Security Directorate to guarantee that it complies with the security rules.

Conclusion:

The position that the Commission would take towards electronic management of records has been expected very long both within and outside the institution, as it could have set a trend or a path towards the e-government for other bodies and for the institution itself. However, it has got critics for being too conservative and cautious. It has been said that the position the Commission took in these texts does not change any practice, any proceeding or any legal basis in order to impulse the electronic management (and production) of records. This conservative attitude is more relevant in comparison to banks, travel agencies, vendors, or even some other public administrations that have change the ways their customers and citizens communicate, pay or interact with them.

The Commission has just put on writing the practices already in place in the Institution, adding only the creation of a repository for the preservation of the records, and opens the possibility of implementing a system for the advanced electronic signature.

It could be said that for these, more technical, decisions, a regulatory text won’t be necessary.

Basically, the texts say that, when paper is not needed, an electronic record can be accepted.

Another proof of this, maybe excessive, prudency is that the text does not set a format for long term preservation (which would have facilitate the creation of an IT strategy for the future preservation of the chosen format). Another criticism to the Commission states that, 5 years after the approval of the implementing rules, the European Commission still don’t has an advanced electronic signature mechanism.

However, it is worth to note that this text has help a lot for the mind changing in the institution. Not only in the Commission, but in almost every other public body, electronic records management projects have to face the opposition of part of the staff that does not trust electronic records. These legal provisions are a step forward for this mind change.

Recommendations for the production of Digital Preservation Plans (part 2)

By Luis Corujo, from Portugal, Archivist at the General Head Office of Archives (DGARQ), Portugal.

(Portuguese version available by clicking “Keep reading” ; Versão Portuguesa disponível em “Keep reading”)

Before the implementation of any strategy to preserve their DOs, it is necessary to take into account the best practices for document management and the contributions of the entire agency workforce related to the prodution of information.

Due to this, the development of a Digital Preservation Plan is divided into three main stages, namely:

1) Organizational Appraisal, in terms of the:

  1. Organizations Intermediate System (hardware and software used);
  2. Rules and standards by which the institution is governed and must comply;
  3. Administrative practices of the organization;
  4. Business and Document  Management System of the institution;

2) Identification of the requirements of the electronic record documents and information available in the electronic information system of the organization. These include only those that are produced in the institution, considered as institutional ownership, have a retention period of more than 5 / 7 years, according to the disposal and retention schedule, and are of recognized (vital) importance to the organization.

3) Planning the strategy of preservation, which includes:

  1. Definition of preservation strategies, which should take into account:
  • The characteristics of the DOs, in term of appearance, content, structure and behaviour.
  • The technical characteristics of electronic record documents and information systems, as the context, the storage medium, inter-relationships between different records, files, and metadata;
  • The characteristics of the process of preservation, related to usability, scalability and complexity;
  • The costs, which have a high importance on the choice of preservation solutions, and can be divided into technical and personnel costs.
  1. Definition of formats for preservation, before the creation of the digital object to be preserved, in order to avoid additional costs in its conversion to the final format. The use of standard formats and applications with open specifications ensures a continuity of use and support when compared with proprietary formats and applications.
  2. Choice of software, where, due to the choices defined in the Plan of Digital Preservation is necessary to adapt, change or replace the software. It is advisable to use, where possible, software with open specifications, non-proprietary and well documented, and should allow to execute all the necessary operations defined within the Digital Preservation Plan, be robust, easy to use and include an intuitive interface.
  3. Choice of storage solutions, you should consider the costs and available infrastructure support. This choice has to be designed for long term use, where an option that can prove to be less accurate, may require a shift of the entire plan devised.
  4. Definition of metadata (MI), which allows better management of DOs, intermediate systems, users and the features available. It is advisable to use different MI schemas, to be used in various layers, such as Descriptive MI, Structural MI, Preservation MI, Technical MI …

Before the implementation of the Digital Preservation Plan, the workforce of the agency should test the technical procedures. These tests must be performed on duplicates of electronic record documents and the relevant electronic information systems. The integrity, functionality, structure, content and metadata associated with the preserved digital information, should be periodically monitored after the implementation of the preservation strategy, to ensure its stability and to identify schedules for subsequent applications of preservation measures. Must be taken into account and specify the elements, such as human resources, calendar, etc.., which will be part of the monitoring effort of the entire Plan of Digital Preservation project implementation (manage, implement, test, evaluate, redesign , etc.).

Continue reading

Recommendations for the production of Digital Preservation Plans (part 1)

By Luis Corujo, from Portugal, Archivist at the General Head Office of Archives (DGARQ), Portugal.

(Portuguese version available by clicking “Keep reading” ; Versão Portuguesa disponível em “Keep reading”)

The General Head Office of Archives (DGARQ), being the coordinator of the archival and records policy in Portugal, has since 2006, defined digital preservation as a strategic priority for action. This concern stems from the problems of digital material related to the short to medium term obsolescence, the loss of evidence value, and even the loss of information, and that ultimately have an effect on all the information that must be stored for more than 7 / 10 years, being it information for permanent keeping or disposal. In practice, these problems affect all organizations producing digital information, if not all organizations!

This is in fact a difficult problem to address in Public Administration (PA), which, because of the determinations from the Electronic Government initiative, must base their activities in electronic business processes, to facilitate those same business processes, ensuring a faster, complete and transparent service to citizens. This means that public institutions are producing larger amounts of electronic information, in many cases without any specific preservation operations due to the estimated obsolescence horizon of 7 / 10 years. The absence of any commercial software for this purpose has further aggravated this situation, since the initiative and responsibility for safeguarding the electronic information belongs to the producing organizations.

Against this background, the DGARQ has developed resources, processes and tools in order to address the need to preserve electronic information produced by AP, and that their continued preservation is regarded as necessary. Being said, there have been devised three lines of action, which includes the construction of a digital national archive – RODA – that allows the integration, management and dissemination of digital objects produced by the AP, counselling and technical support about the electronic document management and digital preservation, and, finally, the preparation of technical documents. The publication of the document Recommendations for the Production of Digital Preservation Plans (PDF in Portuguese) is part of this last line of action, being a technical document that aims to help public institutions to understand, manage and develop tools that allow for preservation of their digital objects. As the name indicates, it is a set of recommendations, a guide to developing a plan for digital preservation, not an attempt to provide an instant ready-to-use solution. It is the organization that must define its strategy, formalizing all procedures and their timing to preserve digital information in a document that will be the Plan of Digital Preservation of the institution. The Digital Preservation Plan is, therefore, a strategic document which contains policies and procedures geared towards the creation of a technical and organizational structure that will allow the continuous preservation of the electronic information through actions on digital objects (DOs) that are part of it. The Digital Preservation Plan should therefore be the result of a collaborative effort between the organizational units of the institution such as archival/record management and IT.

(Part 2 available soon)

Continue reading

Arquivo Distrital do Porto: global ‘n’ virtual

By Maria João Pires de Lima, Director of the Porto Regional Archive, Porto, Portugal

(Portuguese version available by clicking “Keep reading” ; Versão Portuguesa disponível em “Keep reading”)

At the beginning of the millennium, the vision of Porto Regional Archive management for the relationship with the different publics/clients/users, envisaged the globalization of its presence using the information and communication technology. In the absence of the adequate resources, this global project could only be materialized through various projects, two of which – Digitarq and CRAV – with specific relevance, either for the financing obtained, or for the innovating nature and “contribution for the development of the Information Society” as recognized by the prize Fernandes Costa – Agência para a Sociedade do Conhecimento (Agency for Knowledge Society), awarded in year 2004.

DigitArq project (2003-2004) allowed that, as from April 2004, the user could search through the Internet on a database of archival descriptions and view digital images of documents whenever available. The project had two fundamental areas:

1) Conversion of all description/research tools of the almost 700 fonds in our custody, produced along the years, from their analogical supports and formats (paper) and digital (Arquebase, Word, Excel, Access, Filemaker) to one standardized from the archival point of view (EAD, ISAD, ISAAR; EAC) and computing (DTD-EAD e XML) tool.

2) Design and development of a structure to receive/integrate and manage digital objects (DO) limited to digitised images of documents.

We developed applications to manage the descriptive and technical meta-information and the search by user. This information system of archival description (DigitArq) keeps under different levels the description of documents that are being produced, the technical information of digital objects reproducing the documents and the images produced for Internet.

Since the beginning of 2008 that Porto Regional Archive allows the online access to the whole of services provided to users of reference and reading, including in this area, besides the search and reading/consultation of documents (since 2004), the supply of reproductions in digital format and the access to functionalities as the electronic payment of services (credit card and ATM). The system will allow, as soon as legally possible, the remote download of digital certified copies by users everywhere in the World.

This second project, CRAV – Consulta Real em Ambiente Virtual (“real reading in virtual environment”) – gave the user access online not only to the information in our custody but also to the services provided concerning the documents: the research, reproduction and certification.

The projects developed allowed the acquisition of knowledge relevant also to the consulting area and technical assistance. On the other hand, aiming to promote the preservation and access to national archival heritage, under the permit of the Directorate General of the Portuguese Archives (DGARQ) – and, previously, of Institute of National Archives/Torre do Tombo – , Porto Regional Archive allowed the use of the application developed by the DigitArq project to more than two dozens of other entities.

Presently these applications are provided by DGARQ “open source and no charge”. This is a significant step in the direct support to the archival community and to the national institutions extended to the international community.

The recognition of the work developed comprises still the adoption by DGARQ of the set of applications DigitArq for its archives network: remaining Regional Archives, Arquivo Nacional Torre do Tombo (National Archive) and Centro Português de Fotografia (Portuguese Centre of Photography, a national archive for photography).

For further information about Digitarq, please consult the following web pages:

1. “Digitarq project” document http://www.adporto.pt/ficheiros_a_descarregar/1-5_digitarq_project.pdf

2. Porto Regional Archive (CRAV) http://www.adporto.pt/index.php?option=com_content&task=view&id=17&Itemid=41&limit=1&limitstart=2

Porto Regional Archive can be contacted at info@adporto.pt

Continue reading

Teaching about international archives in the United States

By Joel A. Blanco-Rivera, from Puerto Rico, doctoral student in Archival Studies at the University of Pittsburg, School of Information Sciences, USA. E-mail: joel.blanco@gmail.com

This summer, as part of the Archival Studies program at the University of Pittsburgh, I will be teaching the course “An International Perspective on Archives.” This will be the first time such course will be taught.  Even thought a variety of courses from academic programs in the US include topics that offer an international perspective (with special attention to archival developments in Australia and Canada), a course that focuses completely in global archival theory and practice has been basically non-existent (my search of academic programs in the US did not find a course about international archives). For this reason, I will like to share in this blog my experience developing this course.

This course will introduce students with a diverse number of archival issues analyzed from an international perspective. The development of archival theory and practice, along with the foundation of professional organizations and the development of archival standards has put into perspective the importance contributions from different countries. In addition, recent developments more closely related with issues of access, accountability, and memory offer a great opportunity to understand and appreciate the international contributions to archival studies.

I divided the course into three main sections. First, I will discuss archival history in the 20th century. This will include a discussion about archival thinkers from different countries that have provided significant contributions to archival theory and practice. Second, I will address the topic of international professional organizations, archival standards, and archival education. And finally, the course will analyze how global discourses about postmodernism, post-colonialism, memory and transitional justice challenge traditional ideas about the archive.

A key to the success of this course will be to offer students an opportunity to learn about events and developments from different countries. That’s why students, in addition to the course readings (and reading blogs like this one), will look at a diverse number of case studies. The strategy to accomplish this will be twofold. First, students will discuss in class news, events or any other development related to archives internationally. And second, the class will have guest lectures by archivists with experiences from abroad. Most of these lectures will be accomplished using videoconference. Therefore, I will like to extend the invitation to colleagues that might be interested in being a guest for the class (it does not need to be necessary a video conference, we can plan a podcast or other form of delivery like blogs). Those interested can contact me through the email address above.

It is my intention to share my reflections while teaching this class, so I will be writing more about this in the upcoming months. I am also very interested to hear experiences from archival education programs in other countries. This is part of my strong belief that collaboration in the archival community, including sharing and discussing ideas with colleagues from all parts of the world, is essential to strengthen our profession.

The Innovation and Strategic Projects Office (DSIPE) from the General Head Office of Archives (DGARQ) as a key element to the Administrative Modernization of the Public Services in Portugal (part 2)

By Luis Corujo, from Portugal, Archivist at the General Head Office of Archives (DGARQ), Portugal.

(Portuguese version available by clicking “Read More” ; Versão Portuguesa disponível em “Read More”)

  • DGARQ, under the aegis of administrative modernization policy, supports the project Electronic Store or Real Conference in Virtual Environment (CRAV), integrated in the SIMPLEX and the National Technological Plan, in order to update and simplify the interaction with customers and provide new services in the field of electronic documents. The CRAV comprehends the integration of an information system for archival management (DIGITARQ). This project aims to offer the user the ease of a single access point for exchanges with DGARQ and its regional archives. This access point includes the searching and viewing of documents, the request for consultation (at the archive office), the reservation of documents for consultation, the request for the research and duplication of documents, the request for a quote for services and its acceptance, the checking of the state of the research or duplication request, the demand of further information on the research subject, the use of electronic payment systems, the reception of certified digital copies through the Internet, the enablement of access for disabled persons, and the purchase of products from the shops of DGARQ. The “virtual” features of the reference service will allow the management of applications, which throughout the process will enable greater control, the flowing of information between the archive services under the same environment and simultaneous support to different participants of the process, and provide clearer, shorter and more accurate indicators of the institution and respective staff performance, the satisfaction of the remote user, the sending of copies, with or without digital certificate, a secure Internet connection and the availability of other services, such as “scan on demand”, a more detailed profile of the users and of the consulted archives. This project should begin in 2010. However, there is already an electronic desk operating at the Porto Regional Archive.
  • The public service and all its organizations must depend on information produced, processed and maintained in digital format throughout its life cycle. However, in spite of the enormous technological development, there is not a common concern for the long-term preservation of the information produced today. DGARQ, aware of the challenge that involves the preservation of documents digitally produced, wishes to address this problem, and has been following three courses of action and/or axis of strategic development:

– The development of a digital archive capable of integrating, managing and distributing digital objects produced inside the Public Administration. The National Digital Archive, or RODA, is an effective response to the necessities of the gathering and management of electronic information. This project is developed by DGARQ, with the technological assistance of University of Minho. The chosen strategy was the gradual development of basic features and the progressive improvement of these features to increase the categories of digital objects and, presently, provide direct response and support to organizations that have digital objects but no resources in this specific area. RODA has been built around the standard OAIS (Open Archival Information System) and on technical documents produced under the project Interpares 2. The RODA repository is based on the FEDORA platform and uses various forms of metadata, including the EAD (Encoded Archival Description), PREMIS (PREservation Metadata: Implementation Strategies), METS (Metadata Encoding and Transmission Standard), Z39.87. RODA currently supports the integration of relational databases, structured text, still images, movies and sound. RODA will be associated to the Network of Portuguese Archives (RPA) and, in the context of RPA, should provide a service of digital converters to begin operating in 2009.

– The production of technical documents and standards to help institutions understand, manage and develop tools which allow the preservation of digital objects. With this objective, DGARQ has published the Digital Preservation Plans Development Guide.

– The accomplishment of other tasks through consultants and the production of specific technical documents to assist institutions in the management of their electronic (and conventional) documents, to promote a proper document management at the initial stage of production and allow a more efficient downstream prevention of problems derived from technological obsolescence and lack of management processes able to identify, classify and retrieve digital information. DGARQ offers expert advice, has published the Electronic Archives’ Management Software Budgeting Guide, a simplified version of the technical manual MoReq (Model Requirements for the Management of Electronic Records), and intends to help organizations select application systems for document management that meet the main requirements of MoReq.

Continue reading

The Innovation and Strategic Projects Office (DSIPE) from the General Head Office of Archives (DGARQ) as a key element to the Administrative Modernization of the Public Services in Portugal (part 1)

By Luis Corujo, from Portugal, Archivist at the General Head Office of Archives (DGARQ), Portugal.

(Portuguese version available by clicking “Read More” ; Versão Portuguesa disponível em “Read More”)

The current General Law of the General Head Office of Archives (DGARQ) gives to the Innovation and Strategic Projects Office (DSIPE) responsibilities in the development of studies and projects on electronic government which may contribute to the preservation of the digital archive heritage. Such responsibilities include the development of standards, technical guidance and the participation in international projects in the fields of management and preservation of digital records and archives, and also the support of the conception, development and implementation of electronic archiving systems for institutional producers and keepers of records and archives, essential to the Public Administration modernization described by the SIMPLEX and the Portuguese Technological Plan. DSIPE is also responsible for developing a national plan for document digitalization, supporting the research, publication and diffusion related to the safekeeping and improvement of archival and photographic heritage. To accomplish this mission, DSIPE has to supervise and support projects for the conception, development and implementation of digitalization projects of Portuguese archival and photographic heritage, and coordinate the management of web resources, to improve the access to the national archival heritage and provide more services to the users.

To achieve such purpose, DSIPE, under the aegis of DGARQ, is in the process of planning and developing several projects, in particular: the implementation, up keeping, control and maintenance of the National Archival Authorities File (FNAA), the National Digitalization Plan (PND), the Real Conference in Virtual Environment (CRAV), and, under the scope of the Digital Preservation, the management of the Repository of Authentic Digital Objects (RODA), and of the recommendations for the production of Digital Preservation Plans.

    • The purpose of FNAA is to open access, in standardized and autonomous form, to descriptors of legal entities, either individuals or families, as producers, owners, or in any other way related to the records and archival documents. Associating this information to the documents’ description will allow a better understanding of the context production, an easier retrieval of information and documents and the virtual listing of all documents scattered throughout different entities, even if such documents were produced by the specific functions and activities of one or more entities throughout time. This function may be especially useful in the case of structural changes inside the public services, being the Central Government Administration restructuring program (PRACE) one recent example. The FNAA will be a reference tool to the public service, allowing the recovery of information in the archival documentation and in the administrative records. This tool should also be made available for citizens, under the requirements of accountability, transparency, simplification and modernization of the public service, and to promote a more informed citizenship. The partnerships promoting the development of FNAA will contribute to the creation of a network of Portuguese Archives, assist in the development of an integrated approach to records/archives (in contrast with the traditional division between “administrative records”, “records” and “archives”), support the integration of new documentation media to answer the new needs and challenges faced by the record and archives management, and, finally, develop record and archival systems through the implementation of better practices and improvements on information-shared work. To accomplish this objective, DGARQ has developed a technical document with the operating rules of FNAA and the functional requirements for its development. This project will be associated to the documentation on Interoperability Metadata (MIP), produced by DGARQ with the purpose of advising the General Office of the Council of Ministers’ Presidency.
    • The PND was created under a framework of international and national EU policies concerning the digitalization, online accessibility and preservation of digital content, the employment of new information technologies, and the strengthening of European cooperation in the field of records and archives. Therefore, under the Portuguese Technological Plan and of the National Strategic Reference Framework (QREN), the organizations with archival heritage – archives, libraries, documentation centres, museums and other private organizations – are responsible for ensuring that their services and products are electronically available and remotely accessible.
    • The PND is a three years initiative, coordinated by a team led by DGARQ in collaboration with the Portuguese National Library (BNP), and composed by members of institutions of the Ministry of Culture. Its aim is to involve these organizations in the identification and exchange of experiences, the understanding of the projects’ impact at national level, the consideration of the difficulties ahead and the identification of core necessities of individual users and organizations. To promote the digitalization and its national accessibility, efforts will be made to diversify and streamline the digitalization practices, maximize the efficiency of investments in digitalization projects and provide the professionals with expertise in digitalization and preservation. This process will also imply the creation of a plan for the long-term preservation which advances the treatment of documents for digitalization. This cooperation in the creation, distribution and preservation of digitalized content and/or original digital content, under a strategy of sustainable development, can facilitate the access to the cultural heritage, through the articulation and integration of measures that ease user access to various archival fonds and collections.

      (Part 2 available soon)

      Continue reading