Introduction to Audiovisual Archives E-Book

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 398

Veröffentlichungsjahr: 2013

Beschreibung

Today, audiovisual archives and libraries have become very popular especially in the field of collecting, preserving and transmitting cultural heritage. However, the data from these archives or libraries - videos, images, sound tracks, etc. - constitute as such only potential cognitive resources for a given public (or "target community"). They have to undergo more or less significant qualitative transformations in order to become user- or community-relevant intellectual goods. These qualitative transformations are performed through a series of concrete operations such as: audiovisual text segmentation, content description and indexing, pragmatic profiling, translation, etc. These and other operations constitute what we call the semiotic turn in dealing with digital (audiovisual) texts, corpora of texts or even entire (audiovisual) archives and libraries. They demonstrate practically and theoretically the well-known "from data to meta-data" or "from (simple) information to (relevant) knowledge" problem - a problem that obviously directly influences the effective use, the social impact and relevancy and therefore also the future of digital knowledge archives. It constitutes, indeed, the heart of a diversity of important R&D programs and projects all over the world.

Leseprobe

Table of Contents

Introduction

Chapter 1: Context and Issues

1.1. The ARA program – a brief historical overview

1.2. The scientific and cultural heritage of the ARA program

1.3. The working process

1.4. Knowledge engineering in the service of the ARA program

1.5. The digital environment and the working process

1.6. Analyzing an audiovisual corpus using ASW Studio

PART 1: THE SEGMENTATIONAND DESCRIPTION WORKSHOPSFOR AUDIOVISUAL CORPORA

Chapter 2: The Segmentation Workshop for Audiovisual Resources

2.1. Introduction

2.2. Segmentation of audiovisual corpora – a general presentation

2.3. Appropriation of the segmentation workshop

2.4. Some additional thoughts about segmentation

2.5. Perspectives relating to the segmentation workshop

Chapter 3: Description Workshop for Audiovisual Corpora

3.1. A general overview

3.2. The “metadescription” part of an audiovisual analysis in ASW Studio: the mark of the editor’s choice

3.3. The “identifying information of an audiovisual resource” part in the ASW description workshop

Chapter 4: Analysis of Audiovisual Expression

4.1. Introduction

4.2. Analysis of the visual shot

4.3. Analysis of the sound shot

Chapter 5: Analysis of the Audiovisual Content

5.1. Thematic analysis

5.2. A concrete example of the description of a topic

5.3. The model of thematic description

5.4. The objects of thematic analysis

5.5. Procedures of analysis

5.6. The different components of a model of thematic description

5.7. Libraries of models for the description of subjects

Chapter 6: Uses of an Audiovisual Resource

6.1. The “Uses” part of the ASW description workshop

6.2. Producing a linguistic adaptation of an audiovisual resource

Chapter 7: Model of an Audiovisual Publication in the form of a Web Portal

7.1. Introduction

7.2. The ArkWork homepage

7.3. Thematic access to audiovisual resources

7.4. Direct accesses to the audiovisual resources

7.5. Access to the audiovisual resources by thesaurus

7.6. Contextualization of the video

PART 2: TECHNOLOGICAL ENVIRONMENT, DEVELOPMENTAND NEW PERSPECTIVES

Chapter 8: The ASW Digital Environment

8.1. Introduction

8.2. General presentation

8.3. SemioscapeLibrary

8.4. Semioscape

8.5. Conclusion

Chapter 9: The ASW Studio

9.1. Introduction

9.2. The common libraries

9.3. SemioscapeData

9.4. ESCoM Update

9.5. ESCoM ffCoder

9.6. ESCoM OntoEditor

9.7. ESCoM-INA Interview

9.8. ESCoM SemioscapeAdmin

9.9. The ESCoM suite 2011 installer

9.10. Semiosphere

9.11. Conclusion

Chapter 10: The Technical Development of the “Web Portal” Publishing Model

10.1. The notion of “publishing module”

10.2. RIAs

10.3. The “Menu” publishing module

10.4. The “Video player” publishing module

10.5. The “contextualization of a video” publishing module

10.6. The “temporal location” publishing module

10.7. The “geographical location” publishing module

10.8. Conclusion

Glossary of Specialized Terms

A

C

D

F

G

M

O

P

R

S

T

V

W

Glossary of Acronyms and Names

A

C

D

E

F

I

K

L

M

O

P

S

Bibliography

List of Authors

Index

First published 2012 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address:

ISTE Ltd27-37 St George’s RoadLondon SW19 4EUUKJohn Wiley & Sons, Inc.111 River StreetHoboken, NJ 07030USAwww.iste.co.ukwww.wiley.com

© ISTE Ltd 2012

The rights of Peter Stockinger to be identified as the author of this work have been asserted by him in accordance with the Copyright, Designs and Patents Act 1988.

Library of Congress Cataloging-in-Publication Data

Introduction to audiovisual archives / edited by Peter Stockinger.

pages cm

Includes bibliographical references and index.

ISBN 978-1-84821-337-1

1. Audio-visual archives. 2. Audio-visual materials--Classification. 3. Research--Methodology. I. Stockinger, Peter.

CD973.2.I68 2011

025.3'47--dc23

2011042380

British Library Cataloguing-in-Publication Data

A CIP record for this book is available from the British Library

ISBN: 978-1-84821-337-1

Introduction1

This collective work deals with the analysis of audiovisual numerical texts or corpora, which may e.g. form part of an audiovisual library or archive.

The development of methods, tools and conceptual frameworks (or models) for the concrete analysis of audiovisual texts or corpora is one of the most important issues for multimedia (audiovisual) digital libraries, archives, collections, etc. and also for any project or program to compile and disseminate knowledge heritage (e.g. cultural, scientific etc.).

Analyzing audiovisual recordings, shoots, sound recordings, film or complex multimodal documents etc. obviously constitutes an essential step for any classification of the (digital) collection of an archive or library.

Above all, however, it is the most important activity by which an actor (an individual, group of individuals, institution, etc.) obtains and exploits numerical audiovisual data to transform them – depending on their own skills, expectations and requirements, but also within the limitations imposed by the tools, methods and models available – into genuine cognitive resources which they regard as “useful”, “pleasant”, “interesting” or simply relevant, i.e. which have a value for them.

Ten years ago now, along with a small nucleus of permanent collaborators from the ESCoM (Semiotics Cognitive and New Media Team), the research center at the Fondation Maison des Sciences de l’Homme (FMSH – House of the Human Sciences Foundation) in Paris, we set up the ARA (Audiovisual Research Archives) program. One of the objectives of this program, which will be described in more detail in the Chapter 1 of this book, is to compile and distribute scientific and cultural heritage, notably through scientific events and field-work carried out in human and social sciences. Another objective of this program is to set up research and development projects aimed at:

a) collecting and producing audiovisual documentation (of field-work, for example);

b) compiling analysis corpora and effectively analyzing these corpora;

c) creating publishable corpora and publishing them;

d) defining and setting up (metalinguistic) models and essential procedures to successfully carry out the aforementioned three “tasks”.

In this book, and another collective work complementing this one (see [STO 12a]), we will present and discuss the results of our research and development relating to the analysis, description and indexing of audiovisual corpora. The question of analysis has been addressed from the start with regard to the following three issues:

1) a good understanding of the activity of analysis must take account of the internal structural organization of the audiovisual text and must have recourse to the semiotics of the audiovisual text or discourse;

2) a true analysis (going beyond, e.g. simply producing unstructured lists of keywords) of audiovisual corpora cannot be carried out without a metalanguage (an “ontology”), i.e. models of description representing the area of expertise covered by a corpus to be analyzed;

3) of course, no analysis can take place without an appropriate working environment.

Thanks to a series of French and European R&D projects1 and to the support of the FMSH, between 2001 and 2009, we were able to make tangible progress towards addressing the three issues mentioned. However, in particular it was the ASW-HSS2 project, financed by the French National Research Agency (Agence Nationale de la Recherche – ANR), that gave us the time and means needed to develop:

– a metalanguage for analyzing audiovisual corpora documenting a wide variety of areas of knowledge/expertise. This metalanguage is a generic ontology (called “ASW3 ontology”) which has helped us to define, use and validate a whole series ofdomain ontologies4 and models of description adapted to thematically limited areas of knowledge/expertise. This book will present it through a wide variety of concrete examples. [STO 12b] gives a more theoretical and more detailed account of this metalanguage;5

– a working environment for segmenting and describing audiovisual corpora entirely based upon the ASW metalanguage of description. The name of this environment is ASW Studio; it is made up of several specialized workshops: the Segmentation Workshop, for (virtually) segmenting an audiovisual object; the Description Workshop, for describing an audiovisual object; the Publication Workshop, for publishing an audiovisual object; the Modeling Workshop, to model the metalinguistic resources needed to undertake an analysis/description of an audiovisual object. In this book we will present the two following workshops in particular: the Segmentation Workshop and the Description Workshop; the presentation of the Modeling Workshop will be the subject of [STO 12b]; as the Publication Workshop is still partially under development, it will be the object of a new publication in late 2012;

– an as-yet relatively simple metalanguage for defining models for publishing/republishing audiovisual corpora in the form, e.g. of themed folders, bilingual folders, theme-limited video-glossaries, themed Websites, etc. These models are indeed used for publishing/republishing audiovisual corpora but the metalanguage enabling us to define them has not yet been made explicit. Clarifying the organization of this metalanguage and incorporating it into the ASW generic ontology will, conditions beyond the authors’ control permitting, constitute the main object of the ESCoM’s research activities during the next few years.

This book is divided into two main parts. In part 1, following an introductory chapter contextualizing our R&D activities since 2001, the different approaches to analyzing of an audiovisual corpus using ASW Studio will be presented:

– strictly textual analysis, consisting of the identifying passages which are relevant to an analysis and to the (virtual) segmentation of an audiovisual object (Chapter 2);

– metadescription, which clarifies the content and objectives of the analysis itself as well as the authors of the analysis, the rights associated with using the results, etc. (Chapter 3);

– paratextual description, the aim of which is to formally identify the audiovisual object being analyzed (title, author, genre, summary of content, etc.) and the relative rights associated with its use (Chapter 3);

– audiovisual description, which relates to analyzing visual, acoustic and audiovisual shots (Chapter 4);

– thematic description, which deals with the content, the subjects dealt with and developed by the audiovisual text being analyzed (Chapter 5);

– pragmatic description which clarifies the potential interest of the audiovisual text in question for a given audience/use and also looks at its possible translation-adaptation (Chapter 6);

– publication of an audiovisual corpus in the form of a Web portal which is the usual form of publishing the audiovisual corpora analyzed and indexed during the ASW-HSS project (Chapter 7).

Part 2 of this book is given over to a technical presentation and a detailed discussion:

– of the ASW digital environment (Chapter 8);

– of the ASW Studio dedicated to work on audiovisual corpora (Chapter 9);

– of the computerized development of the publishing model called “portal with specialized access to audiovisual corpora” – the standard model of publication of the experiments conducted during the ASW-HSS project (Chapter 10).

Let us reiterate that this collective work is accompanied by a second collective work [STO 12a] which deals with new practices in analyzing audiovisual corpora. That book contains in-depth presentations of highly specialized analyses which could not be conceived of without genuine scenarios of analysis, projects aimed at implementing “shared” audiovisual archives using the ASW approach (i.e. the ASW metalanguage and the ASW Studio) and finally, the exploitation of the results of analysis of audiovisual corpora in the context of social media, Web 2.0 and mobile communication. In [STO 12b], the reader will find a more detailed and systematic presentation of the ASW metalanguage and of all the elements which make it up.

To conclude this introduction, let us highlight once more that this book really is the product of a collective and interdisciplinary effort combining “fundamental” research with applied research, and computing with human sciences (particularly semiotics and linguistics). As mentioned above, the work has been carried out over 10 years by a small team of researchers and engineers who are also the authors of this book and of [STO 12a]. The author of this introduction expresses his gratitude and high esteem to each of them.

Throughout the last 10 years of research and development, the team has benefited from the support and the backing of many colleagues and friends in France and abroad. Thanks go in particular to the following individuals: Patrick Courounet, Steffen Lalande, Abdelkrim Beloued, Bruno Bachimont (INA Research Dept.); Jocelyne and Marc Nanard (CNRS-Lirmm); Marie-Laure Mugnier, Michel Chein, Alain Gutierrez (CNRS-Lirmm); David Genest (University of Angers-Leria); Danail Dochev, Radoslav Pavlov (Bulgarian Academy of Sciences); Stavros Christodoulakis, Nektarios Moumoutzis (Technical University of Crete, Chania).

In addition, special thanks go to Muriel Chemouny (FMSH-ESCoM) for having proofread each of the contributions which make up this book, and to Elisabeth de Pablo (FMSH-ESCoM) for formatting this manuscript.

Our special thanks also go to ISTE/WILEY for giving us the opportunity to present our research and development over the past decade to a non-french speaking audience. Finally, we are especially grateful to Benjamin Engel for having realized such an excellent translation in such a short time.

1 Introduction written by Peter STOCKINGER.

1 For more information, see Chapter 1 of this book; see also the glossary of acronyms and project names at the end of this book.

2 See official Website of the ASW-HSS (Audiovisual Semiotic Workshop – Human and Social Sciences) project: http://www.asa-shs.fr/.

3 The acronym ASW means “Audiovisual Semiotic Workshop” and refers, of course, to the ASW-HSS project financed by the French National Research Agency (ANR).

4 As part of the ASW-HSS projects, several experimental workshops dealing with the formation, analysis and publication of audiovisual corpora within limited areas of knowledge/expertise: literary heritage, archeology, cultural diversity, etc. have been defined.

5 The research diary or blog http://asashs.hypotheses.org/ is entirely dedicated to issues relating to the ASW metalanguage of description, its evolution, its reuse and its instrumentation within the ASW Studio framework.

Chapter 1

Context and Issues1

1.1. The ARA program – a brief historical overview

This book presents the results of a 10-year collective research effort on the issue of analysis of audiovisual corpora forming part, e.g. of a digital library. The advantages and issues involved in analyzing an audiovisual corpus are many and often very different from each other. In any case, they far exceed the “standard” framework of library and/or documentary sciences and techniques. On the other hand, they are reminiscent of the issue of monitoring expertise and concrete exploitation of information or knowledge in the different economic sectors.

For all contributions in this book, the reference context for addressing the question – as complex as it is exciting – of analyzing audiovisual texts or corpora is the ARA (Audiovisual Research Archives – in French: Archives Audiovisuelles de la Recherche1 or AAR) program. The ARA program is a research and development project of the Cognitive Semiotics and New Media Team (Equipe Sémiotique Cognitive et Nouveaux Médias – ESCoM) of the Fondation Maison des Sciences de l’Homme (FMSH – House of Human Sciences Foundation) put in place in 2001 following several years of research on the conceptual analysis of digital data and the issues surrounding digital libraries for research, education and culture (see [DFS 97; VHF 97a; VHF 97b; AKV 99]). The ARA program is especially dedicated to the issue of compiling, processing and analyzing audiovisual corpora, as well as publishing (and republishing) them online.

In 2000, by means of a French research project entitled OPALES (“Outils pour des Portails Audiovisuels Educatifs et Scientifiques” – literally, Tools for Educational and Scientific Audiovisual Portals)2 and following an initial assessment of the needs of the scientific community regarding the exploitation of audiovisual contents via the Internet [DPL 01], a prototype was specified and developed for an “online video library”-type generic tool aimed at promoting scientific and educational events.3 The classification of the audiovisual collection of this very first video library, the predecessor of the ARA, was made based on an early and rudimentary metalanguage for describing audiovisual content (i.e. based on a domain ontology).

The “Opales” video library prototype, as well as the very first metalanguage for audiovisual content description, then formed the basis for the definition and implementation of a far more ambitious program of digitization and dissemination of scientific and cultural documented heritage in the form of corpora of all sorts of audiovisual texts, i.e. from almost raw recordings with no notable postproduction to documentaries, reports and other “real world” and “direct” shoots, although not (hitherto) including fictional productions. After some hesitation, this ambitious project was called – in French – Programme Archives Audiovisuelles de la Recherche (AAR), translatable as Audiovisual Research Archives Program (ARA).

The implementation and general running of the ARA program and its different activities was preceded by a considerable amount of previous work, aimed at defining as explicit a strategic framework as possible, and a guiding scheme for specifying the identity, the particular place of the aforementioned program in the context of the research on digital libraries and their concrete exploitation. Thus, when defining the general objectives of the ARA program, we focused on the fact that they should definitely not be reduced to a “simple” program of recording events and “online publication” as is the case for the vast majority of video library, photo library and other multimedia library projects which, indeed, often content themselves with a very modest policy regarding the exploitation, valorization and reuse of their documentary collections.

On the other hand, the ARA program was created from the word “Go!” to fulfill the following two joint objectives:

“[…].

1) compilation and distribution of public research heritage in the form of audiovisual, visual, sound and text files (with digital support), of scientific events such as interviews with researchers, seminars, scientific exhibitions, reports, video montages, documentaries, etc.;

2) design and development of technologies and tools suitable for the production and management of audiovisual and text archives, the processing of audiovisual records and their use in the contexts of research, education and scientific journalism.

[…]” [AAR 04, p.3].

The wording of these two objectives unequivocally shows that, in the context of the ARA program, we absolutely preclude the idea of reducing the work of compilation and distribution/exploitation of knowledge heritage to a simple technical process of capture/digitization of audiovisual data, their computerization and online distribution.

On entirely the other hand, this work depends intrinsically upon more complicated procedures, as regards transforming any digital data (a photo, an audiovisual or sound recording, etc.) into a genuine cognitive resource for a specific audience and specific uses. Yet, this transformation may not be done without suitable approaches, methodologies, conceptual resources (such as scenarios and models for compiling, describing, publishing/republishing and preserving audiovisual corpora in the long-term), appropriate computer tools and, of course, skills and therefore specialized human resources. Hence, naturally, the specificity of the ARA program, as compared to other similar initiatives and projects, relies upon the intrinsic links between:

1. the concrete work of constituting, processing, analyzing and publishing audiovisual corpora to document an area of knowledge;

2. The theoretical and methodological knowledge and know-how, the expertise necessary for constituting, processing, analyzing and publishing audiovisual corpora;

3. the concrete achievements - not only in the form of analyzed and published audiovisual corpora but also in the form of so-called metalinguistic (see section: 1.1) and computer resources – for analyzing and publishing audiovisual corpora.

In this book, we will demonstrate through a multitude of examples, how these three aspects, which are essential to a project of constitution/diffusion of a body of knowledge heritage, stand in for and reinforce one another.

1.2. The scientific and cultural heritage of the ARA program

One of the most important aspects in terms of activities carried out as part of the ARA program is, of course, the concrete work of collecting and diffusing knowledge generated in human and social sciences (HSS) by way of particular “events” such as lectures, conferences, workshops, working meetings, research seminars, higher education classes or by structured and in-depth interviews with researchers and lecturer/researchers working in HSS.

In comparison with initiatives close to the ARA program,4 one of the main points of the ARA program has been to accompany and valorize, as far as possible given its budgetary and logistical limitations, the particular position of the FMSH in Paris5 in the French institutional field; a particular position that the historian Maurice Aymard, former administrator of the Foundation, had defined as that of betting not only on the internationalization of research but also, far more “radically”, on the “de-Europeanization and inter-culturalization of the fundamental concepts and issues of [human and] social sciences”.6 Relying, on the one hand, on the FMSH’s geographical and themed programs7 and international networks, and on the other on the fact that the FMSH received (and still receives) hundreds of researchers from all over the world each year, the ARA program was thus able to compile (particularly between 2002 and 2005/2006) a truly exceptional and unique scientific heritage, made up of contributions from researchers in institutions not only in France but in some 85 countries the world over.

This was not only about “hastily” collecting the additions to scientific knowledge by researchers from a many countries in the world. The stated goal of the ARA program was to methodically collect information from colleagues working in France or abroad. These methodical collections relied on explicit models and field scenarios (see section: 1.4) and were quite deliberately implemented when compiling audiovisual analysis corpora on certain chosen themes. Therefore, from 2005/2006 onwards, a number of particularly important aspects for contemporary research were advantaged, among them the following three:

1. the often conflicting relationships between globalization, cultural diversity, multiculturalism and/or communitarianism and intercultural dialog;

2. the huge (social, political, economic, etc.) need for models and scenarios to understand and evaluate the changes of the modern world;

3. the central questions concerning the construction, the very organization of human sciences, the epistemic and theoretical status of its concepts and models, the “paradigmatic” change from disciplinary research towards inter- or rather trans-disciplinary research on specifically identified issues as well as the relationships between HSS, natural and formal sciences and engineering.

In addition, since 2005/2006, the ARA program has been exploring other field to collect, digitize and distribute knowledge heritage. Hence, projects of collection, analysis, publishing and online distribution of audiovisual corpora concerning traditional knowledge and know-how,8 collective memory,9 geopolitical regions,10 traditions and new forms of artistic expression,11 day-to-day culture,12 European emigration to Latin America,13 etc. have been carried out. The ARA program has thus developed, over the course of its existence, an original and methodologically solid14 approach to the compilation and online publishing of audiovisual corpora.

Among the tangible results of this “policy” of producing scientific and cultural heritage using digital audiovisual technology, the ARA includes, among others:

– a collection of almost 6,000 hours of online videos, made up of a series of thematically-delimited corpora such as, for example, the “Social History” corpus (around 600 hours of online videos), the – “Cultural and Linguistic Diversity” corpus (around 450 hours of videos), the “Globalization and Sustainable Development” corpus (around 250 hours of videos), the “History of Mathematics and Geometry” corpus (around 160 hours of videos), the “Religious History and Study” corpus (around 200 hours of videos), etc.;

– an audiovisual collection whose authors form a 2,500-strong community working in over 900 institutions and 85 countries worldwide;

– an audiovisual collection bringing together videos in 15 different languages;

– an audiovisual collection distributed on the ARA Web portal and/or – a series of other thematically- or geographically-delimited Web portals15 forming part of the ARA;

– an audiovisual collection entirely published in the form of “mini-Websites” with each “mini-site” corresponding to a scientific event – a field of research, a cultural exhibition, etc. (hence, up to the end of 2010 the ARA portal contained and distributed about 650 audiovisual mini-sites including nearly 350 structured and in-depth interviews, 70 research seminars, 150 discussions, 50 reports and documentaries and 15 audiovisual “field” documentations);

– a collection of which some parts are re-published in the form of themed folders (in late 2010, around 85 themed folders), bilingual folders (in total, around 80 bilingual folders including French/English; French/Arabic; French/Russian; French/Chinese etc.) and themed video-lexicon (devoted e.g. to world languages, intangible cultural heritage, etc.).

Therefore, in 2009, the ARA program was qualified by the very official Agence d’Evaluation de la Recherche et de l’Enseignement Supérieur (AERES) [Agency for the Evaluation of Research and Higher Education] thus.

“[…] The ARA are a good example of the promotion of the FMSH’s cultural heritage by the systematic use of new digital technologies based on the activity of the Cognitive Semiotic and New Media Lab (ESCoM). […] The ARA are thus the product of this team’s activity. Their objective is the formation, distribution and exploitation of public heritage of knowledge produced by HSS in the form of video recordings, classes, seminars, interviews, etc. to the benefit of research, education, and learning. Over the years since their commissioning [4 years, 2006–2009, P.S.], the ARA have become a major player in this field in France […]” [AER 09, p. 20].16

1.3. The working process

As has already been said, the ARA’s activities cannot be reduced to technical procedures for capture, digitization, processing and distribution of audiovisual data. These only constitute a particular set of activities among others.

The ARA program does however rely on close coordination between several sets of activities which are essential to the implementation and running of a highly complex working process covering all stages from the production of audiovisual data to their publication.

Thus, alongside a first set of rather technical activities, a second set of activities contributes to the central task of transforming digital data into a cognitive resource. This includes, e.g. the following activities:

– definition and preparation of collections of audiovisual data;

– selection of audiovisual corpora (on the basis of collected data) for analysis and publishing;

– montage and postproduction of the selected corpus according to montage scenarios;

– analysis, description and indexing as well as (linguistic but also cognitive or cultural) adaptation-translation of the selected corpus;

– publication and/or republication of the corpus post-produced and analyzed/ adapted according to a publication model and scenario.

A third set of activities concerns activities whose aim is to preserve the originals, the legally valid documents, safeguarding the heritage but also the legal deposition of all the achievements of the ARA program.

A fourth set of activities, transversal to the first three, is concerned with the R&D activities in the true sense. One of the most obvious objectives of R&D activities as part of the ARA program is to reinforce its internal abilities in order to satisfy to its objectives of compiling and publishing/distributing scientific and cultural heritage.

The issue of strengthening the internal capabilities of the ARA program relates as much to the technical working environments as it does to the approaches to and methods for the collection, processing, analysis, publishing, distribution, exploitation and preservation of audiovisual corpora. To that end, what is required is a team and various networks of researchers, engineers but also professionals bringing together multidisciplinary skills (multidisciplinary skills which allow us to cover computing and a wide variety of approaches and disciplines in HSS) who methodically work according to explicit procedures, on issues which consider the existing and/or potential needs of the ARA program. The R&D activities undertaken as part of the ARA program are primarily aimed at defining and developing two specific types of resources:

1. computer resources suitable for effectively carrying out concrete work on an audiovisual corpus;

2. so-called metalinguistic resources necessary either to compile corpora or to analyze and/or process them, or even publish/republish them (in particular these are models and scenarios for production, analysis and publishing of audiovisual corpora; see section 1.4).

There are other activities which complement those we have just identified. Let us above all remember that it is hugely important to identify the main sets of activities for a project of compilation and distribution of knowledge heritage.

It is only on the basis of this identification that we may define the stages and explicit procedures of working process according to which the tasks of producing, analyzing and publishing audiovisual corpora documenting an area of knowledge or expertise are organized and carried out. This process is, indeed, even more complex than is suggested by an extremely simplistic (but unfortunately still very widespread) vision reducing it to a few technical gestures concerning “putting a video online” which seems to essentially consist of simply uploading the file containing the video, accompanied by a basic computer record.

We were therefore led to define the said process as precisely as possible, for scientific and technical as well as practical and financial reasons:

– scientific and technical reasons: the better to be able to identify the gaps, limits and obstacles to be overcome during the process of production/publishing so as to make it more efficient and more easily adaptable to the expectations of the audiences and stakeholders concerned as well as to the specificities of the corpora themselves;

– practical and financial reasons: the better to be able to define the competences and profiles sought, achieve better management and monitoring of the process itself and finally, also, to better calculate the costs incurred in production, processing and analysis as well as publishing and preservation of an audiovisual corpus.

In view of the experience gained during years of work compiling audiovisual corpora documenting HSS areas (especially between 2001/2002 and 2005/2006), we were able to define and implement, from 2004 onwards, the working process which characterizes the ARA’s activities as far as the compilation, publication and distribution of knowledge heritage are concerned. Figure 1.1 gives an overview of the 5 major stages according to which the said process is organized.

Figure 1.1. The five major stages making up the process of production, processing, analysis and publication of an audiovisual corpus

Figure 1.2 shows a table which, for each stage, details the main activities to be carried out. Besides the fact that this table explicitly shows all the complexity of a project or program of compilation and distribution/conservation of knowledge heritage, it is extremely useful as a reference framework which, on the one hand, enables us to implement genuine management of a team which is necessarily multidisciplinary, and on the other hand to calculate the relative durations of the different activities in view of potential specificities linked to the domain, the corpus to be compiled or its use. We refer the interested reader to Chapters 4 and 5 of the book Digital Audiovisual Archives,17 which describe two concrete examples of the creation of audiovisual archives: the first example is dedicated to the compilation of an audiovisual archive on the intangible cultural heritage of the so-called indigenous communities which live in the Andean regions of Bolivia and Peru; the second to one on a country – Azerbaijan. These examples demonstrate very well all the complexity of the working process including the production, processing and analysis as well as the publication of an audiovisual corpus documenting a given patrimony.

Figure 1.2. The major activities defining the stages of the working process for the constitution/publishing of a heritage of knowledge18

Using the table identifying the different activities which are part of the working process for constituting/publishing knowledge heritage (Figure 1.2), it becomes far easier to predict (with a certain degree of accuracy) the approximate date a corpus will be published, the duration of the work to be undertaken for it to be published, the human resources (i.e. skills) to be mobilized and finally the cost incurred by an operation to compile and publish/distribute knowledge heritage.19

The table (Figure 1.2) representing the different activities of the process of producing, processing/analyzing and publishing an audiovisual corpus encourages us to carefully distinguish between a “video-library”-type project and a project aimed at compiling and distributing an audiovisual piece cultural and scientific heritage. In the first case, we imitate more-or-less accurately a model which in itself is rather conventional (i.e. the unilateral distribution of content model, the paradigmatic example of which is television) of capture/distribution of scientific, cultural or other events. In the latter case, the capture and distribution of a scientific event is only a small part of the work. All its richness but also all its complexity relies on the fact that it has to “solve”, or rather find satisfactory solutions to, the following issues:

– the “correct” constitution of a corpus, i.e. the constitution of a relevant corpus;

– the “correct” analysis of the corpus, i.e. a relevant analysis and;

– the “correct” publication, a relevant publication of a corpus.

These issues lead us directly to the importance of knowledge engineering and semiotics for the ARA program.

1.4. Knowledge engineering in the service of the ARA program

1.4.1. Some questions

During the first period of collection of research testimonies in HSS (i.e. 2002–2005), there gradually appeared a whole series of interrogations and issues which, indeed, constitute the background and the main motivation of a new wave of R&D activities since late 2006. Three of these are:

1. the quality and richness of the content of the collections forming part of the ARA are somewhat overshadowed by the quantity (volume) of hours offered to the interested community (at the end of 2006, the ARA’s collections comprised around 3,500 hours of digital videos; at the end of 2009, around 5,800 hours);

2. the content conveyed by an audiovisual text (a raw recording, a montage, a corpus, etc.) has its own identity;

3. the audiovisual content is almost completely monolingual (i.e. the vast majority of recordings were carried out in a single language).

The first problem is reminiscent of the issue of description, classification and indexing of audiovisual corpora. The second problem relates rather to the explanation of the content of an audiovisual text, taking account both of its specific identity and the cultural and cognitive “profile” of the target audience. The third problem is traditionally associated with the translation of an (audiovisual) text, i.e. linguistic comprehension of the content and metadata explicating the content. These three problems constitute genuine issues, as much for better distribution of cultural or scientific heritage on a digital “market” which is intrinsically multilingual and multicultural, as for an appropriation which is better-adapted to the expectations and needs of the user communities in question?

In addition, the regular statistical analyses of visits to the ARA Website, the surveys put to the ARA audience via an online questionnaire on the Web portal20 and finally regular feedback from users (teachers, researchers, students, etc.) of the ARA’s audiovisual collection, demonstrate the obvious limits of a “simple” multimedia library, contenting itself with a set of more-or-less “standard” accesses to its collections: varying degrees of difficulty in locating and selecting a relevant piece of information from large audiovisual databases; temporal linearity of the audiovisual flux preventing more flexible forms of exploration, such “leafing”; absence of contextual help for the exploration and appropriation of collections of audiovisual resources; absence of usual terminologies which could help to better understand the structure of a collection and consequently explore it better; too many difficulties (technical but intellectual as well) in using or reusing audiovisual resources for specific activities in research but also in education, scientific vulgarization, etc.

1.4.2. Recourse to the semiotics of the audiovisual text

These and many other problems, have brought back to the forefront of debate one of ESCoM’s main objectives in participation in the aforementioned OPALES project; i.e. to define and develop a metalanguage for describing audiovisual texts based – in particular – on a semiotic approach to the text (see [STO 99; STO 03a; EHE 07; FRS 09; GRO 11]).

Without wishing to go into too much theoretical detail here (for more information, see [STO 03a] and [STO 12]), the semiotic structure of the audiovisual text (and of any other type of texts) may be “approached” in an intuitive and simple manner using the following seven standard questions:

1. What are the passages, moments, in the linear flux constituting the discernible (perceptible) part of the audiovisual text which catch/may catch the attention (i.e. what are the “information-carrying” segments for a given audience)?

2. What are these “information-carrying” segments about (i.e. what are the subjects addressed by the segments, what are the selected topics and themes)?

3. How are the subjects tackled and addressed in these segments (i.e. what is the – enunciative, discursive – specificity of the topics and themes selected in the “information-carrying” segments)?

4. How are the selected subjects progressively developed (described, explained, “narrated”) within a segment and also through the different segments of the audiovisual text where they appear (i.e. what is the narrative specificity of the topics or themes selected within an “information-carrying” segment or set of segments)?

5. What is the expression, the audiovisual “staging” of a topic developed in the segments within which they are selected (i.e. what is the multimodal specificity of the topics or themes selected in the “information-carrying” segments)?

6. What are the similarities/differences in procedures of selection, processing, development and audiovisual expression of a subject between several audiovisual texts forming part of a corpus, a collection or, more generally, a historically-, socially- and/or culturally-delimited field of production of audiovisual texts (i.e. what is the intertextual specificity of a topic or a theme)?

7. What are the similarities/differences between the way that a selected subject is tackled, developed and staged and the expectations, needs/desires and skills of an audience (i.e. what is the pragmatic – historical, cultural, social – specificity of a topic or theme)?

These seven questions help to “fix” and orient ideas and habits well before the production of information (i.e. prior to any filming) as well as afterward (i.e. during the publication proper stages of a resource: description, indexing, etc.).

1.4.3. Metalanguage of description, models and scenarios

In reference to the issues in the seven questions formulated above, R&D activities in the context of the ARA program are concentrated around the following four axes:

1. Implementation of models and scenarios for the analysis, description (indexing, classification, etc.) of audiovisual corpora;

2. Implementation of models and scenarios for the publication/republication of audiovisual corpora so as to better adapt them to the expectations (knowledge, skills, etc.) of their potential users;

3. Also implementation of models and scenarios for the collection of audiovisual data documenting a “field” of investigation (i.e. a field dedicated to the production/publishing of audiovisual corpora used for documenting an area of knowledge/expertise);

4. Development of a working environment enabling the semiotic contribution to be used during the processing of audiovisual corpora in view of their online publication or republication (see section 1.5 and Chapter 7).

Let us take a closer look at the metalinguistic resources of the ARA program in the form of models and scenarios – the working environment will be presented later in this chapter (see section 1.5) as well as in Chapter 7 in this book.

The models are metalinguistic resources which define the structure and organization of audiovisual objects and the scenarios are metalinguistic resources which frame and guide the activities leading to the creation of these same objects. Discussing models and scenarios in terms of “metalinguistic resources”, means that they belong to a metalanguage of description (i.e. the one mainly developed in the context of the ASW-HSS project [Audiovisual Semiotic Workshop-Social and Human Sciences] in order to work in a well-reasoned and explicit manner with and around audiovisual corpora), and that they therefore constitute tools, procedures and therefore self-sufficient cognitive instruments, for any actor involved in this type of work.21 In the context of the ARA program, they are used to collect, process, analyze and publish audiovisual data. Hence, we speak of:

1. models/scenarios of collection (of production);

2. models/scenarios of postproduction (of filming, etc.);

3. models/scenarios of analysis (of description, of interpretation, of translation-adaptation, etc.) and;

4. models/scenarios of publishing/republishing of audiovisual data.

The third category, models and scenarios of analysis, forms the main subject of this book. We will present a string of examples of these and show how to use them concretely via a specialized working environment. In [STO 12] there are more detailed explanations relating to the ASW metalanguage of description (including the models and scenarios of analysis). Let us take a brief look at the other classes of models and scenarios identified below:

1. models and scenarios of collection and production which serve either for the constitution of a new audiovisual collection documenting an area of expertise, or for the “reasoned” enrichment of an existing audiovisual collection.

2. models and scenarios of publication which also serve for republication (i.e. reuse of an already-published video in another context by adapting it to the specificities of the new context of publication) as well as the new forms of collective publishing, spread out in time and space (i.e. publication of audiovisual resources by a collective actor – a group, an institution – which may be located anywhere in the world and may also act as an author over time).

1.4.4. Models and scenarios of collection/production of audiovisual corpora

The models and scenarios of collection (production) first and foremost guide the preparation and creating of a shoot or of a series of shoots. Collection (or production) is a very complex task which is composed of a whole series of activities (see section 1.3). The collection may closely follow various strategies: more or less intuitive, more or less well circumscribed, more or less restrictive in terms of the documentation needed, subject or not to explicit procedures and norms (of quality, etc.).

At any rate, this is a deliberately oriented activity, which attempts (with more or less success) to solve the issue of obtaining the primary material (i.e. audiovisual data) which is necessary in order to create the cognitive resources for a given audience. In that sense, the aforementioned activity of constitution of heritage is either compulsorily preceded by the activity of description/modeling of the area to be documented and of the characteristics to which the documentation must conform, either framed by a sort of guide, or even simply by a “mind map” based on which it is carried out.

In other words, any constitution of a “field corpus” is carried out in reference to an intellectual framework. The implementation of an intellectual framework is part of the activity of definition, development and monitoring of models and scenarios of collection (of production) of audiovisual data which contribute to:

1. the definition and conceptual specification of the object (domain) of a patrimony to be digitized;

2. the definition and preparation of the type of field (type of investigation, geographical and temporal framework, social context, stakeholders, sources of information, etc.), and the collection of data documenting the heritage;

3. the reasoned and controlled conduct of the act of filming (i.e. of audiovisual but also photographical, cartographic, verbal, etc. recording);

4. the computerization of audiovisual data based on a field in a database or a digital archive;

5. the location (identification) of relevant rushes from the digital archive to constitute the corpus which will serve as input for the activities of postproduction on the one hand and analysis on the other;

6. “new” forms and dynamics of constitution of audiovisual collections documenting “fields”: “remote” (spatially and/or temporally) constitution of such collections, “nomadic” constitution or even concerted and negotiated constitution of collections by a community of actors and;

7. finally, the long term preservation of the cultural heritage in the form of audiovisual collections which themselves are constantly evolving.

On the ARA Web portal22 as well as on the ASW-HSS project Web portal,23 one can find a wealth of documentation which presents models and scenarios for the preparation of fields of collection of audiovisual data. A particular effort has been made for the preparation and the monitoring of interviews with researchers. Hence, each interview has been prepared with the people concerned (notably with the researcher him/herself) and carried out following a plan, a scenario with the aim of collecting relevant information relating to “problem places” (generic soundbites) defined beforehand in the interview guide. For each interview, a script has been written (either during or after the interview). The script is a kind of form according to which we collect information, references and other data then used for recording the data collected as well as constituting a working corpus for the postproduction and the analysis.

1.4.5. Models and scenarios for publishing/republishing

Let us again briefly consider the class of the models and scenarios of publication. In the context of the ARA program, the publication/distribution of an audiovisual corpus which has been analyzed beforehand and/or post-produced is necessarily carried out according to a publication model.

The definition of the standard publication model relies on the notion of an event [STO 03c]. A (scientific) event such as an interview, seminar, conference or even inquiry, excavation, concert, etc. is documented by a set of audiovisual and other resources including the collected, processed and analyzed material. The advantage of conceiving a publication thus is twofold:

1. the videos which are published online are immediately contextualized (with regard to the event they document) while of course leaving the possibility open to reuse them in other contexts;

2. online publishing is not a process necessarily linked to an author, or rather, to an authorial instance, but it may be the result of a collective process distributed over time and space.

More particularly, the publication of the audiovisual resource itself– on an event’s Website – in the form of an “online video” (i.e. a video documenting such-and-such a part of an interview, such-and-such a lecture during a conference, etc.) has firstly been defined in a metaphorical reference to books like a sort of interactive video-book, i.e. a document made up of chapters (sequences) made available to the interested audience either in the form of free reading or in the form of guided reading.

In 2006/2007 we started to develop and partially realize new publication models – models such as the themed portal,24 the video-lexicon25 about a topic or a theme, the narrative path among a set of sequences which are thematically similar, the themed folder,26 the bi/multilingual folder,27 the educational folder,28 etc. The diversification of the kinds of publication of course pursues the goal of better exploiting the intrinsic richness of the audiovisual collection of a video library such as that of the ARA. We will present some uses in Chapter 10 of this book.

1.5. The digital environment and the working process

The working process (see section 1.3) – i.e. the different activities, tasks and stages necessary to constitute, process, analyze and publish/broadcast knowledge heritage – takes place within a digital working environment possessing appropriate technologies and tools for the collection (filming, sound recording, etc.), processing (digitization, montage, compression and transcoding, etc.) and finally analysis, description/indexing and publication of audiovisual data. As shown in Figure 1.3, the environment defines and “orchestrates” three more specific processes:

1. The process of audiovisual production. This process brings together all the tasks, from the definition and planning of afield (of digitization) to the distribution of digital videos, including the filming proper, the technical acquisition of the collected rushes in the form of computer usable files, cleanup of the files and even their transcoding in such-and-such a distribution format.

2. The processing and basic publication of a filmed field (i.e. of a scientific event, a cultural demonstration, an inquiry, etc.). The video files forming a given audiovisual corpus are analyzed, cut, edited, indexed and enriched according to a set of guidelines explicitly defined in view of their publication in the form of an “event Website” on the ARA portal.

3. Finally, the processing (analyses, descriptions, indexing, annotations, etc.) and specialized (re-)publication suitable for specific uses. This process may be carried out based on pre-existing audiovisual publications which are distributed on the ARA portal.

Figure 1.3. The general digital working environment of ESCoM’s ARA program

Figure 1.3 is a diagrammatic representation of ESCoM working environment. The front office represents the working process that the users follow, divided into successive tasks that are done using specific tools. The back office represents the technological environment of ESCoM’s ARA program. Finally, the third band of Figure 1.3 shows the publications produced by the back office based on the work carried out in front office by the users.

Figure 1.4. The “basic” publishing environment ofESCoM’s ARA program

The second process identified in