Erhalten Sie Zugang zu diesem und mehr als 300000 Büchern ab EUR 5,99 monatlich.
- Herausgeber: Books on Demand
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
While there have been dramatic increases in the use of digital technologies for information storage, processing and delivery over the last twenty years, the affordances of paper have ensured its retention as a key information medium. In this book we review a wide variety of projects and technological developments for bridging the paper-digital divide. We present our information-centric approach for a tight integration of paper and digital information that is based on a general cross-media information platform. Different innovative augmented paper applications that have been developed based on our interactive paper platform and Anoto Digital Pen and Paper technology are introduced. For example, these applications include a mobile interactive paper-based tourist information system (EdFest) and a paper-digital presentation tool (PaperPoint). Challenges and solutions for new forms of interactive paper and cross-media publishing are discussed. The book is targeted at developers and researchers in information systems, hypermedia and human computer interaction, professionals from the printing and publishing industry as well as readers with a general interest in the future of paper.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 402
Veröffentlichungsjahr: 2017
Das E-Book (TTS) können Sie hören im Abo „Legimi Premium” in Legimi-Apps auf:
Ähnliche
To my parents
Abstract
While there have been dramatic increases in the use of digital technologies for information storage, processing and delivery over the last twenty years, the affordances of paper have ensured its retention as a key information medium. Despite predictions of the paperless office, paper is ever more present in our daily work as reflected by the continuously increasing worldwide paper consumption.
Many researchers have argued for the retention of paper as an information resource and its integration into cross-media environments as opposed to its replacement. This has resulted in a wide variety of projects and technological developments for digitally augmented paper documents over the past decade. However, the majority of the realised projects focus on technical advances in terms of hardware but pay less attention to the very fundamental information integration and cross-media information management issues.
Our information-centric approach for a tight integration of paper and digital information is based on extending an object-oriented database management system with functionality for cross-media information management. The resulting iServer platform introduces fundamental link concepts at an abstract level. The iServer’s core link management functionality is available across different multimedia resources. Only the media-specific portion of these general concepts, for example the specification of a link’s source anchor, has to be implemented in the form of a plug-in to support new resource types. This resource plug-in mechanism results in a flexible and extensible system where new types of digital as well as physical resources can easily be integrated and, more importantly, cross-linked to the growing set of supported multimedia resources. In addition to the associative linking of information, our solution allows for the integration of semantic metadata and supports multiple classification of information units. iServer can, not only link between various static information entities, but also link to active content and this has proven to be very effective in enabling more complex interaction design.
As part of the European project Paper++, under the Disappearing Computer Programme, an iServer plug-in for interactive paper has been implemented to fully integrate paper and digital media, thereby gaining the best of the physical and the digital worlds. It not only supports linking from physical paper to digital information, but also enables links from digital content to physical paper or even paper to paper links. This multi-mode user interface results in highly interactive systems where users can easily switch back and forth between paper and digital information. The definition of an abstract input device interface further provides flexibility for supporting emerging technologies for paper link definition in addition to the hardware solutions for paper link definition and activation that were developed within the Paper++ project.
We introduce different approaches for cross-media information authoring where information is either compiled by established publishers with an expertise in a specific domain or by individuals who produce their own cross-media information environments. Preauthored information can be combined with personally aggregated information. A distributed peer-to-peer version of the iServer platform supports collaborative authoring and the sharing of link knowledge within a community of users.
The associations between different types of resources as well as other application-specific information can be visualised on different output channels. Universal access to the iServer’s information space is granted using the eXtensible Information Management Architecture (XIMA), our publishing platform for multi-channel access.
Our fundamental concepts for interactive paper and cross-media information management have been designed independently of particular hardware solutions and modes of interaction which enables the iServer platform to easily adapt to both new technologies and applications. Finally, the information infrastructure that we have developed has great potential as an experimental platform for the investigation of emerging multimedia resources in general and interactive paper with its possible applications in particular.
Zusammenfassung
Obwohl die Verwendung von digitalen Technologien zur Speicherung, Verarbeitung und Verteilung von Information in den letzten 20 Jahren stark zugenommen hat, konnte sich Papier dank seiner speziellen Eigenschaften als wichtiges Informationsmedium behaupten. Entgegen allen Vorhersagen des papierlosen Büros ist Papier bei der täglichen Arbeit präsenter denn je, was sich auch im kontinuierlich steigenden weltweiten Papierkonsum widerspiegelt.
Anstatt Papier durch digitale Medien zu ersetzen, haben sich viele Wissenschaftler für die Erhaltung von Papier als Informationsresource und eine bessere Integration von Papier und digitalen Informationssystemen engagiert. Dies führte in den letzten zehn Jahren zu einer Vielzahl von Forschungsprojekten und technologischen Entwicklungen, die sich mit dem Thema des interaktiven Papiers befassen. Die Mehrzahl der Projekte fokusiert jedoch auf technische Fortschritte im Bereich der Hardware und vernachlässigt die grundlegenden Probleme der Informationsintegration sowie die der medienübergreifenden Informationsverwaltung.
Unser informationsbasierter Ansatz für eine enge Integration von Papier und digitaler Information beruht auf einer Erweiterung eines objektorientierten Datenbanksystems mit Funktionalität für eine medienübergreifende Informationsverwaltung. Die von uns realisierte iServer Informationsplattform führt fundamentale Konzepte für eine medienübergreifende Linkverwaltung zwischen beliebigen Medien ein. Um neue Medientypen in das System zu integrieren, muss jeweils nur der medienspezifische Teil dieser Konzepte, wie zum Beispiel die konkrete Verankerung eines Links innerhalb eines bestimmten Mediums, implementiert werden, da die iServer Kernfunktionalität für alle multimedialen Ressourcen benutzt werden kann. Dieser ressourcenbasierte Plug-in Mechanismus führt zu einem flexiblen und erweiterbaren System, in welches neue digitale sowie physikalische Ressourcen einfach integriert und mit bereits unterstützten Medien verlinkt werden können. Zusätzlich zur assoziativen Informationsverknüpfung unterstützt unsere Lösung die Integration semantischer Metadaten und die Mehrfachklassifikation von Informationseinheiten.
Als Teil des Europäischen Forschungsprojektes Paper++, im Rahmen des Disappearing Computer Programmes, haben wir ein iServer Plug-in für interaktives Papier entwickelt, um Papier und digitale Medien zu verbinden und damit die Vorteile der physikalischen und digitalen Welt zu vereinen. Wir unterstützen nicht nur das Verknüpfen von Papier und digitaler Information, sondern auch Verbindungen von digitaler Information zu Papier und selbst direkte Veknüpfungen zwischen Papierdokumenten. Die dabei verwendete multimodale Benutzerschnittstelle führt zu einem hochgradig interaktiven System, welches es dem Benutzer sehr einfach erlaubt, zwischen Papier und digitaler Information hin und her zu wechseln. Eine erweiterbare Eingabeschnittstelle ermöglicht es zusätzlich zu der von unseren Projektpartnern erarbeiteten Lösung neue Hardware Technolgien, welche sich zur Definition eines papierbasierten Links eignen, zu unterstützen.
Wir diskutieren verschiedene Ansätzte zur Entwicklung von Applikationen welche mehrere unterschiedliche Informationsmedien, wie zum Beispiel gedruckte Information, Video und Ton, umfassen. Die Information kann dabei entweder durch einen Verleger mit Expertise in einem spezifischen Fachgebiet verfasst werden, oder die Benutzer können ihre eigenen medienübergreifenden Informationsräume gestalten. Die vom Verleger verfasste Information kann jederzeit mit persönlich erstellter Information kombiniert werden. Eine verteilte Peer-to-Peer Version der iServer Platform ermöglicht zudem das kollaborative Erstellen und Austauschen von Information innerhalb einer Gruppe von Benutzern. Unsere erweiterbare Architektur für Informationsmanagement (XIMA), eine Publishing-Komponente, garantiert zudem universellen Zugriff auf die von iServer verwalteten Daten.
Unsere grundlegenden Konzepte für interaktives Papier und eine medienübergeifende Informationsverwaltung wurden unabhängig von spezifischen Hardwarelösungen und Interaktionsmodellen entwickelt. Dies ermöglicht eine einfache Adaption der iServer Platform, um neue Technologien und Applikationen zu unterstützen. Zu guter Letzt weist die von uns entwickelte Informationsinfrastruktur ein grosses Potential auf, um als experimentelle Platform für die Evaluation neu aufkommender Technologien für interaktives Papier sowie anderer Medien eingesetzt zu werden.
Acknowledgements
I would like to thank all those who either directly or indirectly contributed to the outcome of this thesis.
First of all I would like to thank Prof. Moira C. Norrie for her unlimited support and interest in my work. She provided me with the necessary freedom to realise my vision and, at the same time, always guided me in the right direction. Many of the results presented have been critically analysed and refined in very inspiring and fruitful discussions with her in which she also introduced me to the secrets and beauty of metamodelling.
A special thank you goes to my co-supervisor Prof. Christian Heath who opened my mind to many new ideas and interests. He showed me how the smallest detail recorded in field observations can have a tremendous impact on the requirements for technical problem solving.
I would also like to thank all the present and former members of the GlobIS group for their support and collegiality. It was always very stimulating to feel the great team spirit in many projects that we have realised together. I look back with pleasure to many of the adventures experienced in the “GlobIS family”!
Part of the research that has been undertaken in this thesis was embedded in the multinational European Research project Paper++ with participants from multiple disciplines. I therefore would like to thank all the other members of the Paper++ project including Guy Adams, Wolfgang Bock, Adam Drazin, David Frohlich, Peter Herdman, Sandy Johnstone, Paul Luff, Rachel Murphy, Nadja Linketscher, Abigail Sellen, Ella Tallyn and Emil Zeller.
The artists Axel Vogelsang and Curt Walter Tannhäuser further experimented with the possibilities of our cross-media platform and its application for interactive installations. I would like to thank them for challenging me in providing a variety of new requirements thereby pushing the development of our architecture. I would also like to thank my good friend Oscar Chinellato for many interesting discussions over lunch. Further, I would like to thank him and my colleague Michael Grossniklaus for helping me to improve the quality of this thesis by proofreading the manuscript.
Last but not least I would like to thank my family who always supported me and showed interest in my work.
Table of Contents
Introduction
1.1 Affordances of Paper
1.2 Integrating Paper and Digital Media
1.3 Contribution of this Thesis
1.4 Paper++ Project
1.5 Structure of this Thesis
Background
2.1 Basic Terminology
2.2 Technologies
2.2.1 Object Identification
2.2.2 Position Tracking
2.2.3 Writing Capture
2.2.4 Information Storage
2.2.5 Electronic Paper
2.3 Applications
2.3.1 Reading
2.3.2 Writing
2.3.3 Annotation
2.3.4 Paper-Based Interfaces
2.3.5 Physical Hypermedia
2.4 Analysis and Hypothesis
Cross-Media Link Model
3.1 Terminology
3.2 OM Data Model
3.3 Information Concepts
3.3.1 Links
3.3.2 Layers
3.3.3 User Management
3.4 Summary
iServer Platform
4.1 Resource Plug-ins
4.1.1 Interactive Paper
4.1.2 Web Pages
4.1.3 Movies, Sounds and Still Images
4.1.4 RFID Tags
4.2 Application Programming Interface
4.3 XML Interface
4.4 Content and Semantic Resources
4.5 Active Content
4.6 Visualisation
4.7 Distribution
4.8 Summary
Interactive Paper
5.1 Requirements
5.2 System Components
5.3 Input Devices
5.4 Address Space Mapping
5.5 Resource Binding
5.6 Summary
Applications
6.1 Nature Encyclopaedia
6.2 Interactive Worksheet
6.3 Annotation of Scientific Publications
6.4 Mammography Screening
6.5 Zurich City Guide
6.6 Edinburgh Festivals Guide
6.7 PaperPoint
6.8 The Lost Cosmonaut
6.9 Generosa Enterprise
6.10 Summary
Authoring and Cross-Media Publishing
7.1 Classification of Authoring Types
7.1.1 Link Authoring
7.1.2 Annotations
7.1.3 Content Authoring
7.1.4 Adaptive Authoring
7.1.5 Dynamic and Collaborative Authoring
7.2 Interactive Paper Authoring
7.2.1 Digital Link Authoring
7.2.2 Paper-Based Link Authoring
7.2.3 Pen-Based Writing Capture
7.2.4 Content Authoring
7.3 Cross-Media Authoring
7.4 Publisher Case Study
7.5 Summary
Conclusions
8.1 Evaluation
8.2 Vision
A iServer Schema
B iPaper Schema
C iServer XML Format
D Software
D.1 iserver Packages
D.2 ipaper Packages
D.3 sigtec Packages
There is only one thing that makes a dream impossible to achieve: the fear of failure.
Paulo Coelho
1
Introduction
Over the next decade, most corporate offices will still be geared to the movement of information on pieces of paper. Office automation will bring improvements in productivity through the use of automatic typewriters and other stand-alone equipment that crank out paper faster. Some interoffice information will move electronically, but what the manager reads and files will be printed on paper.
But during this period, a relatively small but fast-growing group of companies will have moved into the office-of-the-future environment. The leap forward will be led by the “papermakers”— those companies that are involved primarily in generating, modifying, or moving paper. These pioneers will have hooked together word-processing equipment into office systems to transfer information electronically and to move it into and out of central electronic files. For them, it will be the start of the paperless office.
Business Week, 1975 [135]
These two paragraphs were published in 1975 as part of a Business Week article in which the future of the paperless office was predicted for the very first time. Some thirty years later, we are still far away from a realisation of that vision where paper becomes completely replaced by digital technologies. The annual worldwide paper consumption is continuously increasing and has more than doubled since 1975. Paper not only refused to go away but is still prevalent in most offices. Even in research institutions with access to the latest hardware and information management technologies, paper still represents one of the main information sources. Figure 1.1 shows an example of an academic’s office of the early 21st century where paper continues to play an important role in knowledge working. It is a fact that nowadays information is available on different types of digital and physical media. It is no longer sufficient that knowledge workers, people developing or using knowledge, have a tool for organising their information for a single type of media but we need possibilities to support the seamless transition across different types of physical and digital information sources. The integration of information available in a variety of formats results in cross-media information spaces, where users can associate information across physical and digital resources.
Figure 1.1: Example of an academic’s office
Advocates of a paperless future commonly argued that there were just a few technical challenges to be solved and it would only be a matter of time until paper would become completely replaced by digital artefacts. The image quality of the computer screens on which documents had to be read was one such problem in the early 80s. Reading a document on a computer screen used to be much more fatiguing to human eyes than reading the same document on printed paper. In addition, the reading of a digital document was restricted to specific locations since the heavy desktop computers could not be carried around. However, more recently these hardware problems have been solved and small portable electronic reading devices, called e-books, with a reasonable image quality are now predicted as the successor of paper documents. With this, the unfulfilled vision of the paperless office has even been extended to the vision of the paperless home where all our books and daily newspapers will be replaced by a single e-book which can hold multiple documents and dynamically download new content on demand over a wireless network connection.
To understand why even these most recent advances in technology will not lead to a paperless world—at least not in the near future—we have to revisit the long history of paper and the evolution of some of its properties over the last two thousand years. The portability and readability of paper have already been introduced as two of its important features but there are other affordances of paper that are equally important. For instance, paper is not only a simple carrier of information but often is also used as a medium for a set of collaborative activities. Various ethnographic workplace studies have analysed the usage of paper documents and revealed further properties of paper in terms of its embedded usage in work processes. The observations made in these studies influence the design of new systems and tools aiming to support and improve some of these paper-based interactions. In the next section we present the developments in the history of paper and highlight some properties of paper which are difficult to mimic in a digital world.
1.1Affordances of Paper
Paper was invented in China in 105 A.D. It was only in the twelfth century that Arab traders brought it over to Europe. At that time mainly animal skins were used as writing surfaces. The cheaper but more fragile paper medium did not immediately replace animal skins but it took another three hundred years before paper became the major writing material. In 1440 Johannes Gutenberg developed the printing press and the demand for paper increased enormously since it was much better suited to printing than skins. Cotton or linen cloth waste, also known as rags, were used as the source material in the papermaking process. The invention of the printing process led to a shortage of rags in the sixteenth century and people started looking for other materials as a substitute. However, it took almost another three hundred years before wood pulp became commercially used for paper production in the middle of the nineteenth century.
The history of paper covers nearly two thousand years during which paper has seen many major technical improvements to finally become today’s highly optimised writing material. We can also see that adaptation from one medium to another, for example from animal skin to paper, did not take place immediately, but rather was a lengthy process. Often it takes a considerable amount of time before people get used to a new kind of medium. Further, a true evolutionary progress in terms of functionality of the new type of medium should manifest itself, which means that the new medium should not simply try to imitate the properties of its predecessor.
While there have been dramatic increases in the use of digital technologies for the storage, processing and delivery of information over the last two decades, the affordances of paper have ensured its retention as a key information medium. Paper has many advantages over digital media in terms of how people can work with it, both individually and in groups. It is portable, light, cheap, flexible and robust. The physical properties of paper support many forms of human actions such as grasping, folding, marking on etc. Furthermore, various forms of paper-based collaboration and interaction are nearly impossible to support in digital environments [98].
Sellen and Harper point out four reading-related key affordances of paper in the The Myth of the Paperless Office [156]. First, paper allows for quick and flexible navigation through a document with the size of a document being a rough indicator for the amount of information stored in it and the readers always knowing where they are while flicking through the pages. A second affordance of paper for reading is the possibility of marking up a document while reading. Free-form annotations help readers to mark important text passages for easier re-reading and to structure their thoughts. Further, Sellen and Harper mention an affordance which is based on the fact that the information on paper is fixed but still the paper documents remain mobile. It is possible to read across more than one document at the same time by placing multiple documents next to each other and thereby defining a spatial order on a work space. Finally, paper supports the seamless interweaving of hybrid activities such as reading and writing. For instance, by placing a document next to a notebook we can take notes while reading the document.
As an example of how we use different types of paper documents in combination with digital writing tools let us have a look at the process of writing this thesis. While writing this thesis various affordances of paper mentioned earlier were used. In addition to the portable computer that represents the primary writing tool, there are plenty of paper documents covering the author’s desktop as shown in Figure 1.2.
Figure 1.2: The author’s writing space
Behind the laptop there are four large folders of research papers about related work that have been read and annotated by the author over the past three years. On the right hand side there are three large piles of books classified by topics: books about information systems, paper, hypermedia and some philosophical and historical books. However, these piles are more than just a classification of different books. The order of the books within the different piles together with other contextual information represents the active process of thinking involved in writing the thesis. The paper notebooks that have been used for taking notes during the last three years and also some new research articles which needed to be read, so-called active or “hot” articles, lie in front of the book piles. Last, but not least, to the left hand side of the laptop there are two English dictionaries and some manuals for the LATEX typesetting system that has been used to write the thesis. While most of these documents would also be available in a digital form it is important to note that the task of writing the thesis would become much harder without the use of paper documents. The spatial layout of documents on the desk allows the author to easily switch from one document to another or use multiple documents next to each other. Further, all the open books and note pages together define the current state and context of the work. This awareness of spatial context is just one of the features that are very difficult to emulate in a digital world.
While typesetting systems on a computer provide great support in writing a manuscript, desks are often covered with paper documents that are consulted while writing articles. As mentioned earlier, it is no longer the case that the resolution on computer screens is not suited to reading documents digitally but other factors, such as the quick navigation among different documents and the flexibility of spatial layout, are critical in the writing process [134]. In addition, knowledge workers often cannot immediately classify a piece of information [81]. They use the physical space, for example their desks, as a temporary spatial state of their ideas and other information to be processed and classified. After a piece of information has been analysed and annotated by the knowledge worker, it may not be important that the handwritten marks are digitised in a way that computer tools can “understand” their meaning.
The marking up and annotation of content is supported in many different forms by paper. We can use a highlighter pen to mark important sections while reading a paper or use a pen to write comments in the margins of a document. There is a smooth transition from completely informal to more formal annotations. Free text or sketches are just two examples of simple informal annotations. The introduction of annotation guidelines leads to more formal annotations which are, for example, used in typographic markup. It is not easy to support the same richness of annotations in digital applications [109, 110].
Various workplace studies have been carried out to investigate the role of paper in working environments equipped with modern technology. For example, the use of paper flight strips in Air Traffic Control rooms was analysed in an ethnographic study by Mackay et al. [103]. A result of their studies is that paper flight strips support safe and effective work practices and that they offer a flexibility hard to achieve with digital systems [104].
In addition to these more evident features, paper provides other less apparent or “hidden” properties. Many processes in working environments involve paper as a medium for collaborative activities without having been specially designed for these tasks. Quite often new digital systems which should support our work activities miss some of these properties of paper for collaborative activities and therefore are less accepted by a specific target community. The detailed analysis of working environments may help in designing effective computer supported cooperative work (CSCW) environments paying attention to exactly these “hidden” properties. Heath and Luff undertook several field studies in different organisational environments [67, 101]. They found out that paper and screen-based media provide rather distinctive support for cooperation and that the use of paper persists not only due to its intrinsic properties or constraints of screens but also because paper affords interactional flexibility based on its mobility. For example, they describe how physicians benefit from the mobility of paper-based medical records. An outcome of their observations is that paper affords interactional flexibility for asynchronous as well as synchronous cooperation that allows individuals a range of ways of participating in tasks-in-interaction.
On the other hand, digital technologies have many advantages in terms of storing and accessing large amounts of information, displaying multimedia and fast full-text searching. Digital information is less static than printed information and can be dynamically updated with ease. Brown and Duguid emphasise the complementary character of paper and digital information when they talk about the digital office in The Social Life of Information [26].
Time has only confirmed this early indication of paper’s importance in the digital office. While other print technologies have come to compete with it, laser printer sales have increased twelve-fold in the past decade. If the digital office from PARC to the present is anything to go by, bits and atoms, the digital and the material, don’t seem so much in opposition as in tandem. Despite confident claims that their only relationship is one of replacement and dismissal, the two look much more like complementary resources.
Brown and Duguid, 2002
The dream of the paperless office has failed—at least until this day. However, we cannot foresee if the move from paper to digital material is something that will never entirely happen or a rather complex task that will take a long time to be fulfilled; maybe another three hundred years such as the change from skin to wood pulp for paper production. If paper cannot be replaced by digital media, we can still try to carry on its evolutionary process by integrating it more closely into digital information environments. We aim for a solution where the highly optimised properties of paper media which have evolved over the last two thousand years are augmented and enhanced with new digital features by tightly integrating paper and digital technologies, thereby gaining the best of the physical and the digital world.
1.2Integrating Paper and Digital Media
The digital era has already influenced and changed the traditional publishing industry. There has been a major shift in the use of paper. More and more publishers not only produce, but also distribute, their documents electronically. While electronic documents are easier to distribute and update, most users still prefer to read and annotate documents on paper. Some of the traditional publisher’s printing process has therefore been outsourced to the consumers.
As highlighted earlier, in the near future paper documents will not be replaced by digital information. However, we should at least try to reduce the gap between printed and digital information. If we find a way to address parts of paper documents, we can build new forms of interactive paper, where printed documents are augmented with supplementary digital information or services.
Over the last decade, there has been a major rethinking of how computers could help us in managing information and supporting our daily work. In the early nineties, Mark Weiser coined the term Ubiquitous Computing. Instead of trying to digitise our physical world and using computers mainly as storage components, small “computers” could be embedded in everyday objects to make them more powerful in terms of connectivity and information exchange with surrounding objects and the environment. In his article The Computer for the 21st Century [181], published in the Scientific American, Weiser describes a scenario of how intelligent paper could be integrated into future working environments.
At breakfast Sal reads the news. She still prefers the paper form, as do most people. She spots an interesting quote from a columnist in the business section. She wipes her pen over the newspaper’s name, date, section, and page number and then circles the quote. The pen sends a message to the paper, which transmits the quote to her office.
Weiser, 1991
It is not only the case that physical artefacts may somehow be connected to, or enhanced by, digital information but they may also be used as input and output devices. The resulting tangible user interfaces (TUIs) are a new form of interacting with computers in a more natural way by electronically coupling physical and digital objects. The user no longer has to work with a digital representation of a physical concept. They can work directly with the physical object that has either some embedded computing power or is connected to a server performing the requested operations. Durrell Bishop’s Marble Answering Machine [76] represents an early project in the area of tangible user interfaces. Incoming telephone messages are “physically instantiated” as marbles by the answering machine. The user can get access to the messages associated with single marbles by grasping a marble and dropping it into the designated slot on the answering machine. A benefit of tangible and ubiquitous computing in terms of work activities is that it can help to reduce the interruption that results from switching from physical to digital information and vice versa. The idea of interactive paper fits well in the scenario of ubiquitous and tangible computing where paper becomes a physical interface to interact with digital information.
The issue of architectures and technologies for the integration of existing data sources has also been a topic of interest for many years within the database community. A number of solutions have been proposed with the approach taken dependent on the degree of heterogeneity, the level of local autonomy and, of course, the required functionality of the target system. In terms of heterogeneity, many forms of data sources have been covered including different categories of database management systems, knowledge base systems, file systems and web documents. But one major form of information source has so far been neglected, namely paper documents.
The integration of paper and digital media involves two main steps. First, a method to link paper to digital actions is required. A lot of effort has been put into the development of new technologies to encode information on paper and capture information from physical paper. We will present many of these technical solutions when discussing related work in Chapter 2.
In a second step, the information acquired by a specific hardware solution is used to access the appropriate digital information or service. This implies the availability of a software infrastructure for the actual link and information management. Too often, existing solutions focus on the development of the hardware and tend to neglect the underlying information infrastructure. In contrast, we turned our attention to the development of a powerful and flexible cross-media information management architecture. The focus on basic cross-media information management concepts naturally leads to generic and powerful concepts for interactive paper as well as the integration of other multimedia resources. Different hardware solutions can then be integrated into the general cross-media information management architecture. This data-centric approach for interactive paper facilitates adaptation to evolving technologies and reduces the risk of a dead end by choosing the wrong hardware solution.
The question remains as to the potential uses and models of interaction afforded by interactive paper. How can we fully exploit the potential of paper as a client device and integrate it seamlessly into semantically rich digital information spaces? While a number of research projects have proposed different variations of interactive paper and presented their visions for its use, none have really addressed the issue of making paper a first-class medium in the context of an interactive information system. This means that it should be possible to link, not only from paper to digital resources, but also from digital resources to paper and even from paper to paper.
The hypertext visionary Vannevar Bush discussed the problem of information overload already sixty years ago in his article As We May Think [28] published in the Atlantic Monthly. In this article, Bush described a mechanical device, the Memex, in which knowledge workers could store their books, records and communications in microfilm and retrieve them later with appropriate speed and flexibility. The important thing about the Memex was that information could, not only be stored, but also associative links could be defined between different pieces of information. He described the job of a trailblazer who was responsible for linking pieces of related information together. However, Bush never proposed using the Memex primarily as a writing tool. His initial idea was that people would still write linear texts but could later link them to specific parts of related texts as part of a hypertext authoring process. The Memex was intended to support the definition of edges in a graph where the nodes were built from existing text documents, but not to support the actual authoring of a document’s content. The knowledge worker could build their personal associative information space which would support them in later knowledge retrieval. In addition to the definition of links, represented by edges between existing nodes, hypertext writing systems such as Eastgate Systems’ Storyspace [169] provide an integrated solution for authoring textual content as well as connections (links) between text fragments. Nevertheless, in hypertext literature there is a clear distinction between the writer of a hypertext and its reader. The hypertext narrative writer clearly defines the paths that can be taken between different fragments of text and the reader can only choose among a set of preauthored links.
Today’s information infrastructures often restrict the authoring process to specific users. For example, if we take the traditional Web, only a person who has write access to the original HTML document can change the web page and add new content and links to other digital resources. We aim at a more open authoring process where each reader may also become an author and therefore dynamically change the hypertext structure. Such a collaborative authoring process can always be combined with information provided by a content publisher who is an expert in their specific domain. Of course, an open and democratic authoring process introduces new problems concerning the quality of information which may be corrupted by users publishing incorrect information. However, in our opinion, the positive effects of collaborative authoring processes in a strong community outweigh the negative consequences of potentially false information published by a few malicious individuals. A positive example of such a new form of an open web publishing platform is the free Wikipedia encyclopaedia [187], a fast-growing knowledge resource, where everybody can add new, or edit existing, information.
Our approach for interactive paper and more general cross-media information spaces is somehow similar to Vannevar Bush’s idea of the Memex which should help in organising information by aggregating meta information in the form of links about related information entities. We do not want to replace paper and accept that the linear writing of original texts is preferable in many situations. However, our architecture for interactive paper and cross-media information management should help to organise and manage information more efficiently by annotating linear paper documents with more dynamic digital media or linking them to specific parts of related paper documents. Our goal was to develop a general platform for cross-media information management, called iServer, providing fundamental concepts for linking different information entities. These link concepts are general enough to support the full range of existing hypermedia solutions including, for example, spatial hypermedia, temporal hypermedia and physical hypermedia. Application developers can use the iServer platform to manage information across different types of media and write their own extensions in the form of resource plug-ins to support new types of media.
1.3Contribution of this Thesis
Over the last twenty years we have seen a dramatic increase in the use of digital technologies for information storage, processing and delivery. However, despite technical advances in digital information management, paper is still the key information medium for many work activities. It is only recently that a new research area dealing with the integration of paper and digital information or services has formed. However, most of the interactive paper projects that have been realised so far, focus on specific hardware solutions for linking paper documents and digital information thereby often neglecting the information management part. This leads to isolated solutions which work only for very specific application domains and clearly defined hardware setup.
In this thesis, we show that by approaching the problem of implementing interactive paper from an information point of view, we can take the integration of the physical and digital information spaces one step further and realise highly-connected information environments that can link together elements of arbitrary resources and enable users to move freely back and forth between physical and digital information sources. As will be shown in this thesis, the potential functionality and flexibility of a cross-media information management system is determined by the underlying information model and architecture.
We identify fundamental concepts for cross-media information spaces, including interactive paper documents, and propose a general link model for cross-media information management bringing together many existing hypermedia concepts. Future extensibility is granted by a resource plugin mechanism ensuring that new digital as well as physical resources can be easily integrated and, more importantly, cross-linked to the growing set of supported multimedia resources. Furthermore, we have integrated a layer and a user management component into the very core of the link model to manage link access rights and therefore enable applications to control the visibility of links in both a user- and context-dependent manner. Associative inter-application and cross-media linking as supported by the model enables effective information management across isolated application domains.
The general cross-media link model has been implemented by extending an object-oriented database management system with the required functionality. The resulting iServer platform introduces fundamental link concepts at an abstract level. Only a small media-specific part has to be implemented to support new resource types whereas the iServer’s core link management functionality is available across different multimedia resources. In addition to the associative linking of information, our solution allows for the integration of semantic metadata and the multiple classification of information units. Further, iServer can, not only link between various static information entities, but also link to active components, which are small pieces of program logic executed on link activation. The active component paradigm has proven to be very effective in enabling more complex interaction design.
A modern information infrastructure has to be able to cater for all forms of mobile devices, user preferences and context-dependent delivery. It is a matter of delivering the right information, to the right person, at the right time. The eXtensible Information Management Architecture (XIMA) is a web publishing framework ensuring that all the link metadata stored in the iServer framework can be delivered on different output channels. Based on a clear separation of content and presentation, it enables documents to be generated dynamically to suit client device, user and context.
An iServer plug-in for interactive paper (iPaper) has been developed to fully integrate paper and digital media. The interactive paper platform supports enhanced reading, where the active reading process [2]—a combination of reading with critical thinking and note taking—is supported with supplementary digital information. Furthermore, it enables the active, paper-based capture of new information, known as enhanced writing. Based on a set of abstract interfaces, the interactive paper platform is very flexible in supporting various forms of input devices.
To demonstrate the applicability of iServer and the iPaper plug-in to different application scenarios, various prototypes have been implemented interweaving paper and digital information. The flexibility of the iServer resource plug-in mechanism has been verified by implementing three additional resource plug-ins to support HTML content, movies and Radio Frequency Identification (RFID) tags as new specialised resource types.
The interactive paper authoring tool which is based on iServer authoring functionality allows active areas on paper to be defined and linked to digital or physical objects. Link authoring and content authoring are introduced as two possibilities for integrating paper and digital information and solutions for preauthored, personal and dynamic links are presented. Since the authoring of cross-media information spaces by a single publisher has proven to be a time-consuming and expensive process, we have investigated alternative authoring paradigms where much more power is given to the user and each user becomes a potential author of new link knowledge. To support this collaborative form of link authoring and sharing, we have implemented a distributed version of the iServer platform based on decentralised peer-to-peer (P2P) technologies. In this collaborative authoring process, each user has their personal link information space which can be dynamically enriched with link information from other users. By applying such an adaptive community authoring process, each user can contribute information according to their domain of expertise and, on the other hand, profit by the knowledge of experts in other domains.
1.4Paper++ Project
The iServer platform and its iPaper plug-in for interactive paper were developed within the European Paper++ research project (IST-2000-26130), which was part of the Disappearing Computer initiative. The goal of the Paper++ project was to integrate paper and digital information and thereby bridge the paper-digital divide.
Paper++ was a multinational project with collaboration from King’s College London, HP Laboratories Bristol, ETH Zurich, Anitra Medienprojekte GmbH, and Arjo Wiggins. Our project partners worked on position encoding patterns and the corresponding hardware to read these patterns, new printing technologies, interaction design and user studies concerned with the general use of paper as well as new interactive paper applications. In this multidisciplinary team of professionals, it was the task of the author of this thesis to design and implement the necessary information management infrastructure for integrating paper and digital media.
As part of the Paper++ project, we implemented multiple demonstrator applications. These applications included an interactive nature encyclopaedia, an interactive city map of Zurich and an interactive worksheet for the Natural History Museum in London. Our project partners helped in the design of these applications and evaluated them in user trials to verify the feasibility of our approach. After the completion of the Paper++ project, we developed various other applications based on the interactive paper software platform. These applications are no longer based on the hardware prototypes that were developed within the Paper++ project but on commercially available digital pen and paper technology.
1.5Structure of this Thesis
In Chapter 2 we discuss different approaches for the integration of paper and digital information and cross-media information management. We introduce the basic actions required for integrating paper and digital information such as document identification and the capture of handwritten information. Various technical solutions for solving these tasks are presented and advantages and disadvantages of specific technologies are discussed. Existing interactive paper projects using one or more of these technologies are presented and classified by the type of actions they support. We give a critical analysis of existing work in the area of interactive paper and present our hypothesis proposing a set of fundamental concepts for a general cross-media information management platform.
Based on the analysis of existing work in Chapter 2, we present a general model for cross-media information management in Chapter 3. The new cross-media information model introduces associations between different physical or digital resources at an abstract level without making any assumptions about a specific type of supported media. The information model builds the theoretical foundation for the cross-media information platform and the interactive paper framework presented in subsequent chapters. In addition to the new cross-media model, we introduce some basic terminology which is necessary to compare our approach to existing solutions.
Chapter 4 introduces the Java implementation of our cross-media information platform (iServer) and the corresponding Application Programming Interface (API) which is available to applications built on iServer. We show what is involved in integrating new media types into the iServer architecture using a resource plug-in mechanism and, in addition to the interactive paper extension, present three plug-ins for HTML documents, RFID tags, and QuickTime media types including movies, sound files and still images. In addition, the application of the iServer framework for cross-media annotations is discussed. We show that annotations are just a special type of links where the creation of a link often also involves active capture of information and point out that the iServer platform is well suited for this task.
In Chapter 5 we outline the requirements for tightly integrating paper and digital information and introduce the main components of our architecture for interactive paper. After presenting the client and server components, we describe the address space mapping component which is used to decode the positional information encoded in the paper documents. The reader interface which makes it possible to integrate different kinds of input devices is presented before describing the OMS Java database system used for information storage. Finally, we present the XIMA framework which is applied for visualisation and supports universal access to the digital information managed by the interactive paper framework from a variety of client devices.
Various applications that have been developed to evaluate specific aspects of the interactive paper framework are presented in Chapter 6. This includes applications for enhanced reading as well as enhanced writing. Furthermore, the implementation of multiple applications helped in validating the flexibility of our cross-media information management platform to support a variety of paper-based interactions as well as different hardware settings.
Different kinds of cross-media authoring are introduced in Chapter 7. In addition to the screen-based generation of interactive paper documents, links can also be created directly on paper documents using writing capture. After presenting our authoring tool for link authoring we discuss the potential of semantic content authoring based on content management functionality. We then introduce collaborative link and content authoring based on a decentralised peer-to-peer version of the iServer platform.
We outline how issues of information semantics and granularity have an impact on the richness of the resulting interactive information environment. However, everything comes at a price, and we discuss the consequences for the content provider in terms of complexity and cost. These richer information environments could require a major shift away from the traditional practices of publishers towards the content management solutions that have already been adopted in the web publishing world.
Finally, in Chapter 8, we summarise the outcomes of this thesis and provide a critical analysis of the results achieved. Last but not least, we point out directions of future research in the area of interactive paper and cross-media information management.
A room without books is like a body without a soul.
Cicero
2
Background
In this chapter we give an overview of existing technologies dealing with the integration of paper and digital information and present various interactive paper research projects addressing different tasks and application domains. Before discussing related work, we provide some basic link terminology and outline minimal requirements for integrating paper and digital information.
Rather than presenting a historical review of related work, we classify the different projects according to their functionality and the working tasks they support. We start by discussing projects dealing with enhanced reading, where the focus is on reading digitally augmented paper documents. We then shift to projects where the writing process is enhanced by digital services. In addition to enhanced reading and writing, we present projects supporting annotation activities, which can be characterised in terms of the way that they combine reading and writing.
We then discuss projects where paper can not only be linked to additional digital information but also applied to trigger specific actions of a digital application workflow. In this case, paper becomes a tangible user interface for specific digital applications and may even replace traditional input devices such as the keyboard or the mouse. Finally, we analyse the presented projects, provide some concluding remarks about the flexibility and extensibility of existing interactive paper solutions and state our hypothesis for new forms of interactive paper based on a general cross-media link model.
