World without history? Digital information is volatile: with it our culture can disappear but its preservation can save us E-Book

Stefano Cariolato

• Herausgeber: Youcanprint
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch
• Veröffentlichungsjahr: 2019

Beschreibung

In the mid-twentieth century the digital revolution began with the introduction of the first electronic computers, which were first introduced into companies and in the state bodies then they spread strongly in the private houses as personal computers; later all these computers were connected to each other by a global telecommunication network called Internet, which had a massive development at the end of the century becoming the backbone of the worldwide information circulation.
At the beginning of the 21st century the digital revolution was completed and the information of any kind (texts, images, video clips and TV broadcasts, music and songs, WEB pages) started to be recorded and disseminated in digital form rather than with a traditional media (paper, film, magnetic tape), with a displacement that engaged all human activities of any type, both collective and individual.
While the development of digital technology continue at an accelerated pace the problem of information retention begin to arise, what was previously mainly entrusted to printing on paper and now is in abandonment phase: printed records are increasingly transformed into digital format and the new information is generated directly in electronic form. But while a book or a letter could be read directly even centuries after their writing, digital information has a short life because of the same technological development, that makes quickly obsolete any recording by irreversibly mutating both its hardware and reading software; other recordings arethen volatile by their very nature, such as e-mails or WEB pages, even if they could host information that could be of value in the future.
Moreover digital recordings are carried out in a great variety of different formats, sometimes incompatible with each other or subject themselves to obsolescence, thus unnecessarily complicating the task of preserving their content.
Most part of human culture, gradually poured into electronic form, is now jeopardized, and we risk of delivering to posterity a world without history: this book describes the current situation and what is sought to do to remedy the danger.

Leseprobe

WORLD WITHOUT HISTORY ? 

STEFANO CARIOLATO

Digital information is volatile: with it our culture can disappear

but its preservation can save us

1st Ebook Edition

cover: old man from Pixabay

to Roberta

PREFACE

In the mid-twentieth century the digital revolution began with the introduction of the first electronic computers, which were initially employed into companies and in the state bodies, then they spread strongly in the private houses as personal computers; later all these computers were connected to each other by a global telecommunication network called Internet, which had a massive development at the end of the century becoming the backbone of the worldwide information circulation.

At the beginning of the 21st century the digital revolution was completed and the information of any kind (texts, images, video clips and TV broadcasts, music and songs, WEB pages) are now recorded and disseminated in digital format rather than with a traditional media (paper, film, magnetic tape), with a change that involved all human activities of any type, both collective and individual.

While the development of digital technology continues at an accelerated pace, it is arising the problem of information retention, previously mainly entrusted to printing on paper that is now in abandonment phase: printed records are increasingly transformed into digital format and the new information is directly generated in the electronic form. But while a book or a letter can immediately be read even centuries after their writing if paper support has withstood the time passing, digital information still has a short life, even in the absence of deterioration of the used media, because of the same technological development, that makes quickly obsolete any recording by irreversibly mutating both the reading hardware and software; other recordings are also volatile by their very nature, such as e-mails or WEB pages, even if they could host information of some value in the future.

Moreover digital recordings are carried out in a great variety of different formats, sometimes incompatible with each other or subject themselves to obsolescence, thus unnecessarily complicating the task of preserving their content.

To be even clearer, suppose we distinguish the culture in two distinct portions:

- "Current culture" or "running culture", that is, the set of data and concepts usually and commonly used in everyday human activities in all field of activity; this entire set until yesterday was normally contained in traditional format (paper, film, vinyl, etc.) and today also in digital format, and since it is widely used it is also adequately maintained and continuously reproduced in copy.

This process of continuous cultural preservation will also be active in the future, precisely because it will continue to be a patrimony used daily both in the scientific-technological field and in the  production of goods and services , even if tomorrow will be mainly available in digital format and will obviously be made up of future data and notions.

Types of support and reading devices will be those proposed by the technological development of the time and adequately produced by the industry, so as to keep intact the "current" culture of each historical period.

- "Historical" culture, that is the set of data and concepts not included in the "current" culture of each period because they are no longer or only rarely consulted, such as a good part of history, literature, poetry, music and scientific, journalistic, corporate or institutional documentation of the past, as well as all the notions and data that refer to obsolete and no longer used technologies. It will be objected that this has always happened, the world moves on the crest of a cultural wave leaving behind what is no longer needed. Who cares about the novel written in the eighteenth century by an obscure author little read even in his time? Who cares about Charles Guiteau, that in 1881 altered American History shooting the President Garfield ?

However, there is a big difference because the information on the past still exists, mainly in a directly legible form, and it's jealously kept in public and private libraries all over the world or even in homes; we also add information on microfilm of newspaper archives, movies on film and musical compositions on vinyl or tape, even if they need suitable reading devices, which brings them closer to digital recordings and the problems that we intend to deal with.

If we do not find the way and the will to preserve the content of digital documents, similarly to what was done in the past with paper documents, gradually the "historical" culture left in each period to the descendants will diminish until it shall disappear. A story without a future would create a future without history.

Normally we are not very worried about the conditions in which our descendants will live, children apart, and therefore this type of problem is not intended to mobilize public opinion and consequently politicians, but we should remember that we did not have to invent the wheel because others had done it before us; if we are grateful for this then we can perceive the duty that we have to leave the maximum to future generations, who have the right to inherit the result of our life and all previous ones.

All human culture, gradually poured into electronic form, is therefore at  risk of disappearance, risking to give posterity a world without knowledge and history: this book describes the current situation and what we are trying to do to remedy the danger. The future will reserve us great problems and great challenges, which as usual will be faced when they will be fully manifested; the problem that will be represented by the documentary conservation, however, will be the only one whose solution can be found in the past, that is today. Our future is an extension of what we choose to do now.

The reader will quickly realize that, while the banal and succinct exposition of the problem is perfectly and easily understandable, the deepening and the technical examination of this issue are much less, also because of the confusion and widespread inertia existing in this regard that generate a series of initiatives,  disconnected from one another and often transitory.

For this reason and for ease of reading the volume has three levels of analysis:

- the introduction of each chapter, which summarizes its contents in a concise and panoramic form;

- the following paragraphs which examine in detail the various topics dealt with;

- the technical appendices that further investigate the technological and scientific aspects involved and therefore require at least a superficial familiarity with computers and other sciences.

Each interested person can then modulate the reading according to his own interest and technical culture, eventually devoting a first overview to the more general level and then deepening the various topics covered and verifying their different technical aspects.

One last warning: if you finish the reading worried and confused it will be because this book has perfectly achieved its purpose, that is ringing an alarm bell to you.

* As the author is not an English mother tongue, the text may sometime have a weird taste to you, bound to raise a few eyebrows and to think " With the greatest respect .....".

So the author begs your pardon for possible mistakes and, counting on your good will, hopes that it be at least understandable and possibly clear, what was his true target.

Thanks

INTRODUCTION

Since prehistoric times man has left signs of his world, his culture and his history, at first in a naive and graphic form with the paintings in caves that he lived in or with graffiti carved into the rock like those of Acacus, basically representing the environment in which he lived and his activities; later on with the invention of writing the human capacity to record and exchange information underwent an extraordinary development, and even though it was originally destined to carry out government or trade activities of that time, it allows today to know the main cultural traits and to reconstruct the historical events of ancient civilizations.

Lion drawings from Chauvet Cave in Southern France from the Aurignacian period (30,000 years old)

Obviously this is possible as long as these ancient recordings are rediscovered and are still legible, which depends both on the luck and patience of the archaeologists and on the nature and resistance of the support used for the purpose.

Nine mile canyon, Hunters Petroglyph Panel

Petroglyph of a giraffe, in Tradart Acacus, Fezzan, Libia

As a matter of fact information has ever two distinct aspects, that is its abstract meaning and its corresponding materialization on a readable support: it cannot be communicated without its materialization in a physical form. 

Starting with paleolithic graffiti engraved in caves, followed by inscriptions on stones, cuneiform scripts on clay tablets, texts on vellum or silk and papyrus rolls, ending with paper, ever we have registered information we need in a persistent form immediately readable by a human being knowing that type of writing. 

   Cuneiform script

   Rosetta Stele

   Papyrus writing

Tombstone of Bescanuova, written in the Croatian Glagolitic, around 1100, island of Krk

    Historiated psalter on vellum

Availability of information depended on the relative support durability, with a maximum of tens of thousands years for graffiti, diminishing with supports changes through history, ending finally with a paper minimum. For maintaining information availability we adopted also the method of copy and diffusion of the content to be preserved. We owe to the extraordinary duration of the ancient supports and to the monks activity of copying the survival, through the Middle Ages, of a large part of the culture both of that period and the previous one: the author was struck by seeing, for example, in the library of the Abbey of Novacella a perfectly preserved parchment bearing the seal of Federico Barbarossa (second half of 12° century).

Frederick Barbarossa parchment

Obsolescence of information was therefore the same of the support lifetime, obviously in addition to language understandability.

Now technology has given us what is widely called "digital support" (magnetic, optical and solid state memories), the use of which is much more practical and quick than that on paper, as well as allowing both rapid searches and immediate copy of contents. 

But with a flaw, that is it is not any more immediately readable without an appropriate device and reading software. 

  

    Laptop

Availability and usability of these devices and software then impact on persistence and readability of information, thus introducing another factor of obsolescence we have to consider.

While electromagnetic support during communication by radio, tv, Internet transmissions is by definition temporary, other types of "permanent" support allow durability and readability over the time. It is therefore necessary to ask the question of how to keep this information, in order to preserve its content and make it available continuously also in the future. Digital preservation hence is the active management of digital content over time to guarantee continuous access to it.

In 2016 Vint Cerf, who built the internet from its roots with DARPA, said to be deeply concerned about the record we will leave to future generations about our time. “If 100 years from now the digital picture of our society is not accessible, we will be an enigma to the 22nd century,” - Cerf warned -  “I’m very concerned that digital content will be less and less accessible, not because we can't find the bits, but because we don't know what the bits mean.” 

Digital preservation was born in a long super-economic cycle, so long and so great that talented economists congratulated themselves on an endless boom. They were wrong, for a deep economic crise followed.

Digital preservation has not been immune from the resulting chaos. Normally it is hard to convince anyone to invest in something, but it takes a bit of courage to ask for an investment in something as exotic as the data. Take for example the general blocking of hiring and redundancy layoffs that followed the banking crisis. Instead of hiring novices with new and passionate digital skills, employers in the public and private sectors have begun to lose many thousands of archivists, conservators and librarians in about 18 months. The normal cycle of employment has stalled and so, instead of recruiting new resources, the generation of digital archivists rehired has in many cases been selected by the unemployed pool. Unfortunately, the main projects in the field of digital preservation have suffered exactly when they were most needed, a strange and unpredictable consequence of austerity. The current problems of digital preservation are not just about data volumes and workloads, but they also obviously depend on the availability of economic resources, to be allocated to an objective little felt by the public opinion and therefore also by the politicians who have the habit of supporting mainly issues capable of intercepting an immediate consensus.

1   DIGITAL DOCUMENTS

From now on, the extended document definition will be used, meaning any form of information recorded on any physical medium.

Digital documents, regardless of their specific content (texts, images, movies, music, etc.) have now become the new recording standard: in fact most of the new documents produced are directly recorded in digital form (text files, image and TV formats, WEB pages, music formats) and in most organizations, both private and public, the digitalization of paper documents is underway.

All this brings to the fore the problem of the conservation and archiving of these documents, whose consultation depends forcibly on the availability of hardware and software suitable for their viewing and listening. If this is not possible then the document content is virtually lost, which is defined as digital obsolescence, and is added to the material obsolescence of the physical support (partial destruction) and the immaterial obsolescence due to the incapacity of understanding (language now abandoned or unknown).

With the term digital obsolescence we therefore refer to the possibility of losing the information entrusted to a digital resource, since the latter is no longer legible for a number of reasons:

Hardware Obsolescence

When a technological innovation leads to the substitution of one type of support with another more efficient (e.g. cassette tape >> CDs> solid state memory), the industry ceases to produce the relative reader, which consequently becomes increasingly rare and eventually it disappears altogether.

The display hardware becomes unavailable for the disappearance from the production of the player such as happened for the 8 "floppy disks and its readers, now unavailable, or for the replacement of music cassettes with music CDs before and today for the replacement of CD players with flash memory readers in listening equipment. In addition to this, archiving hardware and its supporting software for reading stored data are subject to very rapid obsolescence.

Software Obsolescence

Entrusting the documents to be conserved in the long run to proprietary software (think of a trivial Microsoft Word file), ties the use of the data stored to the fate of the software itself and the interoperability between versions, often absent for marketing reasons: in fact if the new version of the software is incompatible with the previous one the customer is forced to replace it to keep up-to-date, with a consequent benefit for the sales of the producer but problems with access to old documents by the user. Another example of software obsolescence is, for example, the abandonment of a particular recording format and the consequent unavailability of programs able to directly decipher the document content, such as Visicalc electronic sheets which can now only be read with the use of an emulator.

By way of example, two images are shown below, the first represents the actual content of a Microsoft Word text file, the second is its correct visualization through the program.

Actual doc file content

doc file visualization

What could we understand about the text without the special program? Nothing.

For these reasons, digital document preservation must take place using specific techniques and using formats that do not suffer from digital obsolescence or which can easily be remedied.

From this point of view the types of digital documentation can be divided into:

1) Documents produced with proprietary programs and formats, whose destiny in terms of permanence and compatibility depends entirely on the sales policies of a single producer. They, on the other hand, correspond to some of the most durable and famous software on the market. Two of the most widespread problems they present are:

the lack of public specifications that describe their structure and format in detail, which prevents others from reconstructing in the future the document content if the producer has abandoned the product with which it has been written, the fact that often the successive program versions do not guarantee compatibility with documents produced with their previous versions (backward compatibility).

In fact, they tend to evolve rapidly and to be declined in numerous versions for different computer environments, with a deliberately limited backward compatibility.

2) Documents produced with programs and in proprietary formats but with public specifications, which allow other suppliers or organizations to develop software that can use them.

3) Documents recorded using formats with specific open standards, such as those of international standardization bodies (ISO), whose adoption by users and producers is though hindered by the needs of commercial market protection by large companies and also by States for security reasons in the case of "sensitive" documents.

The digital preservation of documents must therefore take into account these considerations to preserve their content over time, independently of the inevitable technological and market developments. More on Preservation generality in the appendices to the chapter.

Documental digitalization

It consists in recording a paper document on digital media: this transformation is currently under way in many companies and public bodies, both to align the historical archives with the new information generated directly in digital format, and to be able to use its greater functionality, in terms of search and copy, even with documents previously recorded on paper. Digitization is also a practice that opens up enormous possibilities for historical research, such as automated quantitative text analysis (the now famous text mining), or the integration of archival catalogs and libraries into a single standard. 

These archives can be proprietary or public, local or registered on external sites (cloud). The latter in particular find particular success, as they are immediately available via the web - thus favouring the use of mobile equipment - and exonerate the customer from the purchase, management and maintenance of important and expensive IT resources, making it interesting for both private users and companies. They can be of three types, public, private or hybrid, that is to say they can be managed by external third parties that offer a service, or managed by the same company that uses them sparing the expensive dispersion of resources in the offices, or finally with a mixed structure according to the type of document concerned. Even for them the problem of documentary conservation arises, at least in part, but it does not seem that it is considered an important element alongside the others that contribute to determining security, efficiency and operating costs. In most cases the old paper copies are destroyed, and if the new recordings will not be properly stored and made legible with continuity, the relative contents will be completely lost. Digitization with time will therefore be complete and paper documents will disappear everywhere, except in libraries and in private homes.

States also intervene in this process, with appropriate laws aimed at regulating and preserving digital content, both those own and private documents concerning relations with the Public Administration (eg tax documents). But not all States, however, trust this profound transformation, and with a stubbornness that may seem anti-historical, continue to preserve the most important documents, both from a legislative and historical point of view, on very long-lasting supports, such as parchment. In fact, Great Britain continues to record and keep its laws on veal or goat's parchment, both in homage to a long tradition and as a testimony to the existing mistrust toward new technologies. The parchment has proven to last for centuries, and even today there are copies of the Magna Charta, such as the Brudenell, written in 1297. In the Victoria Tower of the Palace of Westminster the scrolls of parchment of English laws, both old and current, are still preserved in a vast warehouse, as if the time had never passed. This remind us of the words that 500 years ago Johannes Trithemius said about printed texts: "The word written on parchment will last a thousand years. The printed word is on paper. How long will it last? The most you can expect a book on paper to survive is two hundred years. Yet, there are many who think they can entrust their words to paper. Only time will tell."

Digital obsolescence

Recently, NASA seems to have recovered the availability of a scientific satellite, launched in 2000, which had ceased to function.

This satellite, called IMAGE, was designed to visualize the terrestrial magnetosphere and produce the first complete global images of the plasma present in this region of space. After completing its initial two-years mission in 2002, the satellite was no longer able to contact the base on December 18, 2005. On January 20, 2018, an amateur radio operator again intercepted signals from this satellite, and it was later confirmed by NASA, which however immediately found itself in difficulty: IMAGE is now obsolete. NASA can not decode the data contained in its signals. The types of hardware and operating systems used in the IMAGE Mission Operations Center from 2000 to 2005 no longer exist, and other systems have been upgraded to different versions later than those operating at that time. The recovery of information and the ability to control the devices on the satellite now requires a significant and costly engineering effort. The NASA team was able to read some satellite maintenance data, confirming that at least the main control system is operational, but the scientific payload of the spacecraft, still impressive and of great scientific importance, can not be decoded due to the obsolescence of hardware and software systems installed on board.

This is an example of digital obsolescence, both of hardware and software, which occurred in just 13 years, from which an important scientific damage has resulted: remedying it will cost much more money than having somehow maintained contact capacity with IMAGE, maybe even just with an emulator, or even better to have at the time used design methods that took account of this unfortunate possibility.

Another very similar case that once again involved an organization of a high technical and organizational level such as NASA, was that of the tapes containing the recordings of the probes that the astronauts left on the Moon to record the temperatures. They were not all archived but went missing, and when forty years later the scientists wanted to study some anomalies already detected they had to look for them everywhere, to find only a part of them and realize that the tapes were partially degraded and that the data structures contained were now incomprehensible. It took then seven years of study to come up with a problem that would have been easy to avoid.

One can not therefore underline more the important and pressing need to reflect on the issue of digital preservation and its technical aspects, starting at the same time those initiatives that can protect us from such negative events in the future.

Obsolescence of the support

We can not avoid talking about the more traditional obsolescence phenomenon, that is the mere physical corruption of the recording medium, either magnetic, electric or optical. Thus, the figures here expressed are very general guidelines. The only true way to protect data is to have multiple copies of everything, and the best way to do that is to invest in a good backup solution. There are many examples of problem caused by a lack of attention to data preservation. One key example was NASA’s Viking Lander data. Two landers were sent to Mars in 1975: datasets were compiled by scientists based on the collected data. The resulting data was stored on magnetic tape, in climate controlled conditions, but despite this, the physical tapes deteriorated. In addition, by the late 1990s, scientists were unable to decode the data format. This data was ultimately recovered by re-entering it from microfilms and printouts. 

The digital age has revolutionized the way we handle information. Never before could humankind record and store so much information and in such diversity. While the amount of data has increased exponentially, the predicted life span of the used storage media is still a matter of lively debate, also complicated by the running technological development. 

Normally the producers communicate their - optimistic - evaluations on the duration of the recordings made on the marketed media; they do so in terms of "average duration", which is the time it takes for 5% of the registrations to become illegible. This evaluation, however, is totally useless and misleading when it comes to judging the guaranteed duration of a digital document, because what interests is the "minimum" time within which any recorded document can become partially illegible, corresponding to its actual maximum duration as an information element. In fact, the preservation of documents requires perfect and complete readability, having the appropriate reading device, of any document belonging to a collection subject to aging phenomena, that by their very nature are completely random. It is clear that in the absence of better we must be satisfied, even paper documents can become partially illegible with time, but if this happens in a few years instead of a few centuries we will have a problem.

The second observation regards the technical differences existing between the different types of support, better specified in the appendix, which however involve profound differences in the respective durations of the documents contained. Leaving aside the case of magnetic units, already launched for abandonment as digital media, it is important to clarify the profound differences between the classes of optical units, namely:

- CD-ROM and DVD-ROM discs, read-only supports produced by industry using mechanical moulding techniques, which due to their intrinsic nature guarantee relatively high retention times for the documents recorded therein, provided they are kept in optimal environmental conditions.

- CD-R and DVD-R, recordable media in which the bits composing the document are recorded in the form of burns or stains on films chemically sensitive to heat, subject to phenomena of natural aging and gradual cancellation, as in the case of photocopies or of old photographic prints. For these supports obsolescence is much more rapid and strongly influenced by environmental conservation conditions.

- Various types of CD-RW and DVD-RW, rewritable media multiple times that use as a sensitive layer of metal compounds, which under the action of writing laser melt and pass from the crystalline state to an amorphous state. The operation of eventual cancellation, much slower, is also performed with the laser. However, this sensitive layer is very subject to oxidation phenomena and to the negative influence of non-optimal environmental conditions.

It should be added that the characteristics of recordable discs produced by the industry are extremely variable, from the qualitative point of view, not only between a brand and the other, but also between a disk and the other of the same type and of the same brand. Ultimately they do not seem to guarantee the durability characteristics required by archival conservation, which obviously can not use the mechanical molding systems reserved only for large-scale productions.

Magnetic supports

Magnetic tape can either lose data by losing its magnetic charge (any magnetically charged storage medium will eventually lose its magnetic charge and subsequently its data), or when the layers of the tape start to separate. According to a handful of sources manufacturers claim that tape can last up to thirty years. This can make it a useful medium for archiving. The problem with that number is that magnetic tapes will only last that long under absolutely optimum environmental conditions. That means to keep magnetic tapes in a place where both humidity and temperatures are stable. A more realistic lifespan for magnetic tapes is about ten to twenty years. Obviously they are more susceptible to wear and tear if used frequently.

Floppy disk

Floppy disks were never super reliable, and some didn’t even work quite properly right out of the package. There are appraisals saying that the lifespan of floppy disks is three to five years. But there also others that claim they can last ten to twenty years. Of course, since floppy disks utilize magnetic storage (not unlike tape), it’s safe to say that eventually the magnetism will wear out around the same time a tape’s would (ten to twenty years). That’s if the cheap, flimsy plastic casing on the disk survives that long.

Hard disk drives

Hard disk drives (HDD) can last more than ten years before some component fails. That doesn’t always mean the drive is irrecoverably busted. But ten years is still about how long they normally last, either for  internal drive for a server or desktop, or an external unit. With all of the moving parts inside, something will eventually stop working. Lower rpm disks probably will last a little bit longer, but as with any media storing data it’s important to use quality hardware.

Optical discs

The long-term reliability of optical discs is still unknown, so there are no certain answers. 

CD-R discs, which appear to be the best bet for data registration lasting 10 or more years. Their main limitation is that a CD will only store about 700MB of data. That was once fine, but it is becoming less viable every year because of rising volume of data for each document to be saved.

DVDs, which can store up to 4.7GB of data, are of several different types (single/double-sided, single/dual layer etc), and many different ways to write the data exist, among which choosing the most suitable. However, probably single-sided single-layer DVD+R, which has better error checking and synchronisation than the earlier DVD-R system, is the best choice. DVDs will not last as long as CDs, but DVD+R seems to be the closest to CD-R as for lifespan.

DVD-RAM works like a floppy or hard disk, which makes it extremely convenient. DVD-RAM has even better error checking, so technically it's better for backups than DVD+R. Indeed, according to popular opinion, one of the format's advantages is the long life: without physical damage, data is retained for an estimated 30 years. For this reason, it is used for archival storage of data. The drawback is that DVD-RAM uses phase-change technology to write and rewrite the disc, like CD-RW, what recommend taking extra care to store DVD-RAM discs away from light (especially sunshine), heat and damp.

The Relative Stabilities of Optical Disc Formats in Restaurator, the International Journal for the Preservation of Library and Archival Material, show the results of a test in which a number of different types of disc are subjected to advanced aging techniques. CD-R won, even beating audio CDs, while DVD-RW came last. However, on this test, only CD-Rs made with (usually blue/green) phthalocyanine dyes were the winners. CD-Rs made with azo or cyanine dyes were instead last at the end of the aging process. The best and safest hence appear to be the Taiyo Yuden's Super Cyanine and TDK's "metal-stabilized Cyanine" dyes.

Blu-ray disks come with a lifetime warranty, though there is not any reliable info on how long they supposedly retain data. Under prime environmental conditions, they supposedly last quite a bit longer than CDs and DVDs because the method for recording data results in more durable storage, but even though they likely last quite a bit longer, they’re still optical media, which means they do not tolerate scratching, high temperatures, and sunlight, just as the others.

The M-Disc is an optical media storage disc that is a supposedly “permanent storage solution”. There are claims that it may be able to last up to 1,000 years, even in the face of environmental damage caused by scratching and high temperatures. While the M-Disc is a new format, it can be used with any standard DVD drive to read information, but since the data is engraved into advanced metals, a special M-Disc-ready drive is required to write it. Plus, since this technology is so new, the 1,000 year lifespan is only theoretical, so only time will tell how long these advanced discs will really last.

Advanced optical units

When the Falcon Heavy launched into space, it did not have on board only the red Roadster by Elon Musk led by Starman, but also a tiny crystalline device called Arch, on which the entire Galactic trilogy by Isaac Asimov was optically recorded. According to the Arch Mission Foundation, a non-profit organization in California that has promoted the new technology, similar devices aim to preserve and disseminate human knowledge in time and space, thanks to their extreme capacity and longevity. In fact this small crystalline quartz disk can contain up to 360 terabytes of data (as 76600 DVDs), resist up to a temperature of 1000 degrees and keep the data intact virtually forever. This new technology, developed by Peter Kazansky at the University of Southampton's optoelectronic research centre, uses a five-dimensional laser recording process on crystal or glass; it is the result of one of the current laboratory research aimed at solving the problem of material obsolescence of digital media. If within a generation, as it seems reasonable to expect, technology and industry will have solved the problem of the duration of the support, the only remaining motivations of digital obsolescence will be those related to the availability of adequate devices and reading programs. 

Next mission of the Arch will be carried out with the Moon touch down of the Israelian lander Beresheet, that is toting an Israeli flag and a time capsule. Among the capsule's contents is the "Lunar Library," a collection of materials that includes the full English-language version of Wikipedia, recorded on an Arch disc. 

Solid state devices

Solid state memory units, without moving parts, are normally distinguished into two broad categories:

those in which the single bit corresponds to a current flow, such as the DRAM commonly used for the working memory of the PCs, which consequently must be fed continuously to retain the information;those in which the single bit is registered as an electrical state of a circuit component, which consequently do not need power supply but can keep the information in static mode for a long time if no external disturbance factors are involved (e.g. electric fields). They are also called non-volatile memories.

The non-volatile memories are obviously those on which one aims to replace the magnetic disks both as components of PCs and tablets and as possible long-term storage memories.

They come as three different common storage media: Flash drives, SD cards, and solid state drives (SSDs). Producers say flash drives can last up to ten years, but flash memory doesn’t usually degrade because of its age, instead because of the number of write cycles, which means the more one delete and write new information, the more quickly the memory in the device will start to degrade. SD cards and flash memory register data as a sequence of electric states in a circuit, but we know that the charge will decay over a long period, so SD cards are not recommended for archival storage over very long periods. Worse still, they're vulnerable to static electricity, so any plastic storage/ handling could be fatal. Since all these devices are similar in that they all use flash memory, they’ll all degrade in a similar fashion.

On the other hand they have huge capacity with respect to CD and DVD, and making a copy is straightforward, so they could conveniently be used for periodic archival refreshing. However, one thing is certain: better hardware will pay off. Given the variety of manufacturers, lifespan might differ quite a bit from one device to another, but flash memory devices rated for more write cycles will usually last longer. 

Currently many new types of solid state non-volatile memories are under development and some of them surely will arrive on the market in the next future: 

they are all based on a correspondence between the single bit and a static condition of the memory circuit, that could be a magnetic state, or the phase state of a particular substance, or the changeable resistive characteristic of a component, or even the state of nanoparticles within an organic memory device.

A lot of these great ideas tend to die before reaching the point of mature technological development and viable commercialization, but there are still many hurdles to get over; software alone is a big task, as it is the manufacturing process, but gradually laboratory work will bring this technology one step closer to reality. It is not an exaggeration to say that the equivalent of 400,000 CDs, 60,000 DVDs, or 126 years of MPG music may tomorrow be stored on a polymer memory chip the size of a credit card. But what about data retention and information lifespan ? Researchers claim for instance that PRAM and PCM phase change memories will have a retention time tenfold with respect to current flash memory, without the possible flaws induced by external electric fields: we will certainly see in next decades static non volatile memories with really long lifespan and not susceptible to errors provoked by external environmental conditions.

Biological memory units

In 2013 Nick Goldman and colleagues, at the European Bioinformatics Institute in Hinxton, UK, synthesised DNA to encode a mix of information in its adenine, thymine, cytosine and guanine components. They used these letters to record an audio file of 26 seconds of King's speech, all 154 of Shakespeare's sonnets, a digital photo of their laboratory and the famous paper in which James Watson and Francis Crick first described the double-helical structure of DNA. 

The team built on previous DNA-encoding techniques by adding error correction, allowing content to be retrieved with 100 per cent accuracy. Then they showed that they could read, without any errors, 739 kilobytes of data stored in DNA.

Although it seems a science-fiction topic - in fact, it reminds us of the biological components of the Star Trek Enterprise digital system - it is in fact explored both by basic research and new applied research initiatives financed by the major companies in the digital sector.

DNA-based memory is sought after because DNA could last for thousands of years without special storage, other than being somewhere cold, dark and dry. In theory, it can encode roughly the capacity of 100 billion DVDs per gram of single-stranded DNA, making it potentially useful for storing vast amounts of archived data. As a matter of fact researchers have decoded the genomes of mammoths and a 700,000-year-old horse using DNA fragments extracted from fossils in the past few years. Synthesizing the designed DNA strand is the data-writing part. One can then read the data by sequencing the strands. The flaw is that while sequencing is rapid and chip, the syntesis is extremely complicated and costly. Furthermore, current methods are unfortunately not exempt from errors both during writing and reading. They will also have to find a method for indexing files within DNA storage, because now we can just be sure that a registration is contained in a DNA drop, but we don't know where the file's strands are within it.

In 2016 this possibility stirred up the interest of Microsoft and other tech companies in the field: 

Microsoft Research announced that it ordered to synthetic biology start-up Twist Bioscience 10 million DNA strands designed by Microsoft’s computer scientists to store data. 

Top memory manufacturer Micron Technology is also funding DNA digital storage research to determine whether a nucleic acid–based system can expand the limits of electronic memory. 

This influx of money and interest could lead to research and progress that eventually drive down today’s prohibitively high costs and make DNA data storage actually possible.

Conclusions

Securely storing large amounts of information over relatively short timescales of 100 years, comparable to the span of the human memory, is a challenging problem. Conventional optical data storage technology used in CDs and DVDs has reached capacities of hundreds of gigabits per square inch, but its guaranteed lifetime seems limited to a decade or little more. 

Solid state memories, especially phase change ones, promise noticeable performance in terms of data lifespan, but are still in the development phase and will only reach the market in the future.

Even the use of organic memories is promising, but for using its the advantages it will be necessary to wait. DNA based data storage can hold hundreds of terabytes per gram, but the durability is still questionable and the costs enormous. 

The major current challenge is the lack of an appropriate combination of storage technology and medium possessing the advantages of both high capacity and long lifetime. As for now the only recording and retrieval method of digital data storage with a nearly unlimited lifetime was implemented by femtosecond laser nanostructuring of fused quartz. The storage allows unprecedented properties including data capacity of hundreds of terabytes per disc, thermal stability up to 1000 °C, and virtually unlimited lifetime at room temperature, maybe opening a new era of eternal data archiving; anyway its development, engineering and commercialisation will still require several years, so at the current time we have forcibly to turn toward existing media.

It is quite easy to foresee a definitive solution for the duration of the supports problem within thirty years, but this undoubtedly opens a serious problem for the current epoch and its future memory, also aggravated by the fact that, being the ours a period of change, it will offer future generations a panorama of recordings extremely distributed on a variety of different media, from paper to optical, through those on film or magnetic media.

Hardware Obsolescence

Once the films were recorded on video tape, then they were passed on to DVDs, and now they can be streamed from the Internet. At the same time, the industry has ceased to produce videocassette readers and is now also starting to diminish the production of CD and DVD players, which will gradually disappear from the equipment that previously used them.

Probably you will have noticed that on new cars there is always a radio, but it is now also equipped with a small screen and the ability to connect other devices via USB (memory stick, smartphones). But the music CD player has disappeared, and with it the ability to use during travels that large collection of songs that we had accumulated over time. Now all they can no longer be listened, at least in the car. You will then have to reconstitute that music library by pouring all the songs on solid state memory, perhaps in mp3 format after having appropriately translated them from the original cda format. A long work, for which perhaps neither time nor desire will be found. Alternatively you could buy again all the songs you love, registered on the new support with the new format, for listening to them until the next technological change.

Those mentioned are trivial cases of hardware obsolescence, due to the change and evolution of the recording medium, which has consequently dragged the obsolescence of the related reading devices, no longer produced and bound to disappear.

The future pace of substitution of both writing and reading devices is unforeseeable, as it depends by a mix of market demand, support evolution and diffusion, as well as industry productive capabilities.

It would then much more practical and effective, for archiving purposes and subsequent consultation, to refer to specialized standard devices that the industry could offer public and private institutions in charge of data preservation, but this would precisely involve that largely respected adoption of a common standards - like for instance those proposed by ISO - still not working because all those organizations are currently proceeding in an erratic, diversified and voluntarily driven manner.

That's why we strongly need effective international agreement and political sensibility about an otherwise completely neglected subject, above all in this time of heavy economic problem. 

Software Obsolescence

It consists in the unavailability of the reading software of the document, which can be caused by two different reasons:

- the reading software, owned by a company that has not disclosed its internal characteristics, has been abandoned and its residual copies are constantly decreasing and destined to disappear. A similar situation is also that produced by successive software updates that are not backward-compatible and therefore not able to display documents produced with its previous versions. To avoid this problem at least partially, it would be advisable that some external institutions kept one efficient copy of it, in order to make it available in the future to those who had an interest in consulting documents that had otherwise become illegible. 

- the proprietary registration format of the digital document concerned has been abandoned and consequently the corresponding available interpretation software has become rarefied. It therefore falls in the previous case.

Obviously, if the software and the document format have public specifications, the problem of consultation can always be solved by recreating, if necessary, a suitable reading program, but facing new production costs.

As already explained, the most rational solution of the problem, in the production of digital documentation, would be to establish and to use shared public standard formats, thus simultaneously creating a situation in which the industry would have every interest in maintaining appropriate reading software. That means that, other than the preservation of suitable old reading programs, the software obsolescence revolves around the digital formats matter, whose dissemination on the market and their relative use by customers also guide the possible obsolescence of the reading hardware.

Technical appendices of chapter 1

Preservation Generality

The following points shape the framework of preservation problem:

• Digital archiving and preservation present unique research challenges because of concern with the long-term viability of digital information, 

where “long-term” may simply mean long enough to be concerned about the obsolescence of technology, or it may mean decades or centuries. Digital objects require constant and perpetual maintenance, and they depend on elaborate systems of hardware, software, data and information models, and standards that are upgraded or replaced every few years.  

• Accelerating rates of data collection and content creation and the growing complexity of digital information resources tax current preservation strategies designed to archive relatively simple and self-contained collections of data and documents.  Even established scientific data archives with a track record of three decades or longer cannot preserve all of the data entrusted to them using current methodologies.

• No acceptable methods exist today to preserve complex digital objects that contain combinations of text, data, images, audio, and video and 

that require specific software applications for reuse.  Raw data is rarely useful to researchers without associated models and analytical tools.

• Libraries, archives, museums, and other cultural institutions that have preservation as part of their core mission need solutions to digital preservation challenges if they are to play a meaningful role in preserving our intellectual and cultural heritage.

• Many government agencies, private corporations, not-for-profit organizations, and even private citizens are now concerned with preserving their own digital information assets.  Because digital information is vulnerable to alteration, erasure, and technology obsolescence, digital preservation methods are needed that can span a wide range of operating environments.

• People are an essential part of the process. Today’s curatorial processes for selection, description, and management of digital collections, while remaining important, are very labour intensive. There is a need to understand, evaluate, redesign, and automate many archival and preservation processes to drive down the costs of long-term preservation and to scale them up to the size and complexity of the digital archiving challenge.

• Future users of digital archives will have different needs, expectations, technologies, and analytical tools from those of the communities that 

initially created the digital content.  This raises challenging research questions in the areas of metadata, semantics, and knowledge management technologies that will enable future reuse of collections in digital archives.   

• The funding and business models for digital preservation differ considerably from common business models because an archival repository is expected to preserve digital resources even though their value and usefulness may not become apparent until well into the future.  In this respect, the economic models for digital preservation resemble the economics of public goods.  There is a critical need for research and development of economic and business models that will sustain digital preservation through many downturns in business cycles.  New business models are needed to make digital preservation affordable to individuals, government agencies, universities, cultural institutions, and society at large.

• The challenges of maintaining digital archives over long periods of time are as much social and institutional as technological.  Even the most 

ideal technological solutions will require management and support from institutions that in time go through changes in direction, purpose, 

management, and funding. 

Life span of different Optical Discs

To understand what limits the life span of optical discs, let’s look at how they are built-up. What all optical discs have in common is the presence of three key layers:

coating layer that protects the reflective layer.shiny layer that reflects the laser.polycarbonate layer that protect the storage layer (metal layer or photosensitive layer).

In addition, a label is applied above the coating layer and re-writable discs contain a photosensitive dye layer between the reflective and protective layers. One factor that determines the maximum life span of an optical disc is the type of reflective layer. Other factors include the overall quality of the raw material and manufacturing and most importantly the way the medium is treated by the user. The handling of an optical disc probably has the most significant impact on its longevity, but it is hard to predict exactly how long an optical disc will last since it depends on so many different factors. Nevertheless, estimations are floating around that predict a life span of up to 100 years for recorded CD-Rs and Blu-Ray discs.

As mentioned above, different types of optical discs contain different layers and particularly the reflective layer is susceptible to damage. Standard compact discs typically have a reflective layer made from aluminium. When exposed to air, aluminium oxidizes, which naturally happens around the edges of the CD. However, degradation of the reflective layer is not the only cause of disc decay.

The causes of disc deterioration are manifold and can include one of the following:

oxidation or corrosion of reflective layerphysical damage to disc surfaces or edgeschemical degradation of the colouring or metallic layer of recordable discsgalvanic reaction between layers and coatingschemical reactions with contaminantsultra-violet light damagebreaking down of disc materials, e.g. de-bonding of adhesives between layers

Interestingly, while most types of disc rot are caused by inappropriate use and/or storage, there is one in particular, i.e. CD bronzing, which is caused by a fault in manufacturing.

Optical discs simple visual check: if there are glares shining through tiny little holes when holding a disc against light, then the reflective layer has started to disintegrate. 

Also check CDs for discolouring, especially around the edges. 

See whether the different layers are still tightly together or have started to de-laminate. 

Finally try to copy the optical discs to a hard drive or scan them for data integrity using a suitable software.

There are many ways for increasing the likelihood that CDs and DVDs will last a long time. Here is a selection of the most important ones:

Choose a high quality medium from a good brand.To maximize CD longevity, go for gold as a reflective layer.Treating CDs and DVDs with care, i.e. holding them by the outer edges or the hole in the centre, not touching the surface, avoiding scratches, and keeping away dirt from the disc.Keeping them in a dry, dark, and cool place since humidity, sunlight, high temperatures and pollutants can damage the different layers.Storing them in jewel cases rather than paper slips.Using non solvent-based felt-tip permanent markers, suitable for writing on CD or DVD labels.Rewriting rewritable discs as little as possible.Choosing slow writing speeds to reduce errors and increase quality.

Let's examine them now in detail.

CD-ROM

The compact disc is a standardized type of optical disk used in various areas for storing information in digital format.

The compact disc consists of a transparent polycarbonate disc, generally 12 cm in diameter, at the centre of which is a 1.5 cm diameter hole dedicated to the fixing shaft of the CD player, and a further transparent area with a diameter of about 2 cm (including hole) dedicated to a possible mechanism to improve adherence to the rotation shaft; the remaining area of ​​the disk is coupled in the upper part with a thin sheet of metal material on which a layer of protective lacquer is deposited; in the lower part of the disk between metal and polycarbonate the information is stored as successions of  "pits" and "lands", subsequently read by means of a laser.

The data of the CD are in fact represented by small indentations called "pits", codified in a spiral track stamped on the aluminium reflecting layer. The areas between the holes are known as "lands". The change in height between pits and lands causes a difference in the way the light is reflected. By measuring the intensity change with a photodiode, the data can be read from the disk. Pits and lands do not directly represent zeros and one of the binary data. Instead, the inverted non-return-to-zero encoding is used: a change from pit to land or from land to pit indicates one, while no change indicates a series of zeros.

It is important to underline that in the CD-ROMs the pits are obtained by means of mechanical moulding and impressed in the double layer of polycarbonate-aluminium: they are therefore by their own nature much more durable than the simple chemical imprints used in CD-R and CD-RW. 

 CD-ROM reading

CDs have a structure comparable to that of normal music discs: the data are sorted along a single spiral-shaped track, an organization that is very different from that of magnetic disks (hard disk and floppy disk). The spiral starts at the centre (as opposed to vinyl records) and proceeds outward, allowing it to have smaller CDs than the standard (for example, credit card-shaped mini-CDs or CDs). The spiral structure of the CD-ROM is such as to maximize the performance for sequential access at the expense of direct access. 

 

A.A polycarbonate disc layer.

B.A shiny aluminium layer, with data printed and encoded by using "pits" e "lands", reflects the laser.

C.A layer of lacquer protects the top of the shiny layer.

D.Artwork is screen printed on the top of the disc.

E.A laser beam reads the CD and is reflected back to a sensor, which converts it into electronic data.

Diagram of CD layers

CD-R / CD-RW, acronym of Compact Disc Recordable and Compact Disc Re-Writable

The Compact Disc Recordable does not allow to delete recorded files once they are written. If the burning program allows the writing session to be kept open, files can be added to the CD-R in more than one writing step, but without the possibility of deleting those previously inserted.

A rewritable CD, on the other hand, allows the addition and / or cancellation of files many times, about 1000 if in good condition, obviously also in different writing sessions. On a chemical level, the CD-Rs offered by the industry are not all the same: the storage layer of a generic CD-R can in fact be made with:

Cianina - Tayo Juden Company LTD patent;

Phthalocyanine - Mitsui Toatsu Chemicals product used by Tayo Yuden in CD-R Gold;

Azo - product of Mitsubishi Chemicals Corporation

Formazan - product of Kodak Japan Limited.

About the obsolescence of the supports we can cite the case of Tayo Yuden, widely considered the company producing the best writable discs, which in 2015 has discontinued its production due to the demand reduction by the market and the increased cost of raw materials.

The CD-RW storage layer, on the other hand, is normally composed of a metallic alloy made with silver, indium, antimony and tellurium. Both the colouring layer of the CD-R and the metal alloy layer of the CD-RWs are written by the laser following the virgin control tracks specially printed on the CD during production.

Both CD-R and CD-RW discs have the same basic structure but with significant detail differences. The CD-R disc has a dye-based recording layer, with a reflectivity of 40 - 70 %, while the CD-RW disc has a phase-change recording layer with a reflectivity of 15 - 25 %. Both discs have an additional reflecting layer: golden for the CD-R, which accounts for that disc's distinctive appearance, and silver (aluminium) for the CD-RW. Obviously these differences, as is easily understood, influence the duration of the recording.

CD-Rs 

are WORM (Write Once Read Many) disks that can be written once and read multiple times. Compared to printed silver-coloured CDs, they are less resistant to temperature changes, direct sunlight and mechanical stress. They also have a shorter duration even if their life, after being engraved, is estimated in several years. Their operation, in reading, is the same as that of silvered CDs even if not all CD players, especially older models, are able to read all types of CD-Rs.

A CD-R is, in general, formed by the following parts:

an optional labela printable surface, a protective layer or botha protective lacquer treated with ultraviolet raysa reflective layeran organic dye polymer on which the data are engraveda polycarbonate substrate (transparent plastic).

In practice, compared to the printed CD, there is the addition of a photosensitive layer that the laser "burns" in the writing phase, to then detect these "burns" during the reading phase. 

CD-Rs are injection-moulded with a flat "blank" data spiral. A photosensitive dye is then applied, after which the discs are metallized and lacquer-coated. 

The write laser of the CD recorder changes the colour of the dye to allow the read laser of a standard CD player to see the data, just as it would with a standard stamped disc. Over time, the dye's physical characteristics may change causing read errors and data loss until the reading device cannot recover with error correction methods. The theoretical design life is from 20 to 100 years, depending on the quality of the discs, the quality of the writing drive, and the storage conditions. However, testing has demonstrated that final degradation of some discs could happen in as little as 18 months, under normal storage conditions. This failure is known as disc rot, for which there are several, mostly environmental, reasons. 

CD-RW is a re-recordable medium that uses a metallic alloy instead of a dye. The CD-RW storage layer is a mixture of silver, indium, antimony and tellurium.  This optical disk uses tellurium as the write layer and carbon as a dielectric and oxidation prevention layer. The sandwich style Carbon-Tellurium-Carbon film was deposited on polycarbonate and silicon substrates by plasma sputtering. These films were then characterized with Scanning Probe Microscopy (AFM, SEM, TEM, EELS), and ellipsometry and were tested for writability and longevity. Results show the films were uniform in physical structure, stable, and able to form permanent pits. Data was written to a disk and successfully read back in a commercial DVD drive.