\section{Software Requirements for Digital Libraries}
\label{Sec:Requirements}

Our view of digital libraries imposes several requirements on software platforms on which such libraries are
developed, used and maintained. Major demands result from the following properties in digital library systems:

\begin{itemize}

\item safe extensibility to an open set of new media types, novel library services, extended user requirements etc.
\cite{NueFurLeg+95}  
\item scalability from private stand-alone libraries to shared group libraries or globally networked libraries     
\item global connectivity with free data exchange, open systems communication, unrestricted software migration etc. 
\item customizability for personalized, value-added information and software artifacts.

\end{itemize}

Some of these demands involve language requirements, others imply
architectural requirements for the digital library system itself. Both
sets of requirements are clearly not completely independent of each
other as we will see with requirements such as binding or reflection.


\subsection{Linguistic Requirements}
\label{SubSec:LinguisticRequirements}

Our vision of digital libraries raises specific demands on the naming,
typing, binding and scoping capabilities of the language of our choice.
We argue here for some fairly novel language concepts by which data,
code and threads become first-class citizens, i.e., players with equal
linguistic rights.

\begin{description}

\item[Typing:] Digital libraries essentially impose three important demands on the underlying typing system: 

\begin{itemize} 

\item type expressiveness for extensive model definition, 
\item type control for safe model execution, and
\item type inspection for generic programming.

\end{itemize} 


\begin{description}

\item [Types for modeling:] The information modeling needs of a digital library are abundant and may vary over its
lifetime. Consequently, languages with a fixed, built-in set of models do not suffice. Instead, an add-on approach to
model definition is requested, a demand which can best be met if such models can be expressed by the language
itself. Such a demand leads directly to languages with rich polymorphic type systems, (recursive, parametric) type
constructors, subtyping and higher-order functions. 

Tasks to be addressed by model extensions are bulk data management \cite{Schm77a,AckFie95}, multi-media modeling, safe service integration, etc.
There is a particular need for a wide range of bulk or collection types such as lists, sets, keyed sets and bags. The
framework should provide an add-on bulk type library which satisfies different requirements for search, insert, order
or store. Bulk types dynamically added on should, at the same time, be safe, disallowing, for example, the insertion of
image tokens into book collections.

In our approach, typing comes into effect on the artifact level; tokens are untyped bit vectors, a representation which
simplifies generic low-level operations such as copying, storage, migration or garbage collection. More specific
basic methods on the token level, e.g., for token presentation, should be attached manually, on a case by case basis. 


\item [Types for control:] Types capture constraints on computations. Good type systems allow a rich set of
constraints to be expressed and to be checked as early as possible. Type checking is usually done statically, i.e., at
compile-time: at that time the price for type checking is cheapest and type violations can most easily be fixed.
However, long-lived and distributed systems, such as digital libraries, are frequently faced with constraints which
arise at different sites or change during their lifetime. This requires more dynamic approaches to typing. If, for
example, values are transmitted (via files or communication channels) between independently developed digital
libraries, there is no common scope in which a static type check could be performed to guarantee compatibility
between  data and programs. For tasks like these, (value, type)-pairs have to be available at run-time, and a rich
functionality is required for run-time evaluation and exception handling. 

\item [Types for genericity:]  Another need for dynamic types results from the desire to implement generic
functions with type-dependent behavior. Such functions take a type representation, usually along with a value of
this type, inspect the type and exhibit different behavior depending on the type. Type inspection allows, for
example, iteration over the attribute types and attribute values of an aggregate or construction of an aggregate from a
list of typed bindings. Such possibilities are required, for example, for generic library browsers capable of
displaying and manipulating artifacts of any type.

\end{description}



\item[Binding capabilities:]

Different kinds of bindings between information artifacts and tokens as well as between information and software
artifacts are required. It should be possible to combine them orthogonally: 

\begin{description}

\item[static and dynamic:] In traditional applications bindings to entities are stable over time. For digital
libraries such static bindings are, however, not sufficient since artifacts are created dynamically and may be
updated. Dynamic bindings are functions (queries or other computations) that are dynamically evaluated and establish
bindings on demand.

\item[internal and external:] Internal bindings refer to entities which are under the control of the active system while
external bindings refer to entities outside. External bindings frequently require special treatment because essential
properties of the active system may not apply to external entities. Typical examples of such properties are
orthogonal persistence or transactional access. 

\item[local and remote:] Local bindings refer to entities on the site where the process is currently executing.
Digital libraries also require remote bindings which can refer to entities that reside on a different site in the
network. Remote bindings imply automatic transfer to the local site if the bound entity is referenced, e.g. for
display.

\item[copy and reference:] A reference to an entity may be replaced by copying the entity into the local context,
i.e., the context of the artifact that contains the reference. This facilitates the access to the referenced entity and
increases the autonomy of artifacts. Increased autonomy is required for the construction of a stand-alone artifact
collection that may be used off-line, e.g. on a CD-ROM. 
Further aspects of copying in digital libraries are discussed in \cite{Cam94,ShiGar96}.

\end{description}


\item[Scoping:] The space of library entities reachable from  a given information artifact has to be individually
structured and organized. Therefore, in addition to static binding environments (scopes) there is a need for dynamic
environments \cite{Dea89} in the sense that new bindings may be added or existing bindings may be hidden or removed. Scopes may be
public or protected for private use.  Libraries may require that scopes be named or even possess first-class status.  

\item[First-class status for data, code, and threads:] Besides passive information (data), artifacts may also include 
entities which can be activated (code) or even activities themselves (threads). Instead of constructing artifacts `in
advance', code entities are frequently used to dynamically construct artifacts `just in time'. Thread entities are
required to realize artifacts with autonomous search agents attached to them. Threads may be in the state of running,
frozen, waiting for some event, etc. Requesting code having first-class status implies that our language
of choice be algorithmically complete.


\item[Exception handling:]
In dynamically changing and heterogeneous environments not all services are available always and everywhere. This
results in exceptional behavior that has to to be handled flexibly and safely.

\end{description}


\subsection{Architectural Requirements}
\label{SubSec:ArchitecturalRequirements}

A great number of software services have to cooperate smoothly and steadily to collectively provide the services to be
expected from an advanced digital library. To do so, software has to show a certain degree of `social behavior' which
cannot yet be taken for granted on current platforms. In addition, functionality which is crucial for libraries, such
as persistence and mobility, has to be substantially supported for all kinds of library entities.

\begin{description}

\item[Orthogonal Persistence:] Generally speaking there is no argument supporting why the lifetime of any digital token
or artifact should depend on its type or on the fact as to whether it is composed of data, code or threads.  Only if
persistence is provided orthogonally to such dimensions, general information (data), the knowledge of how to
use it (code), and the actual use made of it (threads) can be preserved and made available any time.

Having a system with orthogonal persistence means, in technical terms, that arbitrary bindings (i.e.,
arbitrary data structures, function closures, activation environments) can be converted between execution
environments and stable storage environments. For elaborate typing and binding models as discussed in the
previous subsection this is obviously not a trivial task. However, once this problem is solved other requirements
such as free data, code and thread migration or powerful tracing environments are almost free for the asking. 

Since persistent storage systems have to deal with all kinds of information tokens and artifacts they do so
according to the lowest common denominator as given by bit vectors and references. Further requirements for persistent
stores are

\begin{itemize}

\item scalability to different implementations
\item transactional semantics
\item standardized import and export formats
\item garbage collection based on reachability.

\end{itemize}


\item[Open Service Integration:]

Digital libraries have to be prepared to accept services from `alien' systems. The diversity of services required by a
digital library could never be provided without external support. The demand for novel services is driven by both sides,
providers and consumers, and is expected to grow rapidly with the number of libraries on the net. External services may be
required, for example, in the following situations:

\begin{itemize}
 
\item external tokens --- images, sounds, but also native code ---  can frequently only be interpreted by software which
is only available on the site on which the token resides;

\item some sites specialize in services not available at the home site; examples are expensive printing or
scanning facilities;

\item many standard services, e.g., for presentation, compression, encryption, communication, etc., are
factored out to generic subsystems; it is often the case that the complexity of a service demands a
specialized service provider; examples are information retrieval and SQL
engines;

\item finally, legacy awareness requires that external service integration is approached on a general level.


\end{itemize}

On the modeling level safe service integration can be achieved by expressive typing (see previous subsection); on the
system level it requires gateways and call/callback interfaces to the  external service's languages (C, C$++$, \dots). In
combination, both requirements support the definition and implementation of new software libraries which integrate external
services safely into our language of choice.  In addition to what is required for service integration in general,
standards for export interfaces and for exported references to library entities are necessary. Finally, each digital
library also should be prepared to export its services to others and to act as information provider in the net. 


\item[Platform Independence:] Digital libraries require that information and services must be made available
network-wide on heterogeneous platforms (see figure~\ref{Fig:DataCodeThread}). Library exploitation can be improved considerably if a library's core
functionality for searching, querying and storage is provided platform-independently. This means that data, code
and thread representations abstract from all platform-dependent details. Technically speaking, this leads to simplified
abstract machine architectures based on bytecode representation and interpretation as well as to run-time systems with
improved portabiliy, minimal in size and composed of standardized components.   

\begin{figure}
\begin{center} 
\leavevmode
\epsfxsize=0.5\textwidth 
\epsffile{Pictures/DataCodeThread.epsf} 
\end{center} 
\caption{Data, Code and Thread Mobility}
\label{Fig:DataCodeThread}
\end{figure}


\item[Migration Technology:] The usability and autonomy of digital libraries is substantially improved by
agents \cite{Bir95} capable of migrating freely in open networks to visit data sets and collect information \cite{BalSho95}. Mobility,
in particular that of threads, means portability at run-time. This requires an extremely flexible binding
technology which can dynamically dissolve and establish bindings, replace local bindings by remote ones and
implement the notion of ubiquitous resources to which bindings can be established at any site. 

The flexibility gained by migration technology should be contrasted with the traditional client-server
paradigm which heavily depends on assumptions on the client's home site and on the servers available in the
net.  

\item[Reflection:] There are numerous reasons why digital artifacts may have to be occasionally
rearranged when used in networked and long-lived environments. Software migration, integration,
(re-)binding, tracing are some such reasons, but also wrapping, persistent storage or optimization.
Frequently, such tasks are highly regular and can be systematically performed by algorithms.


Reflection is a powerful yet (relatively) safe technology for the algorithmic inspection and
readjustment of digital representation. Reflective program analysis and transformation has to
rely on detailed knowledge about the programming language model and its
implementation. Reflection may not only be used for generating or altering data or code (at
compile-time or run-time) but also for transferring compile-time information (e.g. on types) to
run-time. Tasks for which reflection is required in our setting are

\begin{itemize}

\item binding environments: reflective manipulation of bindings may, for example, transform
local references to remote ones or replace remote references with copies;
 
\item tracing environments: reflection can exploit information in the environment in which
threads are executed (e.g. date, user-id, software version numbers, etc.) and attach such
meta-information to the executing thread. Such information is collected automatically as a
side-effect of user actions. This information collection process can be parameterized by
user-defined filters according to user preferences. 

\end{itemize}



\end{description}
