\section{Tycoon as a Framework for Digital Libraries}
\label{Sec:Framework}

The goal of the Tycoon project\footnote{Tycoon: Typed communicating objects in open environments.}  \cite{Tyco92X} is to
provide modeling flexibility as well as system stability for multi-functional, long-lived application systems operating in
heterogeneous, open and networked environments. 

Tycoon contributes to this goal on two levels \cite{MSS95}.

\begin{itemize}

\item The Tycoon language is a persistent polymorphic programming
language with an elaborate higher-order type system
\cite{MaSc91b,SMV93,ScMa93,MaSc93b}.

\item The Tycoon system is architectured to substantially improve
portability, scalability and interoperability as required by
applications in open heterogeneous networks
\cite{MMS96a,MMS96b,MMS95a,MMS95b,MaSc94}.

\end{itemize}


\subsection{Achievements of the Tycoon Language}

Tycoon typifies one of those recent language developments \cite{GMcG95,MBC*94} which fully
exploit the increased capacity of modern computing and storage
facilities. Although, according to conventional standards, Tycoon's
computational model may be considered resource consuming, it is our
firm belief that such an investment is well justified by the
substantially improved quality of systems developed in languages such
as Tycoon.


\subsubsection*{Typing}

The Tycoon Language (TL, \cite{MaSc92a,Matt93}) excels in its expressive type system based on existential and universal
type quantification, recursive types and structural subtyping. With respect to typing, TL follows the tradition of the
experimental polymorphic language Quest \cite{Card89}.

TL is based on very few built-in types and a small set of type constructors. Orthogonal combination along with recursive
type definition and (recursive) type operators provide virtually all data structures of interest.

For a simple digital library, books composed of texts and images may be modelled by:   
\begin{xple}
\k{Let} BookComponent = 
  \k{Tuple case} text \k{with} content :String   \k{case} image \k{with} content :jpeg.T \k{end}
\k{Let} Book = 
  \k{Tuple} authors :set.T(Author)  title :String  contents :list.T(BookComponent) \k{end}
\k{Let} Library = set.T(Book)
\end{xple}

By structural subtyping a (recursive) type \p{Publication} becomes a subtype of \p{Book} so that all operations on books,
e.g. the displaying of the contents, are also applicable to publications:

\begin{xple} 
\k{Let Rec} Publication $<$:\k{Ok} = 
  \k{Tuple} authors :set.T(String)  title :String  contents :list.T(BookComponent)
        references :list.T(Publication) \k{end}
\k{let} displayBook(book :Book) :\k{Ok} = \dots
\k{let} napoleonicWars :Publication = \dots
displayBook(napoleonicWars)
\end{xple}

Generic structures such as sets or lists can be defined within TL through its higher-order polymorphic type
system \cite{ScMa94,MaSc91b}. In the following example the recursive type operator for lists depends on an element
type \p{E} so that lists of type \p{T(E)} accept only elements of type \p{E}:

\begin{xple} 
\k{Let Rec} T(E $<$:\k{Ok}) $<$:\k{Ok} =
  \k{Tuple case} nil  \k{case} cons \k{with} element :E  tail :T(E) \k{end}
\k{let} new(E $<$:\k{Ok}) :T(E) = \dots
\k{let} cons(E $<$:\k{Ok}  element :E  list :T(E)) :T(E) = \dots
\end{xple}

TL supports dynamic types and dynamically-typed values. A dynamic value is a pair of a value \p{v} and a run-time
representation \p{t} of its type. If a dynamic value component \p{v} is extracted, its associated type representation
\p{t} can be inspected. This enables a boolean subtype test whether \p{t} is a subtype of a given supertype \p{T}. Such
functionality is required to assure, for example, that an artifact transferred from an external site is a book:

\begin{xple}
\k{let} dynamicValue :dynamic.T = receive(channel) (* a dynamic value is transferred *)
\k{let} book :Book = dynamic.be(:Book dynamicValue) (* test if this value is a book *)
displayBook(book) (* book is now statically typed *)
\end{xple}

Another application of dynamic types is, for example, a generic artifact browser that is capable of working on
arbitrarily structured artifacts:

\begin{xple}
\k{let} artifactBrowse(\k{Dyn} T $<$:\k{Ok}  artifact :T) :\k{Ok} = \dots
artifactBrowse(:Library historyLibrary)
artifactBrowse(:Book napoleonicWars)
artifactBrowse(:BookComponent napoleonPicture)
\end{xple}
 

\subsubsection*{Binding Capabilities}

Tycoon supports a wide variety of binding concepts covering a substantial range of what is requested in
subsection~\ref{SubSec:LinguisticRequirements}. The more traditional binding alternatives include immutable, mutable,
static and dynamic bindings to types and values:

\begin{xple}
\k{Let} Book = \k{Tuple} \dots \k{end} (* type binding *)
\k{let} napoleonicWars = \k{tuple} \dots \k{end} (* value binding *)
\k{let var} actualBook = \k{tuple} \dots \k{end} (* mutable binding *)
\k{let} bookMark = actualBook (* static binding *)
actualBook := \k{tuple} \dots \k{end} (* bookMark and actualBook refer to different values *)
\k{let} displayBook(book :Book) :\k{Ok} = \dots
displayBook(napoleonicWars) (* dynamic binding *)
\end{xple}

In Tycoon it is completely transparent whether a binding refers to a
local or a remote entity \cite{MMS95a,MMS95b}.  If an entity leaves the
local scope in which it is bound, two alternatives exist: Either the
entity (and its closure) is copied or the binding is replaced by a
remote reference. On request remote references may be eliminated by
copying.

Bindings to external functions are established by the predefined
\p{bind} function. Calls of external functions are indistinguishable
from calls of internal functions. Tycoon takes care of the
machine-dependent part of the parameter conversion and call mechanisms.
The following example shows the binding to a machine-dependent function
which displays JPEG images:

\begin{xple}
\k{let} displayJPEG = bind(:\k{Fun}(:JPEG) :\k{Ok}  "imageLib" "display\_jpeg\_image")
displayJPEG(image)
\end{xple}

It should be noted that call and binding transparency is a language
requirement with heavy demands on the architecture of the system.


\subsubsection*{First-Class Data, Code and Thread Entities}

Artifacts contain data but may also require code and thread components.
Code entities are used to construct artifacts dynamically `just in
time'. Each time the subsequent function \p{reutersNewsOnElection} is
evaluated the Reuters news service is asked for the latest news on the
election:

\begin{xple}
\k{let} reutersNewsOnElection() :set.T(News) = \k{begin}
  \k{let} reuters = openConnectionTo("Reuters News Service")
  searchFor(reuters "Election")
\k{end}
\k{let} actualNews = reutersNewsOnElection()
\end{xple}

Even more value may be added to artifacts by higher-order functions
expressing specific personal preferences, such as an interest in news
which mention both candidates:

\begin{xple}
\k{let} reutersNewsOnElection(preference(news :News) :Bool) :set.T(News) = \dots
\k{let} bothCandidates(news :News) :Bool = 
  newsIncludes(news "Clinton") \k{andif} newsIncludes(news "Dole") 
\k{let} actualNews = reutersNewsOnElection(bothCandidates)
\end{xple}

With threads components artifacts can work off-line based on local
state information. To collect only the latest news from a news
service a thread may be employed which remembers the time it last collected
the news:

\begin{xple}
\k{let} collectLatestNews() :\k{Ok} = \k{begin}
  \k{let var} lastCollectionTime = never()
  \k{loop}
    \k{let} latestNews = reutersNewsOnElectionSince(lastCollectionTime)
    lastCollectionTime:= now()
    deliverNews(news)
    thread.sleep()
  \k{end}
\k{end}
\k{let} newsThread = thread.start(collectLatestNews)
\end{xple}

Our \p{newsThread} collects (in parallel to other activities) all news
available since its last wakeup; then it goes to sleep until
reactivated by \p{thread.wakeup(newsThread)}.

Threads are first-class entities in Tycoon and can be observed (traced)
reflectively. In this way context data of artifact construction processes
can be created and stored persistently \cite{MaSc94}. With Tycoon's
reflective capabilities such processes can be parameterized by
user-preferred filters for the information to be attached to the
artifact. This may not only add value to the constructed artifact but
also help improving the artifact construction process.


\subsubsection*{Exception Handling}

Since unexpected events are frequent in open systems, Tycoon provides strong support for the handling of exceptional
situations. If an exception is raised, execution stops and continues where the last exception handler was installed,
possibly crossing function or module boundaries.

The exception handling routine decides which action is sufficient to
handle the exceptional situation. For example, when the display routine
for a JPEG image is not available on a particular workstation, a
conversion from JPEG to GIF may help:

\begin{xple}
\k{try} displayJPEG(jpegPicture) \k{when} notAvailable \k{then}
  \k{try} displayGIF(convertJPEGtoGIF(jpegPicture)) \k{when} notAvailable \k{with} name :String \k{then}
    message("Cannot display picture " $<>$ name)
  \k{end}
\k{end}
\end{xple}


\subsubsection*{Scopes}

Tycoon applications, frequently,  are very large. As a result, they can
be structured into interfaces, modules and libraries which restrict the
scope of bindings \cite{SMV93}. Tycoon libraries are nested high-level
entities that declare a definition order for interfaces and modules so
that they can only use identifiers which were defined previously,
preventing cyclic dependencies. Interfaces export public identifiers
while modules define the bindings for these identifiers using also
private bindings. Through nesting and hiding mechanisms the scope of
identifiers can be controlled.

Currently, an orthogonal suite of scoping primitives is under
investigation which specializes in the requirements of personalized
scopes in open networked environments. It is  provided by a generic
Tycoon module that may be combined with modules for distribution,
versioning, access control \cite{RMS95}, etc. thus enabling the
flexibility required by personalized digital libraries.



% ----------------------------------------------------------------------

\subsection{The Tycoon System Architecture}

The architectural requirements for digital libraries can be seen from two perspectives. 

\begin{itemize}
\item Since digital libraries hold a vast variety of information content and provide extended information
handling services there is an intrinsic complexity involved in libraries seen as a performing software system. 

\item Complexity also results from the fact that library systems will never stay stable; instead it has to be
anticipated that libraries will develop during their virtually infinite lifetime, even more so when connected to
a  virtually unlimited network space.
\end{itemize}

Tycoon contributes to both complexity issues, first by enabling the construction of sytems with a higher degree of
regularity (through orthogonal persistence, platform independence, migration technology etc.), second by investing in
important cases of change and in change management technology in general (open service integration, scalability,
reflection). The layered architecture of the Tycoon System (see figure~\ref{Fig:TycoonLayers}) scales from single-user PC-based applications to distributed applications in open heterogeneous networks.


\begin{figure}
\begin{center} 
\leavevmode
\epsfxsize=0.5\textwidth 
\epsffile{Pictures/TycoonLayers.epsf} 
\end{center} 
\caption{Layered Architecture of the Tycoon System}
\label{Fig:TycoonLayers}
\end{figure}



\subsubsection*{Persistence}


Tycoon provides orthogonal persistence for data, code and threads. The Tycoon Store Protocol, TSP  \cite{MMS96a},
defines a uniform, data model-independent call-level interface to multiple (commercially distributed) persistent 
object stores abstracting from implementation details of the underlying persistent stores. A typical TSP server is
implemented by a store adaptor which maps TSP data structures and functions to data structures and operations of
an  existing object store.  

TSP contributes significantly to Tycoon's system scalability since TSP clients can decide between different

\begin {itemize} 

\item garbage collection strategies 
\item object faulting mechanisms 
\item error recovery, logging and persistent savepoint mechanisms
\item commit protocols (for distributed systems)
\item security and authentication support

\end {itemize} 

or choose between a single- or multi-user environment.

A primary design goal of TSP is to provide efficient data storage independent of the data and language model used
by TSP store clients.  TSP supports polymorphically typed models where store objects may contain values of
different types.  TSP therefore uses an untyped low-level store model which distinguishes only (uninterpreted) bit
vectors, used for information tokens, and arrays of references used for artifacts. The lifetime of TSP store objects
is defined by reachability and storage is reclaimed by garbage collection. 



\subsubsection*{Open Service Integration}

External services, e.g. visualisation systems or database systems, can be integrated into the Tycoon system through
bindings to external functions \cite{ScMa93}. The binding mechanism allows the Tycoon system to call external functions as
well as it allows the external system to call back the Tycoon system via higher-order functions. Examples are
event-controlled windowing systems that call application-defined functions when a button is pressed or a window closed etc.

Such services are wrapped by Tycoon in portable and type-safe functions exploiting the full polymorphic power of the
Tycoon type system even in cases in which external functions are completely untyped. When calling external functions from
Tycoon or, in reverse, calling (back) Tycoon functions from external systems, parameter conversion is handled by the
Tycoon run-time system.

The following example shows an interface \p{SQL} that exports polymorphic functions to access different SQL database
systems:

\begin{xple}
\k{interface} SQL \k{import} \ldots\ \k{export}
  error :\k{Exception with} sqlError:String \k{end}
  Table(E, K $<$:\k{Tuple end})  $<$:\k{Ok}
  \ldots
  openTable(\k{Dyn} E, K $<$:\k{Ok}  tableName :String) :Table(E K)
  \ldots
  lookup(E, K $<$:\k{Ok}  from :Table(E K)  key :K) :E
  \ldots
  selectFromWhere(E, K, R $<$:\k{Ok}  project(:E) :R  from :Table(E K)  where(:E) :Bool) :Iter.T(R)
  \ldots
\k{end}
\end{xple}

Tycoon libraries hold, for example, two different modules \p{ingresSQL} and \p{oracleSQL} which implement the above
interface by bindings to the dynamic SQL call interfaces of the Ingres and Oracle relational database management systems. 
The type operator \p{Table(E K)} exported from the interface describes the type of SQL tables with element type \p{E} and
key \p{K}.  For example, a value of type \p{oracleSQL.Table(News Int)} is an Oracle table with rows that have attributes
as defined by the Tycoon tuple type \p{News} where a running number identifies a row.

The polymorphic function \p{openTable} opens a named table for further processing and takes dynamic type
variables \p{E} and \p{K} as its first arguments to ensure that the database table structure matches the Tycoon
type information. If a schema mismatch is detected, the Tycoon exception \p{error} is raised at run-time. All
other operations of the SQL interface (queries, table updates) can be checked statically by the Tycoon compiler
based on the polymorphic signatures assigned to the SQL functions. For example, the signature of the function
\p{lookup} expresses the type constraint that from a table of type \p{Table(E K)} only tuples of a matching type
\p{E} can be retrieved.

The function \p{select} shows how polymorphic typing can be used to statically describe a (unary) select statement. The
selection on relation \p{from} is controlled by the predicate \p{where} that works on relation elements. \p{project} builds
the resulting tuples. Note that this works only if the external database system is capable of including arbitrary function
calls dynamically in queries. 

The Tycoon service integration technology sketched above is open to arbitrary services of interest.  Since, besides
formatted data and SQL, texts are essential for digital libraries, information retrieval technology has to be available
based on fuzzy queries and ranking lists . Therefore, Tycoon also maintains a text retrieval library which integrates the
services provided by the information retrieval system Inquery \cite{cch92,bcc94}.


Besides the integration of external services, Tycoon is also open for being integrated as a server into external systems.
For this purpose, Tycoon provides a callback mechanism which allows external systems to invoke Tycoon functions.

In the following example, Tycoon is set up as a server, e.g. as WWW server, that accepts external communication
requests. In our example, remote procedure calls are used for communication and a RPC server is installed and started which
stores and retrieves JPEG images:
 
\begin{xple} 
\k{let} jpegDB = \k{tuple}
  \k{let} store(image :jpeg.T  name :String) :\k{Ok} = \dots
  \k{let} retrieve(name :String) :jpeg.T = \dots
\k{end}
\k{let} server = rpcServer.new()
rpcServer.register(server  "JPEG DB"  jpegDB)
rpcServer.dispatch(server)
\end{xple}


\subsubsection*{Platform Independence}

The Tycoon Virtual Machine (TVM) is an abstract call interface above the  TSP layer that defines a bytecoded
instruction set based on a higher-order,  functional execution model.  TVM bytecode is either interpreted by a 
virtual machine or is compiled on the fly into target machine code.   The TVM interpreter and its associated
run-time system are written in ANSI-C. 

The platform-independence of the TVM model makes it possible to dynamically transfer portable bytecode between
heterogeneous nodes in distributed digital libraries without recompilation.  Utilizing TSP's linear external
data representation (TXR), it is also possible to migrate a thread across system boundaries
\cite{MMS95a,MMS95b,MaSc94}.

For example, the higher-order query function \p{reutersNewsOnElection} can be rewritten in the following way for a client-server approach where the function \p{bothCandidates} is shipped to a server via RPC and evaluated at the server site:
\begin{xple}
\k{let} reutersNewsOnElection = 
  rpc.bindTo(:\k{Fun}(preference(:News) :Bool) :set.T(News)  serverSite  "reutersNewsOnElection")
\k{let} bothCandidates(news :News) :Bool = newsIncludes(news "Clinton") \k{andif} newsIncludes(news "Dole")
\k{let} actualNews = reutersNewsOnElection(bothCandidates)
\end{xple}


\subsubsection*{Migration Technology}

Even more flexibility is achieved in Tycoon because threads can migrate between the nodes in a network \cite{MMS95a,MMS96b}. This enables, among others, the implementation of agents. An agent may include the trace of the search in its search process, e.g. the items it has already found, the number of nodes visited or the nodes it has been recommended to visit:
\begin{xple}
\k{Let} SiteData = \k{Tuple} news :set.T(News)  recommendedSites :set.T(Site) \k{end}
\k{let} homeSite = agent.thisSite()
\k{let} sitesVisited = set.new(:Site)
\k{let} sitesToVisit = set.create(newYork washington losAngeles)
\k{let} newsFound = set.new(:News)
\k{while} not(set.empty(sitesToVisit)) \k{do}
  \k{let} nextSite = set.getAny(sitesToVisit)
  \k{let} local :SiteData = agent.migrateTo(nextSite)
  set.insert(sitesVisited nextSite)
  set.include(newsFound searchFor(local.news "Election"))
  set.include(sitesToVisit  set.exclude(local.recommendedSites sitesVisited))
\k{end}
agent.migrate(homeSite)
deliver(newsFound)
\end{xple}



\subsubsection*{Reflection}

The Tycoon system supports different manners of reflection. By \e{linguistic reflection} \cite{SMA91,SSF92} the
execution of the compiler (compile-time reflection) or the application (run-time reflection) can be influenced.

Compile-time reflection is achieved by executing user-defined code during compilation. Dynamic types, for example,
are implemented by compile-time reflection. The compiler collects type information and stores it persistently for
run-time use. Similarly, operations on compile-time information are performed as, for example, by the \k{Repeat}
construct which repeats a list of Tycoon signatures (for the definition of \p{Book}, see above): 

\begin{xple} 
\k{let} createBook(\k{Repeat} Book) :Book = \dots 
\end{xple}

Here, \k{Repeat} \p{Book} is reflectively replaced by

\begin{xple}
  authors :set.T(Author)  title :String  contents :list.T(BookComponent)
\end{xple}

Run-time reflection is achieved by typed bindings to compiler subcomponents (parser, type checker, code generator,
evaluator, module  manager, \dots), thus made available to applications at run-time. In this way code may be
evaluated or even generated depending on run-time (computed) information. In Tycoon full type safety is
guaranteed, even in the presence of run-time and compile-time reflection, by the consistent use of dynamic types
in the Tycoon compile-time and run-time environments. The following example shows the binding of the subtype
checking function of the compiler and its use for dynamic values at run-time:

\begin{xple}
\k{let} isSubType = reflect(:\k{Fun}(:typeRep\_T :typeRep\_T) :Bool "isSubType")
\k{let} dynamicValue = receive(\dots) (* transmitted dynamic value *)
\k{if} isSubType(typeOf(dynamicValue) :Book) \k{then} \dots \k{end}
\end{xple}

\e{Behavioral reflection} \cite{KCMS96} is a second kind of reflection supported by Tycoon. It is achieved  by influencing
Tycoon's interpreter loop and the run-time system of the Tycoon Machine. For example, the following code marks the
image \p{napoleon} as immobile. Consequently, if the book \p{napoleonicWars} which references the image \p{napoleon} is
transferred, the image is not copied; instead a remote reference to the image is introduced:

\begin{xple}
markAsImmobile(napoleon)
sendTo(newYork napoleonicWars)
\end{xple}

\hspace*{3em}

In summary, Tycoon is a powerful framework which provides much of the functionality required to make multi-functional and
multi-media application systems persist in turbulent environments such as open, heterogeneous and networked information
infrastructures. 

