% File:   rationale.tex
% Date:   13.2.1991
% Status: Last version by FM & JWS
%
\documentstyle[lncs]{article}
\newcommand{\e}[1]{{\em #1\/}}
% \pagestyle{empty} ##

\title{The Rationale behind DBPL
\thanks{This research was supported in part by the European Commission
under ESPRIT BRA contract \#~4092~(FIDE).} }
\author{Joachim W.\ Schmidt \and Florian Matthes}
\date{ {\small\em Department of Computer Science \\ 
Hamburg University \\ 
Schl\"uterstra{\ss}e 70 \\ 
D-2000 Hamburg 13 \\ 
\verb$schmidt@rz.informatik.uni-hamburg.dbp.de$}
     }

\begin{document}
\maketitle
% \thispagestyle{empty} ##

\begin{abstract}
\noindent The DBPL language orthogonally integrates sets and first-order
predicates into a strongly and statically typed programming language,
and the DBPL system supports the language with full database
functionality including persistence, query optimization and transaction
management. The application of modern language technology to database
concepts results in new insights into the relationship between types and
schemas, expressions/iterators and queries, selectors and views, or
functions and transactions. Furthermore, it allows the exploitation of
type theory and formal semantics of programming languages and thus
connects database application development with results from program
specification and verification.

\end{abstract}

\section{On Data-Intensive Applications}

The development of data-intensive applications, such as information
systems or design applications in mechanical or software engineering,
involves requirements modeling, design specification, and, last but not
least, the implementation and maintenance of large database application
programs. Although the database programming language DBPL concentrates
on the last issue and offers a uniform and consistent framework for the
{\em efficient implementation} of entire database applications, DBPL was
designed, extended and evaluated with a specification and design
methodology in mind.
                          
Data-intensive applications may be characterized by their needs to model
and manipulate {\em heavily constrained} data that are {\em long-lived}
and {\em shared} by a user community. These requirements result directly
from the fact that databases serve as (partial) representations of some
organizational unit or physical structure that exist in their own
constraining context and on their own time-scale independent of any
computer system. Due to the size of the target system and the level of
detail by which it is represented, such representational data may become
extremely voluminous -- in current data-intensive applications up to
$O(10^{9})$ or even higher.  In strong contrast to the need for global
management of large amounts of {\em representational data},
data-intensive applications also have a strong demand to process small
amounts of local {\em computational data} that implement individual
states or state transitions.

In essence, it is that broad spectrum of demands -- the difference in
purpose, size, lifetime, availability etc.\ of data -- and the need to
cope with all these demands within a single conceptual framework that
has to guide the design of a database programming language.

DBPL \cite{SEM88,ScMa90} is a successor of Pascal/R \cite{Schm77} and
addresses the above issues by incorporating a set- and rule-based view
of relational database modelling into the well-understood system
programming language Modula-2.  DBPL extends Modula-2 into three
dimensions:

\begin{itemize}
\item bulk data management through a data type {\em set (relation)};
\item abstraction from bulk iteration through {\em access expressions};  
\item {\em persistent modules} and {\em transactions} for sharing, concurrency and recovery control.
\end{itemize}


\section{Conceptual DBPL Foundations}

The leitmotiv behind the DBPL design was the exploitation and {\em
integration} of a solid, well-understood conceptual framework capable to
consistently capture the wide range of data-intensive application
requirements outline above --- and not the desire to implement new
theoretical concepts in isolation.

Classical \e{type systems} and the \e{relational data model} constitute
the cornerstones of the DBPL language design. Both contributions provide
an enlightening foundation from a technological as well as from a
theoretical point of view. However, the exciting new experiences are
made at the borderline where types (and expressions, selectors,
functions \ldots) and data models (and queries, views, transactions
\ldots) meet and have to be understood, one in the light of the other.

During DBPL development, several, mostly unnecessary and ad-hoc
restrictions on both sides became obvious. Some limitations, as, for
example, the lack of data {\em type orthogonality} (i.e. the restriction
to first normal form relations) had already been realized, others, like
{\em orthogonal persistence} (providing persistence for all types, not
just for relations) were new for the database community (but already
discovered by the persistent programming language people). Other
insights as, for example, the relationship between abstract access
expressions and queries, iterators and views, or between recursive
access expressions and deductive databases, came as surprises, at least
to us.

Globally speaking, the DBPL language and system both heavily rely on
conceptual and theoretical input from following areas:
\begin{description}

\item[Programming Language Foundations:] typing, scoping, binding; 
\cite{MaSc89,ScMa90a}
\item[Compilation Technology:] type checking, local data flow analysis,
intermediate languages, separate compilation; \cite{ScMa90b}
\item[Extended Relational Models:] data constructors, bulk operations, query
languages and their expressive power; 
\item[Query Optimization:] transformations at compile- and
run-time, cost models, search strategies; \cite{JaKo83}
\item[Recursive Query Evaluation:] fixed point semantics, stratification,
recursive rule definition; \cite{ERMS91}
\item[Concurrency Control:] multi-level concurrency control and recovery
for complex objects; \cite{ScMa90b}
\item[Distribution Models:] transaction procedures, 
compensating transactions \cite{JGLR88,JLRS88}.
\end{description}

In summary, one can say that the DBPL project stayed more or less inside
the boundaries drawn by conventional language technology (e.g. by {\em
mathematical} typing schemes) and concentrated on improving the
linguistic support for state-of-the-art database technology. Currently,
however, we are heavily intrigued by novel programming language concepts
(e.g. by {\em taxonomical} typing schemes \cite{Card88b}) and are quite
positive that they provide the basis for substantial extensions of
database technology \cite{MaSc91}.


\section{Language Design Considerations}

The DBPL language design is also influenced by requirements of the DAIDA
environment for database application development
\cite{SWBM89,BJMSV89,JMW*90} that asks for a target language that
makes extensive use of sets and first-order predicates, and provides a
complete separation of typing and persistence.

An essential guideline for the design of DBPL can be characterized by
the slogan ``power through orthogonality''.  Instead of designing a new
language from scratch (with its own naming, binding, scoping and typing
rules), DBPL extends an existing language and puts particular emphasis
on the interoperability of the new ``database'' concepts with those
already present in the programming language.  In particular, DBPL aims
to overcome the traditional competence and impedance mismatch between
programming languages and database management systems by providing

\begin{itemize} 
\item a uniform treatment of volatile and persistent data (there are,
for example, relational variables local to DBPL procedures or as
function parameters);
\item a uniform treatment of large quantities of objects with a simple
structure and small quantities of objects with a complex structure, as
well as
\item a uniform (static) compatibility check between the declaration and the
utilization of each value.
\end{itemize} 
         
Implementation details (e.g., storage layout of records, clustering of
data, existence of secondary index structures, query evaluation
strategies, concurrency and recovery mechanisms) are deliberately hidden
from the DBPL programmer. A key idea in the design and implementation of
DBPL is to let the runtime system choose appropriate implementation
strategies based on ``high-level'' information extracted from the
application programs. As it turns out, the widespread use of {\em access
expressions} (i.e. first-order logic abstractions of bulk data access)
in typical DBPL programs facilitates such an approach.

DBPL also follows Modula-2 by occasionally sacrificing ``language
orthogonality'' out of engineering and efficiency considerations.  For
example, DBPL does not support persistence of pointers and procedure
variables. This was not only motivated by the predominance of
associative identification mechanisms in the classical Relational Data
Model, but also by the far-reaching consequences of this identification
mechanism on the concurrency control and distribution support
\cite{BJS86,JLRS88}.

Modula-2 was chosen as the basis for DBPL because of its software
engineering qualities. It provides a clear module concept and a strict
type system and is further excelled by its balance between simplicity
and expressive power. Finally, our group had a long tradition in
Modula-2 compiler construction.

A design decision with far-reaching consequences is the compatibility
between DBPL and Modula-2. DBPL is designed to be fully upward
compatible with Modula-2, i.e. every correct Modula-2 program has to be
correct DBPL program. This decision not only limits the freedom in
language extensions (e.g. the keyword \e{SET} is already used for bit
sets in Modula-2 and is not available for ``true'' sets in DBPL), but
also forces adherence to language mechanisms (e.g. variant records,
string handling, local modules) for which nowadays ``better'' solutions
are available. The main advantage of our compatibility decision is to
lower the conceptual and technological burden for DBPL users since they
do not have to learn yet another language.  Furthermore, this decision
allows to smoothly integrate DBPL into existing, fully-fledged software
development environments (e.g.  debuggers, profilers or version
managers), to benefit from software libraries and from the interfaces to
a variety of other languages and systems.

In contrast to some persistent programming languages, the evolution of
database schemata and of application programs is left outside the scope
of DBPL. This was based on the insight that this is a complex issue in
itself that should be delegated to a specific environment (see DAIDA
environment, e.g. \cite{JMW*90}).
                                                                   
To summarize, the most prominent feature of the DBPL language is the
type-complete integration of sets and first-order predicates into a
strongly typed, monomorphic language with persistence as an orthogonal
property of individual compilation units. The DBPL system is further
characterized by covering a wide range of operational database demands,
such as query optimization, transaction and distribution management and
computer-aided support for database application development.  Readers
interested in specific language and system aspects are referred to the
following publications \cite{JLS85,SEM88,JGLR88,MaSc89,SGLJ89,ScMa89,ScMa90,ScMa90a,ScMa90b}


\section{The DBPL System and its Use}

The DBPL project always had a strong commitment to implementability. A
multi-user DBPL system under VAX/VMS is in use at the Universities of
Frankfurt and Hamburg since 1985 for lab courses on database
programming.  There exist several DBPL system extensions that experiment
with alternatives for concurrency (optimistic, pessimistic and mixed
strategies) \cite{BJS86} and integrity control \cite{Bott90}, storage
structures for complex objects, recursive queries \cite{JLS85,ScLi85}
and distribution \cite{JLRS88,JGLR88}. The construction of a distributed
DBPL system is based on ISO/OSI communication standards and involves,
for example, a re-implementation of the DBPL compiler to generate native
code for IBM-PC/AT clients in cooperation with VAX/VMS servers.

During the last year a substantial effort was made to integrate the
experience gained with these prototypes into a new, portable
implementation of the DBPL runtime system on various platforms (VAX/VMS,
Sun-3, Sun-4/Unix). By utilizing Sun's optimizing compiler backend, the
DBPL compiler achieves ``production-quality'' performance and
interoperability. Implementation aspects of the various layers of the
DBPL database system are discussed in \cite{ScMa90b}.

We expect the DBPL language and system to be used in research and devleopment mainly for the
following three tasks: 

\begin{description} 

\item[Concept validation:] As outlined above, it is often impossible to estimate the value of a
proposed system solution (e.g. of a new concurrency control protocol or
a new workstation-server architecture) by a single figure (e.g. maximum
transaction throughput or number of packets to be transmitted). In many
cases, the interaction with several system components (e.g. the recovery
management or the query optimizer), or the lack of universality severly
impairs the utility of a seemingly advantageous paper-and-pencil
solution.

\item[Database education:] Our experience in using DBPL intensively in lab
classes convinces us that it is an appropriate tool for teaching the
essential problems and solutions in database application development.
Without being distracted by idiosyncratic surface syntax and
deficiencies of traditional preprocessor database interfaces, it is much
easier to isolate and communicate the repeating patterns in database
applications and to concentrate on an \e{abstract and complete} picture
of database application programming.

\item [Application prototyping:] From Modula-2 the DBPL language has
inherited software engineering qualities that can not be found in
commercial database environments and which qualify DBPL as an
appropriate tool for designs and implementations. This use of DBPL is
further supported by the quality of the commercial platforms
(DEC/VAX/VMS and SUN/RISC/UNIX) on which the DBPL system is realized and
the depth of its integration into professional environmets for software
development and maintenance.\footnote{The DBPL system is ready for
external distribution through Hamburg University as of late spring
1991}.
\end{description}

\bibliographystyle{florian}
\bibliography{../bib/florian}
\end{document}