Although World Wide Web use of the Internet has been growing at a phenomenal rate over the past 3 years, there remains a dearth of interesting and useful client/server applications. Traditional programming languages do not provide an optimal environment for programming such applications because of the syntactical and paradigmatic distance between traditional languages and the target output, namely HTML.
The stateless and dynamic nature of the WWW requires a level of support from traditional programming languages for which they are hard pressed to provide. Must each application programmer write his or her own tool set to accomplish goals which are fundamentally similar across applications?
This paper presents an alternative language choice for the design and implementation of Web applications called META-HTMLtm, and describes the three methods of delivery that have been completed. Requirements, desirable features, and implementation considerations are presented. Finally, future directions of the language and implementation are discussed.
Hello World!
Traditional programming languages attempt to solve the general problem of expressing algorithms in a way which can be efficiently implemented on a generic set of hardware. Scripting languages attempt to hide the details of the underlying hardware (and often software) implementation by providing high-level ``primitives'' which can be used to provide functionality in an atomic fashion.
Looked at in this light, one may consider the Hypertext Markup
Language (HTML) as a weak scripting language meant to be
executed on a generic piece of hardware called a browser. It
contains high-level primitives for controlling the output (e.g.,
<P ALIGN=RIGHT> ... </P>
), and for receiving
input from the user (e.g., <INPUT TYPE=TEXT
...>
). Yet, unlike a true scripting language, HTML is
missing variables, iteration, data structures and other general
programming constructs.
Because the capabilities of the underlying hardware (i.e., the browser) are extremely limited, writing even simple Web applications requires the server side of the connection to do genuine computational work. This includes processing input, keeping track of state, and decision making based on that data, resulting in an output view for the client.
The necessity of allowing the programmer to dynamically change the client's output view based upon client data was realized early on in the development of the Web, and a general mechanism called the Common Gateway Interface (CGI) was designed.
However, the implementation languages available for use in CGI programs have been limited to the world of general programming languages. Compiled languages such as C and Pascal, and interpreted languages such as Perl and TCL have proven themselves as less than optimal languages for the implementation of easily maintainable Web applications through the length of time that it takes to create an application, and the length of time it takes a programmer who is new to the existing application code to ``get up to speed''.
The syntax of META-HTMLtm is the syntax of HTML. The HTML syntax model is quite simple; language statements may only appear within matched less-than (<) and greater-than (>) symbols, the keyword must appear directly after the less-than character without intervening spaces, and the keyword must be on a list which the browser supports. If the keyword is not one which the browser understands, the open angle-bracket, containing text, and close angle-bracket are ignored. All other text is treated as straight text to be displayed.
The model used in META-HTMLtm is identical. However, the language interpreter does not recognize any HTML keywords, and thus passes them on to the browser to be executed.
A short example is in order:
<body <get-var body-spec>>
In the above example, the text <get-var body-spec>
is a META-HTMLtm language statement which retrieves the
value stored in the logical name body-spec
. If that
variable has a value, the value replaces the entire language
statement. If that variable has no value, then the empty string
replaces the entire language statement. The text <body ...>
is an HTML statement, and is ignored by the
META-HTMLtm language interpreter. Thus, the resultant text sent to
the browser is either:
<body >
if the variable body-spec
has no
value, or
<body bgcolor="ffffff">
if the variable body-spec
has the
value bgcolor="ffffff"
as its value.
1.2 General Programming Constructs
Variables allow the type of symbolic manipulation that is required for the vast majority of programming functions. A collected group of such variables is called a package in META-HTMLtm. Among other things, packages allow programs to switch contexts easily and rapidly. META-HTMLtm has primitive statements which directly manipulate the contents of packages (thus allowing direct and indirect symbol manipulation), which create and destroy packages, and which copy packages.
if
,
ifeq
, when
, var-case
, and
while
. Boolean operators can be used to combine tests
and invert return values: and
, or
, and
not
are all provided.The META-HTMLtm flow control statements all have at least two parts: the test and the action. Test clauses are evaluated, the result of that evaluation is checked for non-nullness, and, if an action is indicated, that action is then executed.
This issue is addressed by the inclusion of fundamental arithmetic operations, including addition, subtraction, multiplication, division, and remainder operators. A few boolean operations on numeric quantities are also provided; there are less than, greater than, and equality test operators.
The arithmetic operators use either integer or floating point operations depending on the granularity of the input and desired output.
1.2.4 Macros and Substitutions
If we do not require a change to the Hyper-Text Transfer Protocol (HTTP), exactly one solution exists for the problem of storing and recalling user input history across a stateless connection: the client must pass state information back to the server for each interaction.
If all of the interaction history must be kept, and is passed back and forth between the browser and the server, the growth of this information becomes problematic. Existing Web servers often have static limits on the amount of information which can be passed in an Universal Resource Locater (URL), and existing browsers also have static limits on the amount of information that can be represented in an URL.
One solution addressing this problem is to represent large amounts of information with a single (smaller) piece of information, called a token. Such tokens can be generated by the server when the user first connects, and can be passed back and forth by browser and server. When the user submits some information, the new information is saved away in less ephemeral storage, using the token as a key under which the data may be found. Previously submitted information can be retrieved from the storage in the same fashion.
The method used to pass the token can be to encode the token as part of the URL that the user sees, or, when allowable, to utilize a feature of some browsers (e.g., Netscape) which allows the embedding of token information in the surrounding HTTP.
Note that the token should be a sufficiently random set of bits that it is difficult to derive from any other information, such as the time of day, the hostname of the server machine, etc. Since the token represents the input history of a particular user, (essentially identifying that user uniquely to the server), it is conceivable that another user could masquerade as the owner of a token. Solutions for this problem include the prevention of network sniffing through the use of strong encryption(2) and continued randomization of the token information that is passed back and forth. Such solutions are not discussed in detail here.
Naturally, this situation is less than ideal. Programs written this way are generally strongly tied to a specific database product, which, in turn, can limit the hardware platforms available to run the application, limit the functionality of the application, and/or limit the future of the application (e.g., a necessary database feature might be missing).
META-HTMLtm addresses this issue by providing a small set of database manipulation primitives, thus making a clean abstraction to the underlying database engine. A variety of different database engine products can be linked into the language interpreter, so engines with ISAM capabilities, SQL parsers, and the like may be utilized. In fact, the programmer may easily switch between different database engines in the course of a single program, thereby maximizing the benefits of each.
The abstraction provides database creation, file locking, record-locking, key storage and lookup, and generic query operations as part of the basic functionality. Additional features of a specific database engine model can be utilized by means of a single primitive.
A single string operator, match
, is provided. This
operator allows for complex string comparision, and substring matching
and extraction. No string indexing functions are provided; there
apparently is no driving need for such functions.
A pseudo-random number generator is provided. The generator can be restarted at a specific point to allow testing of code which utilizes random numbers with a known set of input values.
Other functionality which is deemed desirable includes:
redirect
, and
cgi-encode
are examples.
This approach has paid off in many ways. Since tasks were well-defined, the implementation of any particular task was a quick and straight-forward process. Separate libraries for symbol manipulation, textual buffer manipulation, textual translations, memory management, database control functions, and session tracking and manipulation were written and tested individually, and then linked together with a language parser, implementing the complete command set of META-HTMLtm.
Three models were chosen as targets: a standalone language processor, which simply takes a filename of META-HTMLtm statements and produces an HTML page as output, a standalone CGI program, which takes information from the environment and produces an HTML page as output, and a full featured Web server, with the interpreter built in.
Additionally, a source-level debugger similar in look and feel to GDB has been implemented, with features including symbol and file name completion, Emacs style command line editing, full META-HTMLtm language comprehension (you can simply type in META-HTMLtm statements and see the results immediately), breakpoints, single-step, recursive debugging levels, and more.
The driving force for each of the interpreter implementations was to provide the most transparent use of META-HTMLtm as possible. Points were given for minimizing user configuration, for ease of use in hiding the language processor from the connecting client, for reductions in execution overhead, for lowest impact on code portability, and for minimizing required language statements in the target file.
This model amortizes the cost of installation and use over all of the criteria considered. It requires only minimal server configuration, and was found to work with the full set of servers considered(3), requires essentially no installation other than placing the binary somewhere on the system, but the programmer must take care to perform various initializations within each target file, and the language interpreter must be invoked for each target file individually.
The only pre-assigned variables are that of the calling environment; these are the same variables passed to any CGI program by existing Web servers and they are provided as META-HTMLtm variables for the target file, and each target file written for this model requires minor massaging if it is to be moved to another site or execution model.
Thus, the model is a good one for sites which only require the features of META-HTMLtm for a small subset of their pages.
The CGI interpreter takes its information from the calling environment(4) and acts as a mini-server, processing the information, making decisions on the type of return document, returning images and the output of other CGI programs as well as processed META-HTMLtm files.
Features include the ability for the programmer to specify a prologue document (a file of META-HTMLtm code which is interpreted before the target), various translations on the URL which was supplied, and the definition of a set of variables describing the environment and various aspects of the target file (i.e., the physical location of the target file on disk, the URL that the client requested, etc.).
Target files written for this model are transferrable without modification to another Web server (including one which interprets META-HTMLtm) or site, or can be easily relocated within the site.
One obvious benefit of the server model is the ability for the user to write META-HTMLtm code which can be executed in the server environment, effectively creating a user-level programmable server. For example, a META-HTMLtm function can be defined which will be called after the client request is read, but before the request is acted upon. Based on information found in the client request, the called function may change aspects of the request, perform calculations and/or logging functions, or any other operation which is of interest to the programmer (e.g., one could send mail whenever a specific host accesses a specific URL).
Configuration of the server is done by writing META-HTMLtm statements in a configuration file. This gives the user the power of a genuine programming language, and allows maximum flexibility.
Target files written for this model are transferrable without modification to another host, or to the mini-server model.
The compiler construction is underway at the same time; areas of refinement include selection of fundamental machine operators and instruction caching.
This can be accomplished in two ways: direct compilation of META-HTMLtm statements into C statements, or, the implementation of a byte-code to C compiler. The latter approach has the advantage of allowing the language syntax to undergo changes without affecting the compiler, and would take advantage of the work already done during byte-compilation, such as loop unrolling and peephole optimization.
1. | For example, language variables. |
2. | RSA, triple-DES, Public-key, and other encryption methods are widely available and detailed elsewhere. Many of these mechanisms are implemented in various browsers and servers. |
3. | The standalone filter works easily with servers from NCSA, CERN, Netscape, and even the Plexus server written in Perl. All of the tests have only been run under Unix-like operating systems. |
4. | E.g., PATH_INFO, the existing input stream, etc. |