CA2201882A1 - Method and apparatus for translating source code from one high-level computer language to another - Google Patents

Abstract

A method, system, apparatus, and program for translating one computer language to another using doubly-rooted tree data structures. A doubly-rooted tree (7, 19) is the combination of two sets of hierarchically related objects sharing a common set of leaves. An N-rooted tree is also described. When a doubly-rooted tree is constructed in the specified manner and then translated to a second doubly-rooted tree, source language code is transformed into target language code. In addition, the translation preserves preprocessor characteristics of the source language code including macros, conditionally compiled regions of code, source inclusion statements, and comments.

Description

-~ 0 96/11437 PCT~US95/13260 METHOD AND APPARATUS FOR TRANSLATING SOURCE CODE
FROM ONE HIGH-LEVEL CO~IPUTER LANGUAGE TO ANOTHER

BAC~GRO~nND OF THE IN~ENrrION
This invention relates to automated translation between high-level computer programming languages.
This invention relates particularly to improved preser~ation in a target high-level lanquage of preprocesSOr characteristics (such as macros, source file inclusion structure, and ~om~entary) contained in a source high-level langua~e. A feature of this invention is that preprocessor characteristics need not nece~sarily be processed by a preprocessor.
High-level computer lanquages enable computer programmers to ~mmt~ni cate with computers. Statements programmers write in a computer languaqe form a computer program which in turn instructs a computer to perform a set of tasks. "Compilation"
is the manner in which high-level computer lanquage programs are converted into instructions, generally called machine code, which the computer can understand and execute. A compiler is a computer program which performs this translation.
In general, each brand of computer understands a different set of machine code instructions. Therefore, a different compiler must exist for each computer to translate a high-level computer lanquage. Because compilers for every high-level computer lanquage do not exist on every brand of computer, not every program can execute on every machine. Progr~mers can only write programs in the lanquages for which compilers exist for their tarqet computers.
Nonetheless, it is highly desirable to have a single computer proqram run on as many brands of computers as possible.
Application programs are typically complex and difflcult to write; rewriting prosrams in multiple ianguages to run on multiple brands of computers is impractical. Likewise, W O 96111437 PCTrUS9~/13260 compilers are difficult to write; providing them for every language for every brand of computer is equally impractical.
One way of addressing these problems has been the development of well known, widely used, st~n~Ardized high-level languages. r Compilers for these languages are available for a wide variety of computers.
The development of stAn~Ardized languages has not been a complete solution. There exist numerous high-level languages, and many large programs written in them, which are exotic, highly specialized, little used, obsolete, or designed for specific computers. Many computers do not have compilers available for these languages.
Because many high-level computer languages, whether or not they are standardized, cannot be compiled on every computer, programs have to be translated to other languages. While translation can be done by hand, it i8 a laborious, time consuming, and expensive process prone to error. ~o address this problem, automatic translators have been and continue to be developed to translate programs written in one high-level language to another.
Automatic translators may be used in either of two distinct strategies to solve the problem of an unavailable compiler for a particular language on a particular computer. First, programmers may continue to write and maintain programs in the original source language. The translator converts these programs into intermediate code in a target language. An available compiler for the target language then converts this intermediate code into machine code which the target computer can understand. Although the target language is usually a standard widely availa~le language, the translator does not have to produce readable or maint~in~hle source code.
The second strategy requires a translator to produce readable and maintainable code. PL~Y' ammers going this route want to AhA~on the original language in favor of the target.
Building this type of translator is a more difficult task and is the foc~s of this invention.
Prior art attempts to build translators which produce readable code have had differing goals and various levels of 0 96/11437 PCT~US95/13260 success. Syntax of one high-level language has been successfully transformed into syntax of another high-level lanquage. Some translators have produeed att,ractively formatted target code. While source code comments have been migrated to target code, their placement has not always been optimal, Translators have also attempted to transform the style of programs to make them more readable. Others have used knowledge-based systems ~o extract the meaning of the source program and rewrite it in the target language.
However, prior art translators have universally failed adequately to preserve programming constructs generally known as preprocessor characteristics. Many high-level languages include a preprocessor language separate ~rom but coexisting with the lanquage itself. Characteristics (which are also herein referred to as invocation expressions) of the preprocessor language may include a conditional compilation mechanism, a macro m~Ch~is~ a source inclusion mp~h~nism, a variety of compiler directives, and a comment ~ ch~n;~n. At the risk of oversimplification, the preprocessor allows programmers to use shorthand expressions for lonyer constructs. Thus, invoking the shorthand expression triggers a text substitution when the source code is run through the preprocessor.
The failure of translators to h~le adequately preprocessor characteristics is a well known problem. Experts in the field, when considering how to replace a source language macro definition with a target language macro definition, have stated: "We do not know o~ a general, fully automatic m~h~
for achieving this replacPm~nt. n Two suggestions have been made:
- Attempt to parse macro bodies outside of their context of use, with the hope that some well-formed and complete source language constructs will translate them correctly.
- ~ecognize common sequences of tokens in macro bodies via pattern matching.
The ~ormer is not a general solution because languages may make no restrictions for using macros, and such restrictions, in any event, would make macros suscepti~le to semantic errors. The latter sugqestion would only work in special cases.

-WO96/11437 PCT~S95/13260 ~

SUMMARY OF THE INVENTION
A general object of the present invention is to provide a method and apparatus for automatically-translating the source code from one high level computer language to the source code of another language.
A specific object of the present invention is to provide a method of translating code from a source computer language to a target while preserving the preprocessor characteristics of the code as written in the source language.
Another object of the present invention is to provide a data structure useful ~or automated translation, whether it be from one computer language to another or from a natural language to another. The data structure, termed a doubly-rooted tree, combines two prior art tree data structures so that their leaves coincide. By creating and filling the doubly-rooted tree in a specified manner, and then translating it to a second doubly-rooted tree, preprocessor characteristics of a computer language may be preserved while translating other statements, constructs, and expressions.
A dou~ly-rooted tree may serve as the basis for more complex data structures. For example, additional trees may be joined at the leaves to form an N-rooted tree. Such a linked set of trees has computer database applications.
Other and further objects of the present invention will be apparent from the following description and claims and are illustrated in the accompanying drawings which, by way of illustration, show preferred embodiments of the present invention and the principles thereof and what are now considered to be the best mode contemplated for applying these principles.
Other em~odiments o~ the invention embodying the same or equivalent principles may be used and structural changes may be made as desired by those skilled in the art without departing from the present invention and the purview of the appended claims.

-~ 096/11437 PCTrUS95/13260 BRIEF DESC~IPTION OF THE D~WINGS
In the Drawings:
FIG. 1 ~epicts a prior art tree structure.
FIG. 2 depicts a doubly-rooted tree data structure, joined at the leaves, in accordance with the present invention.
FIG. 2A is a simplified block diagram of a computing system for use with the present invention;
FIG. 3 is a block diagram depicting data flow through a simple prior art compiler.
lo FIG. ~ is a block diagram depicting data flow through an automated source code translator in accordance with one em~odiment of the present invention.
FIG. s is a flowchart describing a method for building a ~irst tree in a doubly-rooted tree in accordance with one em~odiment of the present invention. The method builds the tree from the ~op down in depth first traversal order.
FIG. 6 is a flowchart descri~ing the building of a second tree in the dou~ly rooted tree introduced in Fig. 5, in accor~n~ with one em~o~im~t of the present invention. An inorder tra~ersal of the first tree yields an ordered set of leaves thst is input to the parser 623. The method ~uilds the tree from the bottom up in depth first traversal order.
FIG. 7 is a flowchart descri~ing the translation of a second tree in a first doubly rooted tree to a corresponding 2S tree in a second doubly rooted tree in accordance ~ith one embo~; ~Pnt of the present invention.
FIG. 8 is a flowchart describing the translation of a first tree in the first doubly rooted tree referred to in FIG. 5 to a corresponding tree in a sPcn~ doubly rooted tree in accor~nr~
with one t~m4odiment of the present invention.
FIG. 9 depicts an example fragment tree and syntax tree, using a high-level computer language, pTAL, as the source language.
FIG. 10 depicts the ex~mple pTAL source fragment tree and source syntax tree, including the leaves (tokens) that link them, introduced in FIG. 9. It also shows a target syntax tree of another high-level co~puter language, C+~, that represents a .

WO96/11437 PCT~S95/13260 ~

translation of the source syntax tree and hal~ of the target doubly-rooted tree.
FIG. 1l depicts the source doubly-rooted tree and the target dou~ly-rooted tree introduced in FIG. l0, after the translation o~ the source fragment tree. All four trees are illustrated: the example pTAL source fragment tree and source syntax tree as well as semantically equivalent C++ target fragment tree and target syntax tree.

DET~ILED DESCRIPTION OF q~E lNV~;N~l~lON
This invention implements a strategy for translating legacy software to a different high-level progr~mming language. This translation process preserves text preprocessor m~-hAn; ~me: such as macros, conditionally compiled regions of code, and source inclusion. This section specifically discusses the design of the Rosetta Translato,, which implements this strategy. The Rosetta Translator translates from Tandem's porta~le Transaction Application Language (pTAL) to C++.
8Ource-To-8OurcQ Tr~nslation Source-to-source translation to a st~n~d, portable, and widely-available language, such a~ C, has been shown to be an excellent means of making exotic languages availa~le on many platforms 2, ~ 4~ 9. Programmers continue to write and maintain programs in the original source lanquage, and use the translated source as intermediate code. These translators need not address the problems inherent in producing readable and maintainable code:
preserving macro definitions and program structure, and ret~ining ~ingful n~m;ng, comment placement, and format.
Attempts have ~een made to implement source-to-source 30 translators that transform the style of a program, thus rendering it more readable, or to extract the m~i ng of a program and use a knowledge-based system to express the program in a higher-level language 8. We were unable to apply such techniques to our advantage.
pTAL, Cll, and many high-level progr~mm;~g languages consist of a preprocessor language separate from hut coexisting with a high-level language. The preprocessor language might include a conditional compilaticn mechanism, a macro mechanism, a source 096/11437 PCTrUS9S/13260 inclusion mecnanism, a variet~ of compiler d'rectives, and a comment mechanism. The preprocessor language need not be implemented by a separate preprocessor. The Rosetta Translator not only translates high-level language constructs but also preprocessor cons~ructs; preserving the latter is a difficu7t ~ut important part of producing reada~le and maintaina~le code.
While some published acco~nts descr~e translators that produce readable high-level code l~ 5, we have encountered no pu~lished accounts of experience with translators which produce lo readable and maintainable code and which preserve preprocessor mechanisms.
L~5uz~ge8 The source language, pTAL, is a proprietary block-structured systems programmins lanquage, originally derived ~rom Algol-60.
The vast majority of Tandem's system code was written in pTAL or its precursor, TAL. The Rosetta Translator uses pTAL as its source language instead of TAL because pTAL excludes various architecture-specific TAL ~eatures that are not availa~le in s~n~rd progr~mming languages, such as assem~ly language code statements and ar~itrary register manipulation. All TAL code will eventually migrate to pTAL.
The translated code is dominated by the C language subset of C+~, but includes occasional constructs specific to C++. The Rosetta Translator does not generate object-oriented code. C+l is a much better target language than C for a number of reasons, including strong type chec~ing and easy emulation of several important pT~L language constructs, and the opportunity to use the ob~ect-oriented paradigm in the future. The ideas presented here apply to other progr~mming languages containing preprocessor m~hAni~s that one would have a practical need to translate.
Co~cepts The ~ollowing concepts are key to this description of a method for preserYing macros.
V~rtu~l 80urca -3~ A program's virtual source is a stream of tokens that a compiler parses to create a syntax tree representing that program.

~irtu~l 50urce Production MQch~nisms W O 96/11437 PCT~US95/13260 Vir~ual source procuction mechanisms are used Dy a scanner or preprocessor to create virtual source. Virtual source production mechanisms available in pTAL and C+~ are:
- source inclusion of files 5- macro expansion - macro formal parameter substitution r A virtual source production ~Anism can appear anywhere in a source file and yield any portion o~ a syntactic construct; it need not honor syntactic construct ~ounàaries. Virtual source does not lonecessarily exist in textual form in any single source file; the token stream is created by a series of source inclusions, macro expansions, and macro formal parameter substitutions.
The Rosetta Translator translates virtual source production mechanisms as well as hi~h-level language constructs. It does not 15merely expand the vir~ual source with a preprocessor before performing its translation. The macro that the programmer ~new and loved appears in the ~_anslated code, and means the same thing.
The file structure of the program is also preserved.
Fr~gment A "fragment~ represents the result of an invocation of a virtual source production m~ ni ~: source file inclusion, macro expansion, or macro formal parameter substitution. Each fragment is linked to the tokens that make up its invocation syntax: a macro formal parameter, a macro invocation (including the parameter list), or a token representing a source inclusion directive. Each fragment contains the to~nc composing its expansion: a macro actual parameter, a macro bo~y, or an included file. The to~ens composing the invocation syntax are themselves contained in another fr~P~t, which represents its context of use. For example, a macro invocation can be embedded in a macro body, or in the main program f ile.
Fragments can be divided into n partitions n by compiler directives. Because some compiler directives affect the se~antics of the program, and some determine what text is scanned (for example, conditional compilation directives), directive ~oundaries must ~e preserved.

-~ WO96/114~7 PCT~S95/13260 Partit_ons are bounded by:
- compiler directives - beginnin~ and end o~ macro bodies in-a macro definition - beginning and end of a file The details of preser~inq partition ~oundaries are not discussed in this document.
Fragment Tree A fragment tree represents each source inclusion, macro expansion, and macro actual parameter su~stitution that was employed to create the virtual source.
8t~tic Fr~y~ents A static fragment is a mold from which fraqments are made;
each fragment is an instance of a static fragment. A source file is a static ~ragment, and it may contain nested static frasments.
The Rosetta Translator pieces together textual representations of C++ fragments to form C++ output files, fitting a textual representation of a macro ~ody, for example, into its definition.
Because the Rosetta Translator's design requires that each source language macro maps to exactly one target language macro, it checks the textual consistency of every macro use, to ensure that the macro text "works" in all contexts of use. It checks the textual consistency between fraqments generated during previous translation sessions and those generated during the n-urrent" session, to ensure that pieces of an included file "work" in all contexts of use.
Translator ~ver~iew This section introduces a "fragment tree" as a means of representing macro invocations, macro actual parameter substitutions, and source inclusions in a proqram. It ~ives an overview of the translator phases, including steps to preserve the ~ragment tree. It then describes in detail the structure o~ the fragment tr~e and gives a fraqment tree translation method.
The Rosetta Translator behaves like a normal compiler, with a few additional steps. A pTAL syntax tree represents the ~ 3~ syntactic structure of the virtual source. Tokens appear on the leaves of the otherwise traditional a~stract syntax tree. Each W O 96/11437 PCTrUS95/13260 token is also associated with a fragment, indicating which virtual source production mechanism brought ~t into the virtual souroe.
The syntax tree and the fragment tree are said to be "joined at the tokens."
The scanning phase builds the pTAL ~ragment tree. The pTAL
syntax tree is translated to a C++ syntax tree. An additional translation step converts the pTAL fragment tree to the C++
fragment tree.
After translation, each Cf+ macro invocation fragment or source inclusion fragment contains a sequence of C++ tokPnc that represents text belonging in that macro ~ody or included file. The source generator pieces together translated instances to form a generated text file. Tokens and rommPnts are formatted (assigned file positions) based on the syntax that contains them, taking cues from the formatting of the corresponding source language construct.
FIG. ~ depicts a general diagram of the Rosetta Translator's data flow.
Following is some sample pTAL source code, and the translated C++ source code.

2 0 pTAL Code Ge~erated Ct+ Code def ine leqs (x) = a < x#; #def ine le~B ~x~ a ~ x ... ...
while less (42) do ....... while (less (42)) FIG. 9 illustrates the fragmen~ tree that represents the pTAL
while loop from the previous example. The tokens in the virtual source are in both the fragment tree and the syntax tree.
Properties of Pr~smqnt~
Fragments ~ave some interesting properties:
A pTAL fra~ t is semantically e~u~Yale~t to the C++
fraoment to which i t translates . For a macro to "work", it must expand to a m~ningful token stream in every context of use in the generated code. For a macro to be readable and maint~;n~hle, it must be invoked for the same purposes in the

3~ generated code as in the source code; it must provide the same benefit by encapsulating ecuivalent target language text. These considerations for macros also apply to code inc7uded from another ~ile and to macro actual parameters.

W O 96/11437 11 PCTrUS9S/13260 Tokens in a pl'AL fragment must be translated in their con text or con texts of use . Macros cannot be translated immediately at their point of definition because, even if the text could be parsed, semantic analysis would not be possible.
5 only when the macro is expanded in context is the semantic information necessary for translation available. Macros are, therefore, translated after their invocations have been ~Y~n~eA
and analyzed. The actual parameters of a parameterized macro and regions of included text are processed analogously. The bodies of unused macros are not translated, and unused actual parameters are not translated.
An important implication o~ this property is that shared interface files cont~;n;ng macro definitions or conditional compilation tend to be translated incrementally, as a side effect of translating the modules that use them.
A pTAL fra~ment should translate to the same C++ text in every context of use. A macro body is translated once for each expansion in the virtual source. The Rosetta Translator choofies to produce exactly one target language macro for each source language macro; this design decision requires that the translation of every instance of a macro expansion be textually identical. The Rosetta Translator can only be sure that the translation of a macro "works" in every context of use if every instance of the macro ~p~cion translates to textually identical generated code. A textual m; ~m~tch might occur, for example, when the syntax expressing two distinct operations is the same in the source language ~ut different in the target language. The pun (one phrase with two distinct n~ ni ngs) does not translate. Strategies for avoiding textual mismatches and for coping with mismatches that cannot be prevented are discussed later in this document. The same considerations apply to code included from another file and to macro actual parameters.
Construct trar~slations should not blur fragment boundaries.
Each pTAL syntactic construct is mapped to a corresponding C++
syntactic construct, and pTAL tokens within that construct are W O96/11437 PCTrUS95/13260 mapped to corresponding C++ tokens within the corresponding C~f construct. While the tokens in the resultant token stream need not maintain the same relative ordering as the pTAL tokens that generated them, fragment boundaries are points of ordering enforcement. Each token in a given C++ frasment is generated from some token in the same pTAL fragment. Strategies for avoiding fragment boundary violations and for coping with violations that cannot be prevented are discussed later in this document.
Tran~lat~ng the ~ynt~x Tree The Rosetta Translator translates the source language syntax tree, generating an equivalent target language syntax tree. The translation routines for most source language constructs generate idioms in the target language when appropriate.
The Rosetta Translator associates each generated target language token with one source language token. The "translation" of a source language token creates a target language token and establishes this association. The choice of which pTAL token is associated with a given C++ token determine~
the fragment in which that C++ token belongs: the C++
counterpart of the pTAL fragment in which the associated pTAL
token resides.
Some constructs allow a one-to-one mapping from source language tokens to target language tokens. If tokens do not map one-to-one, the Rosetta Translator chooses token mappings that m~Y;~;ze "semantic congruence, n that is, it tries to associate tokens with their logical counterparts. Choosing appropriate mappings helps ensure meaningful macro translations. The collection of tokens contained in a macro body (or macro actual parameter) in the source language tends to generate a collection with the same meaning in the target language.
FIG. 10 illustrates part of a pTAL syntax tree and frasment tree, joined by the pTAL tokens making up the virtual source.
The dashed lines depict links between each C++ token and the pTAL token that generated it. The tokens in the relational -96/11437 PCT~S9~/13260 expression map one-to-one from the source language to the target language. The source language "whilen keyword generates the target language parentheses tokens in-addition to the target language "while" token.
Translating the Fragment Tree After building the C++ syntax tree, the Rosetta Translator traverses the target language virtual source token stream, and builds the C+~ fragment tree, ~ased on the position in the pTAL
fragment tree of the source language token associated with each target language token encountered in the virtual source stream.
The method given in this section builds a target language fragment tree. Operations are as follows:
Attach(x, y): Attach x to y: y b~?C the parent of x. x is added positionally after pre eXisting siblings.
Translate(n): Translate n ttoken or fragment) to the target language version of the object, and return the translation.
Translation(n): Return the latest translation of the fragment n;
nil if it has not been translated.
Parenttn): If n is a token, return the fragment in which n resides. If n is a fragment, return the invocation syntax of the fragment (macro name token, for example).
IsOpen(n): Return true if the fragment n is open in the targ~t fragment tree. The "open" nodes in the target language fragment tree include the ancestors o~ the last token that was added to 2S the ~ragment tree. Adding a child to a fragment "closes" nodes that are no longer ancestors of the latest token added.
Begin by translating the main source include fragment. This fragment is initially open.
FOR (TargetToken = each token in the target language virtual source) BBGIN
SourceToken ~ the 30urce token a~30ciated with the current TargetToken SourcePrag ~ Parent (SourceToken) IF (IsOpen (Translation (SourcePrag))) THEN BEGIN
add a ne~ token to an exi3ting ~ragment that i8 already open ~/
TargetPrag - Translation (SourceFrag) Attach (TargetToken, TargetPrag) END ELSB BBGIN
J~ create a new ~ragment, and add the new token to it ~/
- TargetFrag - Translate (SourceFrag) Attach (TargetToken, TargetPrag) SourceInvSyntax - Parent (SourceFrag) TargetInvSyntax ~ Translate (SourceInvSyntax) Attach (TargetFrag, TargetInvSyntax) W O96/11437 PCT~US9~/13260 0 SourceFrag = Parent (SourceInvSyntax) /~ Create new ~ragment~ and invocation syntax until TargetToken is connected to the tree. ~/
WHILE (NOT IsOpen (Translation (SourceFrag)) ) BEGIN
TargetFrag Y Tran~late (Source~rag) Attach (TargetInvSyntax, TargetFrag) SourceInvSyntax , Parent (SourceFrag) TargetInvSyntax = Translate (SourceInvSyntax) Attach (TargetFrag, TargetInvSyntax) SourceFrag = Parent (SourceInvSyntax) END WHILE
TargetFrag = Translation (SourceFrag) Attach (TargetInvSyntax, TargetFrag);
END ELSE
END FOR

Co~e Replicat~on While the target fragment tree usually has a shape similar to the source frayment tree, this method allows for code replication and reordering to produce new subtrees or subtrees in new places (when, for example, the replicated or reordered code represents an entire macro expansion).
The "for" loop from the example in the next example uses a macro invocation in place of the control variable.
pTAI, Code Generaced C+~ Code define i(x) - an[x]#; #de~ine i(x) an[x]
for i(4) :~ O to 4 do for ( i(4) ~ 0; i(4) <~ 4; i(4)++) The Rosetta Translator preserves the semantics of the original code only if the code in the control variable of the for loop executes without side-effects. This issue is independent of the presence or absence of macro invocations in a for loop; the issue relates to the replication of code.
~xperienc~ With the Rosetta Trancl~tor Automated translation is an important vehicle for migrating critical system code out of a proprietary programming language.
Taken together, system products contain millions of lines of code, of which hundreds of tho-lc~n~ of lines are shared interfaces which contain many tho~lc~n~ of macros. These macros contain a wide variety of language constructs, and the language constructs are often not well-formed. The Rosetta Translator ~acilitates this migration. At this writiny, prel;min~ry translations have converted major parts of Tandem's core system software to C++.

.

W096/11437 PCT~S95113260 This section descri~es the principles and requirements that guided the design of the Rosetta Translator.
RQguirQments of the Ro~ett~ Transl~tor The Rosetta Translator was conceived as a way to reduce and eventually eliminate dependence on a proprietary systems programming language while supporting huge bodies of legacy software. To that end, the Rosetta Translator performs highly automatic translation from Tandem'~ portable Transaction Application Language (pTAL) to a subset of C++. The translated software is more portable and is compatible with commercially-available programming tools. New software can make use of the object-oriented programming paradigm while interoperating with legacy code. The resulting C++ source code is sufficiently readable and maintainable that the original pTAL source code can be retired.
The Rosetta Translator has several requirements:
The translation process must be highly automated. The Rosetta Translator must translate an entire product in a short elapsed time. The Rosetta Translator must be "highly automated, n but not necessarily completely automated. The Rosetta Toolset includes a set of problem analysis tools to speed elapsed translation time. Automatic source-to-source translators can be criticized for automatically translating only a subset of the source language or for generating obfuscatory code, and source-to-source translators that decline to translate some features of a language can be criticized as useless when the source code is very large. The Rosetta Translator is an industrial strength toolset that must not substantially subset the source language for the convenience of the translator. It also must make use of commonly used idiomatic features of the target language, and must avoid translations that, while strictly safe, are not particularly readable or maintainable.
The Rosetta Translator can take cues from a human expert to help it avoid generating awkward code.
The translation process must allow incremental translatio~
on a module-by-mod~le basis. To ease the transition from one W O96/11437 PCTrUS9~/13260 0 implementation language to another, a product must be able to export translated interfaces before translating all of its implementation modules, and it must be able to phase translation of the product, one module at a time. The resultant C++ code must be interoperable with pTAL code.
The resultant code must faithfully reproduce the semantics of the original program.
The resultant code must be readable and maint~inAhle. The generated program must be as readable and maintainable as the original code to a programmer familiar with both languages and the product that the code implements. A programmer familiar with the product must find that after translation, he or she still holds a valid mental ~oA~l of the implementation.
Identifiers must be mapped to reasonable spellings. Comments must appear adjacent to the code that they describe, even when that code has been moved. Patterns or pictures embedded in end-of-line comments must be preserved to the extent possible. The format of the code must appear natural in the target language, though some aspects o~ the format might follow the style and alignment o~ the original code, reflecting the preferences of the author. Macros defined in the source code must, to the extent possible, encapsulate the same meaning in the translated code.
The Rosetta Translator generates output that looks natural in its new language but does not significantly alter the design of the program. One of the great investments that software companies must make is in software maintenance, and one of the great assets that a so~tware main~ e~ might have is familiarity with the implementation details of a large and complex software system. Substantial redesign is best done by the programmer, who is not only best qualified to make a complex judgement call, but can later make good use of the experience of doing so.
The resultant code must be efficient. Execution speed of a generated program must be comparable to that of the original pTA~-coded program. While fulfillment of this goal lies more in -WO96/11437 PCT~S9~/13260 the hands of the compiler that implements the target language, we did have the opportunity to take into account performance of various constructs when designing their translations.
These requirements led to a number of challenges for the Rosetta Translator, including maintaining dual language interfaces, mapping tokens in lan~uage constructs, and dealing with inconsistently translated fragments.
Maint~i n~ ng Dual L~nguag~ IntQrfaces Unfortunately, converting a product to C++ does not immediately obviate the need to use the pTAL language. Products that export a pTAL-coded interface must continue to export that interface in pTAh as well as C++ until no pTAL clients remain.
Because of the size and complexity of externalized interfaces, and because of the opportunity for error inherent in manually maintaining parallel interfaces in both languages, it is essential that a product maintainer update just one copy of an interface and use a tool to generate the interface in the other language. The Rosetta Translator is that tool. The product maintainer uses the pTAL version as the reference copy.
This problem implies that the Rosetta Translator must emit code that is ready-to-roll. If any adjustment or enhancement to an interface file is necessary, then the adjustment or enhancement must be made to ~he pTAL code so the generated C++
can be read and used as is. Because pTAL does not support many useful C++ features, we supply a way to express new C~+-only constructs in the pTAL source. The pTAL compiler ignores such constructs.
Ob~ects in shared interfaces are only available to clients that have converted to C++ because the reference language, pTAL, cannot express them. Thus, a language conversion does not im~e~; Ately clear the way for a paradigm shift ~isible in externalized interfaces.

W O 96/11437 PCTrUS95/13260 0 Translating Langu~gu Const ~ cts Translation of each syntactic construct carries out a lexical mapping from source language lexical elements to target language lexical elements. The choice of target language construct and mapping of tokens from a source language construct to a target language construct is a matter of m~;m; zing semantic correspondence between the lexical elements.
A few simple principles guide choices of construct translations in the Rosetta Translator. These principles are intended to make the resultant code appear more natural than most machine-generated code.
Generate simple resultant code to emulate simple source code, even if the transformation is not general. Generate complex resultant code only when necessary to emulate a dif fi cul t corner of the source language. For example, the pTAL
language allows structure fields to overlay any other type field arbitrarily. If the space allocating field is at least as large as the fields that overlay it, the Rosetta Translator generates a C++ union. If it is not, the Rosetta Translator generates a macro invocation that, in turn, provides access to the appropriate memory location when the field is used. The next example depicts the translation of a structure type and varia~le.
pTAL Code Generated C++ Code struct g; class _ g {
begin public int i; union {
int j ~ i; short i;
int a; short j;
int(32) b = a; };
end; ~hort a;
_redef (long, b, a);
The Rosetta Translator generates references to macros, such as the invocation expression n _redef" in this example, that encapsulate the emulation of difficult pTAL features. These macros reside in an include file, ~rosetta.hn.
Use simplifying tra~sformations when possible. For example, in C+~ the address of the beginning of an array i5 easiest to express using the name of the array, so if the W O 96/11437 PCT~US9~/13260 Rosetta Translator initially generates n &arrayname[0] n, it applies a simplifying transformation to produce "arrayname. n Atternpt to generate idioms common7y used in the target language. For example, C++ users frequently use the ~++11 idiom for incrementing variables; the Rosetta Translator generates this breed of idiom when appropriate.
When several equally plausible choices of coding style present themselves, preserve the style that the programmer used.
For example, when choosing between C++ "*~arname" and "varname~0]", the Rosetta Translator emulates the style of the pTAL code.
8trategies for Coping With Poorly ~atche~ L~nguago Con~truct~
Several strategies are possible to cope with translating one source language to another when the language constructs do not map precisely to one another at the token level. The Rosetta Translator uses the following strategies, each where it is most useful:
--Restrict the source language to constructs that can be translated directly into the target language. This approach simplifies construct translation. The main disadvantage is that the user must either avoid using the untranslated constructs or correct the generated code. This is only practical when an equivalent supported construct exists, or when virtually none of the legacy source code actually contains the forbidden construct. We chose to restrict certain source language constructs that cannot be expressed in the target language, such as taking the address of a label. We also chose to restrict certain source language features whose translation would especially lack readability, such as pTAL expressions that depend on order of evaluation. The restricted constructs can be recoded easily. The Rosetta Toolset offers semiautomatic help in removing dependencies on these features.
--Extend the source program to give extra information that leads to a ~etter translation. With a small amount of information from a programmer familiar with the source code, the Rosetta Translator can create much more natural looking generated code.

-WO96/11437 PCT~S95/13260 0 The main disadvantage is that the user must modify his or her source code before translating.
--The Rosetta Toolset offers tools to make that activity less tedious and less prone to error, and to make optimal use of a programmer's time. The Translator detects when a specific ~ind of information would be helpful and issues an appropriate diagnostic. A programmer familiar with the source code responds to the Rosetta Translator's request for semantic hints by embedding a special comment in the source code from which the Rosetta Translator can glean the extra information it needs.
Supplying hints this way does not distur~ the behavior of the source code and does not require ext~nA;ng the source language's compiler. The information is also retained in the original source code for future retranslations. The problem analysis tools simplify the tasks performed by the human expert by inserting the desired special ~o~m~nt in exactly the right place.
For example, the pTAL language has no declarations of unsigned variables, but instead offers unsigned operators that carry out unsigned arithmetic. A user can mark a pTAL variable as unsigned ~y inserting a particular comment into the variable declaration. The Rosetta Translator translates it as a true unsigned variable, obviating the need for type casts to force unsigned operations. The next example depicts the translation 25 of an unsigned operation.
pTAL. Code~e~erated C++ Code int steve,short steve;
~teve '+~ 1234;i - (unsigned short)steve + 1234;
!unsigned~ int dano,unsigned short dano;
i :~ dano ~+' 1234;i - dano + 1234;
Hide the complexity of the translated code by emitting a meta-language (via macros) to implement ~ery complex corners of the language. This is a portable way of extending the target language. This approach can preserve readability even when a simple construct from the source language translates to a fairly complex construct in the target language. The main disadvantage is that the software maintainer must learn how to l~se the WO96/11437 PCT~S95/13260 special macros. For example, bit extraction and deposit operations on integers are supplied as macros in an include file, rosetta.h. The next example depicts the translation of bit extraction and bit deposit.
S pTAL Code(~ erated C++ Code x ~ 4:5~;x . _bit_extract~i, 4, 5);
i.<0:3~ :~ x;_bit_depo~it(i, 0, 3, x);
Extend t~e target language to handle hardware dependencies that cannot be e~pressed in the st;~n~rd target language. The target language was extended to support some special functional requirements of Tandem systems software. The target language was not extended merely to a~om~date automatic translation.
The Rosetta Translator translates systems progrAmming constructs lS from the source language to target language extensions. For example, pragmas were added to produce similar data layout between C++ and pTAL.
~n~l t ng t`~mmant8 Source-to-source translators can use one of several strategies for translating ~m~?nts that appear interspersed in the source language program:
--Gat~er and t~anslate Comm~nts separately from the source code, then collate the translated comm~nts with the translated program text into the output stream 5. This strategy simplifiea translation by separating the issue of comment association from the problem of language construct translation. One disad~antage is that comments describing code that was reordered are difficult to place sensibly.
--Associate cnmm~nts with the syntactic construct in which they are emb~ 1 This strategy reorders ~om~nts and code together, and the comment placement issues do not clutter language construct translations. One disadvantage is that very fine-grain associations might not be preserved. Text-based preprocessors, such as document generators, that use special comments as cues might accept a new lan~uage as input more easily if finer-grain associations were preserved.
--Associate comm~nts with a nearby token. The Rosetta Translator's strategy is to bind comments closely to a token and CA 0220l882 l997-04-08 W 096/11437 PCT~US9~tl3260 0 to translate the comments associated with each token when the token is mapped to the target language. One disadvantage is that when source language tokens do not correspond one to one with target language tokens, the translations of syntactic constructs must explicitly specify comment mapping. This strategy trades of~ complexity for control, and increased control is desirable to preserve appropriate comment placement.

The Rosetta Translator is able to preserve end-of-line comments in their original form, which is especially useful when the information contained therein is tabular or pictorial in nature. All comments remain associated with the tokens they describe, even if part of the enclosing construct must be moved.
When a source token is mapped to a target token, comments tied to the former are automatically translated and attached to the latter. Routines implementing language construct translations must account for the comments associated with each source language token. If a source language token maps to no target language token, then its cor~nts must be moved to an appropriate token that will be mapped.
The appearance of the generated code can benefit from construct translation routines that create comment associations that might differ from token associations. For example some construct translations appear more natural if the cormPnts that appear before the first source language token in a particular construct also appear before the first tarqet language token in the translated construct, even if the first source token does not map to the first target token.
Comments scanned in fragments that contain no tokens are associated with the static fragment, translated separately from the tree translations, and appear in the analogous C++ static fragment.
~ragment Cons~stency Because the Rosetta Translator's design requires that each source language macro maps to exactly one target language macro, it checks that every macro use is textually identical, to ensure that the macro text "works" in all contexts of use. It also checks included files. The scope of a macro actual parameter is limited to the macro body itself, so the Rosetta Translator need only check that all expansions of a formal parameter within a given macro ~rAnsion are tex~ually identical.
This and the next few sections discuss diagnosing fragment textual m;~m~tches, strategies for minimizing mismatches, and remedies for m;~tches that cannot be prevented.
Fr~gmQnts Co~t~ ni ng Pun8 When the syntax for expressing two different operations is different in the target language but the same in the source language, a fragment translation might not "wor~" in every context of use. In progr~mm;~g languages as in natural language, puns tend to translate poorly. When type compatibility rules differ between the two languages, for example, type casts can lead to a ~i~m~tch in macros that are used in conjunction with multiple types.
The following example illustrates an error condition: the translation of two ~r~ions of the same macro result in different macro body text. All instances of a macro ~ody must be checked for consistency before output source generation. The Rosetta Translator generates re_erences to macros, such as ~_redef n and n _tobitfield" in this example, that encapsulate the emulation o~ difficult pTAL features. In this case, the integer field "a" overlays the bit field "k". The "_tobitfield" macro takes the name of the surrounding class type as a parameter.
That name is dif_erent for the two uses of the macro: ~f_ n versus ~g ". The next example depicts a macro whose two uses result in macro ~ody text that does not match; mismatching tokens are in ~old.
pTAL ~acro Generated C++ Macro ~zero-length array causes a to overlay kl define fields - int a [0:-1]; #define fields ~ERROR~
un~igned~1) k #;
pTAL Code Generated C++ Code ~truct f; class f_ {
begin fields;
fieldq; } f;
end;

-WO96/11437 PCT~S9S/13260 0 struct g; class g_ {
begin fields;
field~; } g;
end;
pTAI, Code Exparlsion C++ Code ~:xpansior ~truct f; class f_ {
begin _redef(short, a, _tobitfield(f_, k));
int a [0:-1]; un~igned k:1;
un~igned(1) k; ~ f;
end;
struct g; class g_ {
begin _redef(short, a, _tobitfield(g_, k));
int a [O:-l]; un~igned k:1;
unsigned(1) k; ~ g;
end;
Blurred FragmQnt Boundaries If a construct translation blurs fragment boundaries, the translated fragment tree looks as if two instances of the _ragment were created, and the contents of those fragments are not textually identical. Fragment boundary violations are detected by the same fragment consistency check that detects puns that do not work. The following example illustrates a blurred fragment boundary in a use of an ill-formed macro.
I+- precedence rules differ between the two languages, then introducing parentheses to preserve the original expression's semantics can lead to a consistency violation. The pTAL
language assigns higher precedence to the left shi+t operator than to the addition operator. The C+~ language assigns higher precedence to the addition operator than to the left shi t operator. Parentheses appear in the target lanquage version of the expression to retain the order of evaluation. The parentheses tokens happen to be generated from the higher precedence operator, which determines their fragment assignment.

pTAL Code Expa~slon of pTAL Code Ge~erated C++ Code define fred(p~ ~ p << 2#;
. . .
a :~ ~red (x + 1) ; a :- x + 1 << 2 ; a ~ x + ( 1 << 2 ) ;

The translation of the previous example results in a C++
~ragment tree that appears to have two instances of the actual parameter fragment which do not contain identical text.

096/11437 25 PCTrUS9~/13260 8tr~t~gi~s for Y~; mi zing InconsistQntly Tr~nslat~d ~croQ ~ ~cro Actu~l P~r~m~ters Inconsistently translated macro bodies and macro actual parameters can be mini~i zed by building special rules into the token mapping methods, when possible.
Suppress simplifying transfo~nations on the generated code when inside of a macro expansion. Generally, syntax transformations are not sensitive to whether or not the generated code is inside of a macro body expansion or macro formal parameter expansion. Some routines that perform construct translations are sensitive to fragment boundaries, though they do not translate fra~ments. These routines suppress simplifying transformations within a macro ~Yr~n~ion or macro formal parameter expansion in order to make translations for different macro invocations textually identical. For example, the transformation simplifying the text "&arrayvar[o] n to "arrayvar" is suppressed within a macro when the index is in a macro parameter. Likewise, the transformation simplifying the text n (*ptr).field" to "p->field" is suppressed when n*p~l is a macro actual parameter.
Absorb tokens ~at di ffer between invocations of a macro into an actual parame~er, when possible. Source language tokens might generate not only the logical translation of themselves, but also some attendant baggage. If that baggage can differ in two different invocations of the same macro, there is no problem if the baggage is mapped to the actual parameter rather than the macro body. For example, C++ parentheses generated to force the appropriate operator precedence should be absorbed into the translated actual parameter if they surround C++ tokens generated only from pTAL actual parameter tokens.
Likewise, if a Cl~ ~*~ operator was generated in the translation of an implicitly dereferenced pointer in a macro actual parameter, the ~*~ operator should be absorbed into the translated actual parameter so that nonpointer variables can be passed into the macro.

W 096/11437 PCT/US95/13260 ~ -Extrude tokens t~at differ between invocations of a macro lnto the ~ragment cont~ining the macro invocation syntax.
Similar to the strategy previously described, if the differing tokens are on the edge of the C++ translation of a macro body, there is no problem if they are mapped outside. For example, if a C++ type cast applies to an entire macro expansion, then the generated text should show the type cast applied to the macro invocation syntax.
~em~dies For ~ncon~istently Tr~slatQd Macros ~nd Macro Actu~l P~rameters Although a careful design of token associations, taking into account common coding practices in the source language, can ensure that fragment mismatch problems are a rare event, it cannot prevent all possible problems. When an inconsistently translated macro or macro actual parameter occurs, the user can choose from three remedies. Each of the remedies can solve any fragment m; ~tch problem. If the macro in question is exported, changing the macro interface in the original language implementation is usually not possible.
Recode the macro body so that the translation ~wor~.
This strategy is viable if recoding the body of the macro avoids the problematic construct and does not disturb the interface or the usage of the macro. If recoding the macro requires changes at all points of use, then this might not be a viable solution.
Give the Rosetta Translator permission to expand all uses of the macro. This strategy eliminates any fragment mismatch problem at the cost of deimplementing the macro that contained a problem. It is not a desirable solution if the macro was useful for improving the usability, main~AinAhility, or readability of the software, or if the macro was exported. It is a viable solution if the product maintainer is not interested in preserving the macro. The next example illustrates the expansion of a macro.
pTAL Code Gener~ted C++ Code define lexpand! ~hifty (x, y) ~ x ~< y~;
a :~ shifty(2, 3); a - 2 << 3;

Give the Rosetta Translator permission to create an additional macro parameter to encapsulate the differences. This strategy eliminates any fragment mismatch problem. It has the potential of producing actual parameters in the target language code that are less sensible than a programmer would write.
Recalling the example of a mismatching macro body from the previous section, the differences can be encapsulated. The next example illustrates this encapsulation.
pTAL Macro Generated C++ Macro !zero-lenght array causes a to overlay k !
define fields !FORMAL:stype! =
int a [0:-1]; #define fields (stype) \
unsigned (1) k #; r e d e f ( s h o r t , a , _tobitfield(stype, k)); \
unsigned k: 1 pTAL Code GeneratedC++ Code struct f;
begin class f_ ( Field; Fields (f_);
end; } f;
struct g; class g_ ( begin fields (g_);
fields; } g;
end;

References 1. Albrecht, P. F., Garrison, P. E., Graham, S. L., Hyerle, R. H., Ip, P., Krieg-Bruckner, B., "Source-to-Source Translation: Ada to Pascal and Pascal to Ada", Proceedings of the ACM SIGPLAN Symposium on the Ada Programming Language, November, 1980.

2. Atkinson, R., Demers, A., Hauser, C., Jacobi, C., Kessler, P., and Weiser, M., "Experiences Creating a Portable Cedar", Proceedings of the SIGPLAN '89 Conference on Programming Language Design and Implementation, June, 1989.

3. Bartlett, J., SCHEME->C, a Portable Scheme-to-C
Compiler. Technical Report, Digital Western Research Laboratory, 1989.

4. Lemkin, P., "PSAILA: SAIL to C", Computer Language, 2(8), August, 1985.

5. Moynihan, V. D., and Wallis, P. J. L., "The design and Implementation of a High-Level Language Converter", Software--Practice and Experience, 21(4), April, 1991.

W O96/11437 PCTrUS95/13260 ~ -

6. Stroustrup, B., The C++ Prograrr~ning Language, Addison--Wesley, 1991.

7. Tandem computers, Inc., Tran~action Application Language (TAL) Reference Manual, Version D10.

8. Waters, R. C., "Program Translation via Abstraction and Reimplementation", IEEE Transactions on So~tware Engineering, 14(8), Auyust, 1988.

9. Weiner, J., Ramakr;~:hn~n, S., "A Piggy-bac~c Compiler For Prolog", Proceedings of the SIGPLAN '88 Conference on progr~mming Language Design and Implementation, June, 1988.
lS
DESCRIPTION OF THE PREFERRED EMBODIMENT
A prior art tree data structure is outlined in FIG. 1. The purpose of this data structure is to relate a set of "things, n or objects in a hierarchy. One object in this set, called a root node, is at the vertex. From the root, links, called branches, connect to other objects, called nodes. In turn, the nodes may branch to other nodes ultimately terminating in a set of objects called leaves. Viewing the tree from the root to the leaves, tracing a single path o~ branches, nodes are ancestors of each other. Conversely, viewing the tree from a leaf to the root, again tracing a single path, nodes are descendants of each other. Choosing an ar~itrary node, it is considered the child of the node to which it branches toward the root and the parent of the node branching toward the leaves.
FIG. 2 illustrates a contribution of the present invention to the prior art. It is termed a dou~ly-rooted tree data structure. This doubly-rooted tree has two roots 511 and 517, but only a single set of leaves 515. From each root emanates a 3S tree, as in the prior art, but the nodes 513 and 519, branches 521 and 523, and hierarchical arrangement of the two trees may be different as long as the two trees co-terminate at a sinyle set of leaves 5'5. In the present invention, the leaves of a doubly-rooted tree are considered tokens and are collectively referred to as the "virtual source. n Fig. 2A is a simplified block diagram of a computer system, designated generally with the reference numeral 10A, incorporating the data structure of Fig. 2. As Fig. 2A
illustrates, the computing system 10A includes a central processing unit (CPU) 11A that is coupled to a memory 14A and an input/output channel (IOC) 16A by a bus structure 18A. The IOC
16A, in turn, connects the CPU 11A to a data storage area 22A by and input/output (I/O) 20A. The data storage area 22A may take the form of a magnetic disk media storage device, an electronic memory, or a communications link to other storage areas.
While a doubly-rooted tree is two dimensional, the concept can be expanded to three, four, or more dimensions (not illustrated. Multiple trees may co-terminate at a single set of leaves to form an N-rooted tree.
Prior art trees are useful in computer language compilers such the prior art compiler, source code 51 flows first through a scanner 53, where a stream of characters is divided into recognizable "words" or tokens and then to a parser 55, which determines a hierarchical relationship among the tokens and builds a syntax tree 57. An additional step, performed by a semantic analyzer 59, further refines the syntax tree 57. A
syntax translator 61 converts the syntax tree 57 into an intermediate code tree 63. In turn a code generator 65 converts the intermediate code tree 63 into object code 67. Also commonly called machine code, object code 67 is an instruction sequence which a computer can directly execute and is usually stored on magnetic media of in electronic memory. In the normal course of usage, machine code is not stored in a manner which humans can read and interpret. For background information on the construction of compilers, see Aho, Sethi, and Ullman, Compilers: Principles, Techniques, and Tools, Addision-Wesley, 1985.
As illustrated in Fig. 4, the source code to source code translator of the present invention, in simplified form, generally follows the flow of the prior art compiler of Fig 3.

WO96/11437 PCT~S95/13260 However, instead of employing simple tree data structures, the translator uses two doubly-rooted trees to preserve contextual information such as macros, conditionally compiled regions of code, source inclusion, comments, and other preprocessor characteristics.
FIG. 5 is a flowchart that describes a method for building of the first tree in doubly rooted tree, joined at the leaves.
This method is applicable to building the source fragment tree l9 of FIG. 4. The method builds the tree from the top down.
The flowchart uses the following data and operations:
Source is a set of ordered set of objects employed in step 603, one of which is identified as primary.
Ordered set of objects, operations:
Open 603 Test for whether or not the set is exhausted 605 Get the next object 607 Object is a leaf or an ordered set Operations:
Test for whether the object is a leaf or an ordered set Attach to parent node Node operations:
Accept ordered attachment of children Get parent node FIG. 6 is a flowchart that describes a method for building a second tree in the doubly rooted tree, joined at the leaves.
This method is applicable to building the source syntax tree 7 of FIG. ~. The ordered set of leaves in step 621 that is input to the parser 623 is yielded by an inorder traversal of the first tree. The method builds the tree from the bottom up.
FIG. 7 is a flowchart that describes the translation of the second tree in the doubly rooted tree, joined at the leaves.
FIG. 8 is a flowchart that describes the translation of the first tree in the doubly rooted tree, joined at the leaves. The flowchart uses the following data and operations:
Node operations:

0 96/11437 PCT~US95/13260 Translate Attach to parent node Test for whether the node is "open": The "open" nodes in the tree include the ancestors of the last leaf that was added to the tree. Adding a child to a node "closes~
nodes that are no longer ancestors of the latest leaf added.
Leaf operations:0 Attach to parent node Input set operations:
Test for whether the set is exhausted Another way of describing FIG. 8 is that it shows a method of translating a fragment tree from the source doubly-rooted tree to the target doubly-rooted tree. This is a critical stage in the translation process; it is the point at which preprocessor characteristics are transferred from the source code to the target code. After building the target language syntax tree, the translator traverses the target language virtual source token stream, and builds the target language fragment tree, based on the position in the source language fragment tree of the source language token associated with each target language token encountered in the virtual source stream.
Operations are as follows:
Attach(x, y): Attach x to y: y becomes the parent of x. x is added positionally after pre-existing siblings.
Translate(n): Translate n (token or fragment) to the target language version of the object, and return the translation.
Translation(n): Return the latest translation of the frasment n;
nil if it has not been translated.
Parent~n): If n is a token, return the fragment in which n resides. If n is a fragment, return the invocation syntax of the fragment (macro name token, for example).
IsOpen(n): Return true if the fragment n is open in the target fragment tree. The "open~ nodes in the target language fragment tree include the ancestors of the last token that was added to W O 96/11437 PCTrUS95/13260 the fragment tree. Adding a child to a fraqment "closes" nodes that are no longer ancestors of the latest token added.
Begin by translating the main source include fragment. This fragment is initially open.
FOR (TargetToken = each token in the target language virtua source) BEGIN
SourceToken = the source token associated with the current TargetToken SourceFrag = Parent (SourceToken) IF (IsOpen (Translation (SourceFrag))) THEN BEGIN
/* add a new token to an existing fragment that is already open */
TargetFrag = Translation (SourceFrag) Attach (TargetToken, TargetFrag) END ELSE BEGIN
/* create a new fragment, and add the new token to it */
TargetFrag = Translate (SourceFrag) Attach (TargetToken, TargetFrag) SourceInvSyntax = Parent (SourceFrag) TargetInvSyntax = Translate (SourceInvSyntax) Attach (TargetFrag, TargetInvSyntax) SourceFrag = Parent (SourceInvSyntax) /* Create new fragments and invocation syntax until TargetToken is connected to the tree. */
WHIrF (NOT IsOpen (Translation (SourceFrag)) ) BEGIN
TargetFrag = Translate (SourceFrag) Attach (TargetInvSyntax, TargetFrag) SourceInvSyntax = Parent (SourceFrag) TargetInvSyntax = Translate (SourceInvSyntax) Attach (TargetFrag, TargetInvSyntax) SourceFrag = Parent (SourceInvSyntax) END ~HILE
TargetFrag = Translation (SourceFrag) Attach (TargetInvSyntax, TargetFrag);
END ELSE
END FOR

Taken together, FIGS. 9, 10, and 11 show an example of how the method of FIGS. 5, 6, 7, and 8 operates on a specific two line macro:

define less(x) = a < x#;
while less( 4 2 ) do The macro was written for a compiler of a Tandem proprietary high level language, pTAL. The target source code is a well ~VO 96/11437 PCT/US95/13260 known standardized language, C++. Correctly translated to C++, the macro reads:
. . .
,ydefine less(x) < x while (a less(42)) . . .
FIG. 9 depicts an example fragment tree and syntax tree, using the pTAL language as the source language. This diagram represents an explosion of the pTAL syntax tree, pTAL fragment lo tree, and the pTAL tokens that connect them depicted in a more general way in FIG. ~: the source fragment tree 19, the source syntax tree 7, and leaves 9 that join them. The pTAL fragment tree and syntax tree is a doubly rooted tree jointed at the tokens. The tokens m~k; ng up the body of the define "less" are represented in their context of use.
Fragment lZ1 represents the primary input file. Fragment 127 represents the invocation of the macro "less". Tokens 153, 155, 157, 159, 161, 163, 165, 167, 169, 129, 131, 135, and 177 are in fragment 127. The macro's invocation syntax 129, 131, 135 is associated with the f~ragment 127. The tokens 171, 173, and 1~1, which compose the macro body, are in frA~^~t 127. Fragment 137 represents the substitution for macro formal parameter "x" of~
the actual parameter text, which is the token 175. The macro formal parameter's invocation syntax 1~.1 is associated with the fragment 137. The token 175, which is the macro actual parameter, is in fragment 137.
Node 17g represents a pTAL procedure. That procedure is composed of a macro declaration represented by node 1~7 and a while statement represented by node 1~9. The macro declaration 1~7 contains the tokens 153, lS5, 157, 159, 161, 163, 165, and 167. The body of the macro is expanded in its context of use.
The while statement 1~9 i8 composed of the token 169, an expression represented by node 151, and the token 177. The expression 151 i8 composed of the tokens 171, 173, and 175.
- 35 FIG. 10 depicts a pTAL fragment subtree and syntax subtree, including the tokens that link them. It also depicts the C++
syntax tree that represents the translation of the pTAL syntax W O96/11437 PCTrUS95/13260 tree. The syntax translator has translated each pTAL syntactic construct to a semantically equivalent C~+ syntactic construct In addition, it maps pTAL tokens to-C++ tokens. The pTAL
fragment tree has not yet been translated to its C++
counterpart. The source language while statement 1~9 has been translated to the target language while statement 239. The source language expression 151 has been translated to the target language expression 2~1. Each C++ token in the C++ syntax tree 243, 245, 2~7, 2~9, 251, 253 was "generated" by exactly one pTAL
token. C++ token 2~3 was generated by pTAL token 169, C++ token 245 was generated by pTAL token 171, and so on as depicted by dashed arrows. The association between a C++ token and a pTAL
token, depicted as .l:~Ch~l arrows, determines in which virtual fragment the token belongs; these associations make it possible to build a target language fragment tree. One pTAL token may generate several C++ tokens: 169 generates 2~3, 2~5, and 253.
One pTAL token may generate no C++ tokens: token 177 generates no C+~ tokens. Each C++ token is qenerated by exactly one pTAL
token.
FIG. 11 depicts a pTAL fragment tree and syntax tree, including the tokens that link them. It also depicts a semantically equivalent C++ fragment tree and syntax tree, including the tokens that link them. It depicts FIG. 11 after the translation of the fragment tree. In order to perform the translation, traverse the leaves of the C++ syntax tree; these leaves will become the leaves of the C++ ~ragment tree.
Following is a walkthrough of building the example C++
fragment su~tree that is equivalent to the pTAL fragment subtree depicted in FIG. 0 and FIG. 11. A fragment in the C++ fragment tree is "open" if it is an ancestor of the leaf that was most recently ~ to the C++ fragment tree.
Translate the root of the pTAL frayment tree: the pTAL
fragment 121 is translated to the C++ fragment 273. The C~+
fragment tree root node 273 i6 initially "open".
Consider the C++ token 2~3.
This token is associated with the pTAL token 169.

-096/11437 PCTrUS95/13260 The pTAL token 169 is contained in pTAL fragment 121, which translated to C++ fragment 2~3.
C++ ~ragment 273 is "open".
Add C++ token 2~3 to C++ fragment 273.
5 Consider the C~+ token 2~5.
This token is associated with the pTAL token 169.
The pTAL token 169 is contained in pTAL fragment 121.
pTAL fragment 121 translated to C++ fragment 273.
C++ fragment 273 is "open".
Add C++ token 2~.5 to C++ fragment Z73.
Consider the C++ token 2~7.
This token is associated with the pTAL token 171.
The pTAL token 171 is con~in~l in pTAL fragment 127.
The pTAL fragment 127 has not been translated, so does not 15 appear in the C++ fragment tree.
Translate the pTAL fragment 127 to create the C++ fragment 265.
Add the C++ token 2~7 to t:he C++ fragment 265.
The parent of the pTAL fragment 127 is the pTAL fragment 121.
pTAL fragment 121 translated to the C++ fragment 273.
C++ fragment 273 is "open" in the C++ fragment tree.
Add the C++ fragment 26S to the C++ fragment 273.
Consider the C++ token 2~9.
This token is associated with the pTAL token 173.
The pTAL token 173 is conltained in the pTAL fragment 127.
The pTAL fragment ~27 translated to the C++ fragment 265.
The C++ fragment 26S is "open H in the C++ fragment tree.
Add the C++ token 2~9 to the C++ fragment 265.
Consider the C++ token 251.
This token is associated with the pTAL token 175.
The pTAL token 175 is contained in the pTAL fra~ment 137.
The pTAL fragment 137 has not been translated, so does not appear in the C++ fragment tree.
Translate the pTAL fragment 137 to create the C++ fragment 35 Z61.
Add the C++ token 251 to the C++ fragment 261.

W O 96/11437 PCT~US95/13260 The parent of the pTAL fragment 137 is the pTAL fragment 127.
The pTAL fragment 127 was translated to the C++ fragment 265.
The C++ fragment 265 is "open" in the C++ fragment tree.
Add the C++ fragment 261 to the C++ fragment 265.
Consider the C++ token 253.
This token is associated with the pTAL token 169.
The pTAL token 169 is contained in pTAL fragment 121, which translated to C++ fragment 273.
C++ fragment 273 is "open".
Add C++ token 253 to C++ fragment 273.
Notice that the C++ fragments 261 and 265 are no longer "open": they are no longer ancestors of the last C++ token to be added to the C++ fragment tree.

Claims (11)

WHAT IS CLAIMED IS:

1. A memory for storing data for access by an application program being executed on a data processing system, comprising:
a doubly-rooted tree data structure stored in the memory, the data structure including:
a set of leaves;
a first hierarchically related set of objects, with one or more parents, children, ancestors, and descendants, which may be represented by a tree data structure with a root, a set of branches, and a set of nodes, terminating at the set of leaves;
a second hierarchically related set of objects which terminates at the set of leaves.

2. The memory of claim 1 wherein the set of leaves comprises a virtual source including a stream of tokens created from the set of objects.

3. The memory of claim 2 wherein the tokens are created by a scanner means.

4. A computer system incorporating the memory of claim 1.

5. The memory of claim 1 further comprising an N-rooted tree structure stored in the memory including one or more additional hierarchically related sets of objects terminating at the set of leaves.

6. A method for use in a computer system for creating a doubly-rooted tree data structure and storing information in the data structure comprising the computer implemented steps of:
retrieving a set of objects from a memory;

identifying a first set of hierarchical relationships among the objects, yielding syntactic constructs;
scanning the syntactic constructs into virtual source;
storing the syntactic constructs in a first tree by top down means in depth-first traversal order in accordance with the first set of hierarchical relationships so that the first tree terminates at the virtual source;
identifying a second set of hierarchical relationships among the objects;
storing the objects in a second tree by bottom up means in depth-first traversal order in accordance with the second set of hierarchical relationships so that the second tree terminates at the virtual source.

7. The method of claim 6 wherein the method of identifying the first set of hierarchical relationships among the objects comprises:
a virtual source production mechanisms including:
(a) scanning the objects into tokens:
(b) recognizing a set of tokens which represent an invocation expression for text substitution;
(c) associating the expression with one or more nodes;
(d) performing text substitution as represented by the invocation expression;
(e) repeating steps (b), (c), and (d) until the tokens are exhausted, yielding the syntactic constructs.

8. A method for use in a computer system for translating a source doubly-rooted tree data structure including a first source tree and a second source tree to a target doubly-rooted tree data structure including a first target tree and a second target tree comprising the computer-implemented steps of:
creating the source data structure coupled with storing information into the source data structure;
translating the second source tree coupled with traversing its nodes coupled with means for creating the second target tree;

translating the first source tree coupled with traversing its nodes coupled with means for creating the first target tree;

9. A method for use in a computer system for translating a first source code of a source computer language into a second source code of a target computer language comprising:
means for creating a source doubly-rooted tree data structure including:
means for retrieving the first source code;
means for virtual source production from the first source code to create a source fragment tree;
scanning the first source code to form a virtual source;
identifying the virtual source as leaves of the source doubly-rooted tree:
identifying leaves of the source fragment tree with the leaves of the source doubly-rooted tree;
means for parsing the virtual source into a source syntax tree such that the leaves of the source syntax tree are identified with the leaves of the source doubly-rooted tree:
means for obtaining and storing semantic information completing a description of the source code in the source syntax tree;
means for creating a target doubly-rooted tree data structure including:
means for translating the source syntax tree coupled with means for creating a target syntax tree;
identifying leaves of the target syntax tree with leaves of the target doubly-rooted tree;
associating leaves of the target doubly-rooted tree with corresponding leaves of the source doubly-rooted tree;
means for translating the source fragment tree coupled with means for creating a target fragment tree so that leaves of the target fragment tree are ordered;

identifying leaves of the target fragment tree with leaves of the target doubly-rooted tree;
means for generating the second source code from the target doubly-rooted tree;
means for storing the second source code.

10. The method of claim 9 wherein the means for translating the source fragment tree coupled with means for creating a target fragment tree comprises:
locating the root of the source fragment tree:
translating the source root to the root of the target fragment tree;
examining each leaf of the target syntax tree;
means for building the target fragment tree based on the position of leaves in the source fragment tree including the steps of;
locating a target fragment tree leaf;
examining a source fragment tree leaf associated with the target fragment tree leaf;
examining a parent node of the source fragment tree leaf;
testing whether the most recent translation of the parent node is an ancestor of the most recently attached target fragment tree leaf or is the root of the target fragment tree;
attaching the target fragment tree leaf to the parent node of the most recently attached leaf if the test result is TRUE;
attaching the target fragment tree leaf to the parent node;
examining a next target fragment tree leaf;
translating the parent node if the test result is FALSE;
adding the parent node to the target fragment tree;

attaching the target fragment tree leaf to the parent node:
assigning the parent node to a first temporary object;
assigning a parent of the parent node to a second temporary object;
testing whether the second temporary object is an ancestor of the most recently attached target fragment tree leaf or is the root of the target fragment tree;
adding the second temporary object to the target fragment tree if the test result is "TRUE";
attaching the first temporary object to the ancestor node:
examining a next target fragment tree leaf;
translating the second temporary object if the test result is "FALSE";
attaching the second temporary object to the fragment tree:
attaching the first temporary object to the second temporary object;
assigning the second temporary object to the first temporary object:
repeat testing the second temporary object.

11. An apparatus for translating a first source code of a source computer language into a second source code of a target computer language while preserving text preprocessor statements and expressions comprising:
source computer language file means with memory allocated for storing the first source code;
target computer language file means with memory allocated for storing the second source code;
means for creating a first and second doubly-rooted tree data structure with memory allocated for information to be stored within the first and second data structures;
means for retrieving the first source code from memory;

means for scanning and parsing the first source code into a first virtual source;
means coupled to the scanning and parsing means for storing the first virtual source code while scanning and parsing in the first doubly-rooted tree data structure so that text preprocessor statements and expressions are preserved;
means for translating the first doubly-rooted tree structure into a second virtual source code;
means coupled to the translating means for storing the second virtual source code in the second doubly-rooted tree data structure so that text preprocessor statements and expressions are preserved;
means for traversing the second data structure coupled to means for creating the second source code;
means for storing the second source code coupled to the target computer language file means.