US20070271555A1

US20070271555A1 - Combined verification and compilation of bytecode

Info

Publication number: US20070271555A1
Application number: US11/760,073
Authority: US
Inventors: Beat Heeb
Original assignee: ESMERTEC AG
Current assignee: Myriad Group AG
Priority date: 2000-12-13
Filing date: 2007-06-08
Publication date: 2007-11-22
Also published as: US20050198079A1; US6964039B2; WO2002048821A2; US20060294528A1; US7263693B2; US20050204361A1; US20050091650A1; WO2002048821A3; US20050186625A1; US20020138825A1

Abstract

The present invention is a new method and apparatus to perform combined compilation and verification of platform independent bytecode instruction listings into optimized machine code. More specifically, the present invention creates a new method and apparatus in which bytecode compilation instructions are combined with bytecode verification instructions, producing optimized machine code on the target system in fewer programming steps than traditionally known. The new method, by combining the steps required for traditional bytecode verification and compilation, increases speed and applicability of platform independent bytecode instructions.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional application Ser. No. 10/991,444, filed Nov. 17, 2004, which application is a continuation of U.S. Non-Provisional application Ser. No. 10/016,794, filed on Oct. 29, 2001, now U.S. Pat. No. 6,964,039, which claims benefit of U.S. Provisional Application No. 60/255,096, filed Dec. 13, 2000, all of which are incorporated herein by reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

Not applicable.

BACKGROUND OF INVENTION

1. Field of the Invention
The present invention is related to the combined compilation and verification of platform neutral bytecode computer instructions, such as JAVA. More specifically, the present invention relates to a new method of creating optimized machine code from platform neutral bytecode on either the development or target system by concurrently performing bytecode verification and compilation.
2. Description of Related Art
The benefit of architecture neutral language such as JAVA is the ability to execute such language on a wide range of systems once a suitable implementation technique, such as a JAVA Virtual Machine, is present. The key feature of the JAVA language is the creation and use of platform neutral bytecode instructions, which create the ability to run JAVA programs, such as applets, applications or servelets, on a broad range of diverse platforms. Typically, a JAVA program is compiled through the use of a JAVA Virtual Machine (JVM) which is merely an abstract computing machine used to compile the JAVA program (or source code) into platform neutral JAVA bytecode instructions, which are then placed into class files. The JAVA bytecode instructions in turn, serve as JVM instructions wherever the JVM is located. As bytecode instructions, the JAVA program may now be transferred to and executed by any system with a compatible JAVA platform. In addition, any other language which may be expressed in bytecode instructions, may be used with the JVM.
Broadly speaking, computer instructions often are incompatible with other computer platforms. Attempts to improve compatibility include “high level” language software which is not executable without compilation into a machine specific code. As taught by U.S. Pat. No. 5,590,331, issued Dec. 31, 1996 to Lewis et al., several methods of compilation exist for this purpose. For instance, a pre-execution compilation approach may be used to convert “high level” language into machine specific code prior to execution. On the other hand, a runtime compilation approach may be used to convert instructions and immediately send the machine specific code to the processor for execution. A JAVA program requires a compilation step to create bytecode instructions, which are placed into class files. A class file contains streams of 8-bit bytes either alone or combined into larger values, which contain information about interfaces, fields or methods, the constant pool and the magic constant. Placed into class files, bytecode is an intermediate code, which is independent of the platform on which it is later executed. A single line of bytecode contains a one-byte opcode and either zero or additional bytes of operand information. Bytecode instructions may be used to control stacks, the VM register arrays or transfers. A JAVA interpreter is then used to execute the compiled bytecode instructions on the platform.
The compilation step is accomplished with multiple passes through the bytecode instructions, where during each pass, a loop process is employed in which a method loops repeatedly through all the bytecode instructions. A single bytecode instruction is analyzed during each single loop through the program and after each loop, the next loop through the bytecode instructions analyzes the next single bytecode instruction. This is repeated until the last bytecode instruction is reached and the loop is ended.
During the first compilation pass, a method loops repeatedly through all the bytecode instructions and a single bytecode instruction is analyzed during each single loop through the program. If it is determined the bytecode instruction being analyzed is the last bytecode instruction, the loop is ended. If the bytecode instruction being analyzed is not the last bytecode instruction, the method determines stack status from the bytecode instruction and stores this in stack status storage, which is updated for each bytecode instruction. This is repeated until the last bytecode instruction is reached and the loop is ended.
During the second compilation pass, a method loops repeatedly through all the bytecode instructions once again and a single bytecode instruction is analyzed during each single loop through the program. If it is determined the bytecode instruction being analyzed is the last bytecode instruction, the loop is ended. If the bytecode instruction being analyzed is not the last bytecode instruction, the stack status storage and bytecode instruction are used to translate the bytecode instruction into machine code. This is repeated until the last bytecode instruction is translated and the loop is ended.
A JAVA program however, also requires a verification step to ensure malicious or corrupting code is not present. As with most programming languages, security concerns are addressed through verification of the source code. JAVA applications ensure security through a bytecode verification process which ensures the JAVA code is valid, does not overflow or underflow stacks, and does not improperly use registers or illegally convert data types. The verification process traditionally consists of two parts achieved in four passes. First, verification performs internal checks during the first three passes, which are concerned solely with the bytecode instructions. The first pass checks to ensure the proper format is present, such as bytecode length. The second pass checks subclasses, superclasses and the constant pool for proper format. The third pass actually verifies the bytecode instructions. The fourth pass performs runtime checks, which confirm the compatibility of the bytecode instructions.
As stated, verification is a security process, which is accomplished through several passes. The third pass in which actual verification occurs, employs a loop process similar to the compilation step in which a method loops repeatedly through all the bytecode instructions and a single bytecode instruction is analyzed during each single loop through the program. After each loop, the next loop through the bytecode instructions analyzes the next single bytecode instruction which is repeated until the last bytecode instruction is reached and the loop is ended.
During the verification pass, the method loops repeatedly through all the bytecode instructions and a single bytecode instruction is analyzed during each single loop through the program. If it is determined the bytecode instruction being analyzed is the last bytecode instruction, the loop is ended. If the bytecode instruction is not the last bytecode instruction, the position of the bytecode instruction being analyzed is determined. If the bytecode instruction is at the beginning of a piece of code that is executed contiguously (a basic block), the global stack status is read from bytecode auxiliary data and stored. After storage, it is verified that the stored global stack status is compliant with the bytecode instruction. If however, the location of the bytecode instruction being analyzed is not at the beginning of a basic block, the global stack status is not read but is verified to ensure the global stack status is compliant with the bytecode instruction. After verifying that the global stack status is compliant with the bytecode instruction, the global stack status is changed according to the bytecode instruction. This procedure is repeated during each loop until the last bytecode instruction is analyzed and the loop ended.
It may be noted that the pass through the bytecode instructions that is required for verification closely resembles the first compilation pass. Duplicate passes during execution can only contribute to the poor speed of JAVA programs, which in some cases may be up to 20 times slower than other programming languages such as C. The poor speed of JAVA programming is primarily the result of verification. In the past, attempts to improve speed have included compilation during idle times and pre-verification. In U.S. Pat. No. 5,970,249 issued Oct. 19, 1999 to Holzle et al., a method is taught in which program compilation is completed during identified computer idle times. And in U.S. Pat. No. 5,999,731 issued Dec. 7, 1999 to Yellin et al. the program is pre-verified, allowing program execution without certain determinations such as stack overflow or underflow checks or data type checks. Both are attempts to improve execution speed by manipulation of the compilation and verification steps. In order to further improve speed, a method and apparatus is needed that can combine these separate, yet similar steps, the verification pass, and the first and second compilation pass, into a step which accomplishes the multiple tasks in substantially less time.

BRIEF SUMMARY OF THE INVENTION

It is the object of the present invention to create a method and apparatus which may be used to combine compilation and verification of platform independent bytecode, either on the development system or within the target system, into optimized machine code thereby improving execution speed. Considering the required steps of bytecode compilation and verification, similarities between the two may be used to combine steps thereby reducing the time required to achieve both. The new method consists of a program instruction set which executes fewer passes through a bytecode instruction listing where complete verification and compilation is achieved, resulting in optimized machine code.
The new method loops repeatedly through all the bytecode instructions and a single bytecode instruction is analyzed during each single loop through the program. If it is determined the bytecode instruction being analyzed is the last bytecode instruction, the loop is ended. If the bytecode instruction is not the last bytecode instruction however, the position of the bytecode instruction is determined and if the bytecode instruction being analyzed is at the beginning of a piece of code that is executed contiguously (a basic block), the global stack status is read from bytecode auxiliary data and stored. After storage, it is verified that the stored global stack status is compliant with the bytecode instruction. If however, the location of the bytecode instruction being analyzed is not at the beginning of a basic block, the global stack status is not read, but is verified to ensure the global stack status is compliant with the bytecode instruction. After verifying that the global stack status is compliant with the bytecode instruction, the global stack status is changed according to the bytecode instruction being analyzed. In addition, stack status is determined from the bytecode instruction being analyzed and stored in stack status storage. In doing so, the new method achieves complete verification and partial compilation (the steps traditionally performed during separate verification and compilation in the prior art).
Next, the new method loops repeatedly through all the bytecode instructions and if it is determined the bytecode instruction being analyzed is the last bytecode instruction, the loop is ended, otherwise the pass is repeated for each bytecode listing within each class file. If the bytecode instruction is not the last bytecode instruction, the stack status storage and bytecode instruction are used to translate the bytecode instruction being analyzed into optimized machine code and this is repeated until the last bytecode instruction is translated and the loop is ended.
The new method achieves complete verification and compilation of the bytecode instructions into optimized machine code on the development or target system. Through the combined steps, compilation and verification occur simultaneously using the new method.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, features and characteristics of the present invention will become more apparent to those skilled in the art from a study of the following detailed description in conjunction with the appended claims and drawings, all of which form a part of this specification. In the drawings:
FIG. 1A (prior art) illustrates a flowchart of traditional bytecode instruction first pass compilation;
FIG. 1B (prior art) illustrates a flowchart of traditional bytecode instruction second pass compilation;
FIG. 2 (prior art) illustrates a flowchart of traditional bytecode instruction verification;
FIG. 3 illustrates a main flowchart of the embodiment of the new method;
FIG. 4A illustrates a subset flowchart of the embodiment of the new method; and
FIG. 4B further illustrates a subset flowchart of the embodiment of the new method.

DETAILED DESCRIPTION OF PRESENTLY PREFERRED EXEMPLARY EMBODIMENTS

The present invention provides an improved method and apparatus to perform platform independent bytecode compilation and verification creating optimized machine code on an independent platform. The present invention, by creating a new bytecode compilation method combined with instruction verification, increases the speed and applicability of bytecode programming.
In prior art FIGS. 1A and 1B, an illustrative flow diagram of traditional bytecode compilation is shown. In prior art FIG. 1A, a traditional compilation method is shown as flow diagram 100 which loops through the bytecode instructions, analyzing an individual bytecode instruction during each loop as stated in step 102. After each bytecode instruction is analyzed, the method determines the stack status from the bytecode instruction being analyzed and stores the stack status in stack status storage as stated in step 104. When the last bytecode instruction is analyzed as stated step 102, the loop is ended at step 108 and partial compilation is completed.
In prior art FIG. 1B, remaining compilation occurs in flow diagram 150 which shows further loops through the bytecode instructions analyzing an individual bytecode instruction during each loop as stated in step 152. The stack status storage and bytecode instruction are then used to translate the bytecode instruction into machine code as stated in step 154. When the last bytecode instruction is translated as stated in step 152, the loop is ended at step 158 and compilation is completed.
In prior art FIG. 2, an illustrative flow diagram of traditional bytecode verification is shown in flow diagram 200 which loops through the bytecode instructions, analyzing each until the last instruction is reached as stated in step 202. During each loop, the method analyzes a single bytecode instruction and if the method determines it has reached the last bytecode instruction, the loop is ended at step 214. Otherwise, the method determines the bytecode instruction position as stated in step 204. If the bytecode instruction being analyzed is at the beginning of a basic block, then the method reads the global stack status from bytecode auxiliary data and stores it as stated in step 206. After storage, the method verifies that the stored global stack status is compliant with the bytecode instruction as stated in step 208. If the bytecode instruction is not at the beginning of a basic block as stated in step 204, the global stack status is not read, but is verified to ensure the global stack status is compliant with the bytecode instruction as stated in step 208. In this case, step 206 is omitted. The global stack status is then changed according to the bytecode instruction as stated in step 210. This is repeated for each bytecode instruction until the last instruction is analyzed as stated in step 202 and the loop is ended at step 214.
In FIGS. 3, 4A and 4B an illustrative flow diagram of the new method is shown. It may be noted from earlier prior art Figures that the pass through the bytecode instructions that is required for verification resembles compilation procedures. In the case of verification, the effect of the bytecode instruction on the stack must be analyzed and stored as a global stack status (i.e. a single storage location that is updated for every bytecode). This global storage stack must be filled from auxiliary data each time a basic block of data is entered. In the case of compilation, a similar analysis must be performed, however the stack status must be stored (in less detail) in stack status storage for each bytecode instruction analyzed.
The present invention provides an improved method and apparatus to perform platform independent bytecode compilation and verification creating optimized machine code on an independent platform. The present invention creates a new method in which bytecode compilation is combined with instruction verification thereby increasing the speed and applicability of bytecode programming.
FIG. 3 is a main flowchart of a method 300 for combined bytecode verification and compilation in accordance with the new invention. In step 302, a class file placed on the development or target system is selected and a first method within the first class file is selected in step 304. At this point, the stack status for the first instruction and handler targets is set up in step 306. In step 308 a first bytecode instruction is selected and evaluated to determine if the instruction is setup in step 310. If the instruction is setup, the instruction is analyzed as outlined in FIGS. 4A and 4B. If the instruction is not setup, the next setup instruction is selected in step 312 and types are loaded from the stack map in step 314.
Once the instruction has been analyzed in step 316, the following instruction is selected in step 318. If there are no remaining instructions as determined in step 320, the next method is selected in steps 322 and 328. If there are no remaining methods, the next class is selected in steps 324 and 330. If there are no remaining classes, the evaluation returns in step 326.
FIGS. 4A and 4B are subset flowcharts of a method 400 for the analyses of each bytecode instruction from step 316 in FIG. 3. In step 402, the selected instruction is checked to determine if it is within the scope of the exception handler. If it is, the compatibility between the actual local variable types and the exception handler stack map entry in bytecode is verified in step 404. If not, the instruction is set to “handled” in step 406 and the stack status of the actual instruction is copied to the new stack status.
Next the instruction is evaluated to determine if there is a resulting pop from the stack in step 408 or a resulting push to the stack in step 414. If there is a resulting pop from the stack indicating an overflow condition, the compatibility between the stack status and expected values is verified in step 410 and the new stack status is then modified according to the instruction in step 412. If there is a resulting push to the stack indicating an underflow condition, the new stack status is modified according to the instruction and new actual stack types are set according to the instruction in step 416.
In steps 418 and 422 the instruction is evaluated to determine if the instruction reads a local variable or writes to a local variable. If the instruction reads a local variable, the compatibility between the actual local variable type and the instruction is verified in step 420. If the instruction writes to a local variable, the variable type is modified according to the actual instruction.
In step 426, the first successor instruction is evaluated. The instruction immediately following the actual instruction, determined in step 428, is dealt with in step 438 after all other successor instructions have been dealt with by step 436. Each successor instruction other than the instruction immediately following the actual instruction is evaluated in step 430 to determine if the instruction is marked as “none”. If the successor instruction is marked as “none”, the stack status of the successor instruction is initialized to the new stack status and the successor instruction is marked as “setup” in step 432 and the compatibility between the new stack status and the stack map for the successor instruction in the bytecode is verified. The compatibility between the actual stack, local variable types and stack map for the successor instruction is verified in step 434 and repeated until no further successor instructions remain.
If the instruction is immediately following the actual instruction, step 438 determines if the instruction is a successor instruction and if so, step 440 determines if the instruction is marked as “none”. If the successor instruction is marked as “none”, the stack status of the following instruction is initialized to the new stack status and the following instruction is marked as “setup” in step 442. The compatibility between the new stack status and the stack map is verified. If there is a stack map for the successor instruction in step 444, the compatibility between the actual stack, local variable types and stack map for the successor instruction is verified in step 446 and types are loaded from the stack map in step 448. Once completed, step 450 returns to the main flowchart at step 318.
Referring to Table 1, the new combined compilation and verification method places each class file in the development or target system, at which point each method in the class containing bytecode instructions is analyzed. The stack status for the first instruction and handler targets is setup. Temporary storage is created for stack status and marks for each bytecode instruction, in addition temporary storage for actual types of stack values and local variables is created.
Next, the method initializes the stack status of the first instruction to empty and the stack status of the exception handler target instructions is initialized to contain the given exception. The marks of the first instruction and handler target instructions are set to “setup” and all other marks are set to “none”. The method signature is then used to initialize actual local variable types and the first bytecode instruction is set to be the actual instruction. This is repeated until no further instructions are marked as “setup”.
The next subsequent bytecode instruction in turn which is marked as “setup” is set to be the actual instruction. The actual stack and local variable types from the stack map belonging to the actual instruction (each bytecode instruction) are loaded. If the actual instruction is within the scope of the exception handler, the compatibility between the actual local variable types and the exception handler stack map entry in bytecode is verified. Once verified or where the actual instruction is not within the scope of the exception handler, the selected bytecode instruction is set to “handled” and the stack status of the actual instruction is copied to new stack status.
If the actual instruction pops one or more values from the stack, the compatibility between the stack status and expected values is verified and the new stack status is then modified according to the instruction. If the actual instruction pushes one or more values to the stack, the new stack status is modified according to the instruction and new actual stack types are set according to the instruction.
A check for overflow and underflow conditions occurs next. If the actual instruction pops one or more values from the stack, check for underflow and verify the compatibility between the stack status and expected values and then modify the new stack status is according to the instruction. If there is no underflow condition, overflow conditions are evaluated. If the actual instruction pushes one or more values to the stack, check for overflow and modify the new stack status according to the instruction and new actual stack types are set according to the instruction.
Once overflow and underflow checks are performed, the instruction is evaluated to determine if it reads a local variable or writes to a local variable. If the actual instruction reads a local variable, the compatibility between the actual local variable type and the instruction is verified. If the actual instruction writes to a local variable, the actual local variable type is modified according to the actual instruction.
The first successor instruction is then evaluated. For each successor instruction except the one immediately following the actual instruction, if the successor instruction is marked as “none”, the stack status of the successor instruction is initialized to the new stack status and the successor instruction is marked as “setup”. The compatibility between the new stack status and the stack map for the successor instruction in the bytecode is verified. Once the successor is “setup”, or if it was already “setup”, the compatibility between the actual stack, local variable types and stack map for the successor instruction in the bytecode is also verified.
If the instruction immediately following the actual instruction is a successor of the actual instruction and the following instruction is marked as “none”, the stack status of the following instruction is initialized to the new stack status. The following instruction is then marked as “setup”. Once the successor is “setup”, or if it was already “setup”, if there is a stack map in the bytecode for the following instruction, the compatibility between new stack status and the stack map is verified. The compatibility between actual stack, local variable types and the stack map is also verified. The actual types are then loaded from the stack map and the actual instruction is changed to the immediately following instruction. The process is repeated for each method within each class file, and thereafter repeated for each class file.

Prior art improvement methods in which computer idle time is filed with compilation steps and pre-verification, do not teach a method of combining verification and compilation steps. Also, idle time compilation is constantly subject to interruption and pre-verification may not eliminate all malicious code present. The result of using the new method shown in FIGS. 3, 4A, 4B and Table 1, is complete compilation and verification into optimized machine code with fewer program operations and reduced process times.

TABLE 1


Combined Compilation and Verification of JAVA bytecode

Object Class File containing bytecode instructions received.

For each Class File to be analyzed

{

For each method in the class containing bytecode instructions

{

Create storage for each instruction to store stack status and marks.

Create storage to store actual types of stack values and local variables.

Initialize stack status of first instruction to empty.

Initialize stack status of exception handler target instructions to contain the given

exception object.

Set marks of first instruction and handler target instructions to ‘setup‘.

Set all other marks to ‘none‘.

Initialize actual local variable types from method signature.

Set the first instruction of the bytecode to be the actual instruction.

Do until there are no more instructions marked as ‘setup‘

{

If actual instruction is not marked as ‘setup‘ /* New basic block */

{

Select next instruction in the bytecode marked as ‘setup‘ as the actual

instruction.

Load actual stack and local variable types from the stack map in bytecode

belonging to the actual instruction.

}

If the actual instruction is in the scope of an exception handler

{

Verify compatibility between actual local variable types and stack map for the

exception

handler entry in bytecode.

}

Set the mark of selected instruction to ‘handled‘.

Copy stack status of actual instruction to new stack status.

If the actual instruction pops one or more values from the stack

{

Verify compatibility between the stack status and types and the values

expected by the instruction.

Modify new stack status according to the instruction.

}

If the actual instruction pushes one or more values to the stack

{

Modify new stack status according to the instruction.

Set new actual stack types according to the instruction.

}

If the actual instruction reads a local variable

{

Verify compatibility between actual local variable types and the instruction.

}

If the actual instruction writes to a local variable

{

Modify actual local variable types according to the instruction.

}

For all successor instructions except the one immediately following the actual

instruction

{

If the successor instruction is marked as ‘none‘

{

Initialize the stack status of the successor instruction to the new

stack

status.

Mark successor instruction as ‘setup‘.

}

Verify compatibility between new stack status and stack map for the

successor

instruction in the bytecode.

Verify compatibility between actual stack and local variable types and

stack map for the successorinstruction in the bytecode.

}

If the instruction immediately following the actual instruction is a successor of the

actual

instruction

{

If following instructions is marked as ‘none‘

{

Initialize the stack status of following instruction to the new stack status.

Mark following instruction as ‘setup‘.

}

If there is a stack map in the bytecode for the following instruction

{

Verify compatibility between new stack status and the stack map.

Verify compatibility between actual stack and local variable types

and the stack map. Load the actual types from the

stack map.

}

Change the actual instruction to the immediately following instruction.

} /* Do until */

} /* For each method */

} /* For each class file */

	TABLE 2

Claims

1. A computer apparatus suitable for use in the combined compilation and verification of platform neutral bytecode instructions resulting in optimized machine code, comprising:

a central processing unit (CPU); a computer memory coupled to said CPU, said computer memory comprised of a computer readable medium;

a compilation-verification program embodied on said computer readable medium, said compilation-verification program comprising:

a first code segment that receives a bytecode listing; and

a second code segment that compiles said bytecode listing into machine code and during the process of compilation simultaneously verifies said bytecode listing is free of malicious and improper code.

2. A computer apparatus as recited in claim 1 wherein said computer program further includes a third code segment that interprets and executes said machine code.

3. A computer apparatus as recited in claim 1 wherein said second code segment evaluates said bytecode listing to detect improper data types and improper stack usage.

4. A computer apparatus as recited in claim 1 wherein said second code segment evaluates said bytecode listing for complete compilation of said bytecode instructions into said machine code.

5. A computer implemented method for facilitating combined compilation and verification of bytecode instructions resulting in machine code, said computer implemented method characterized in that compilation and verification of byte code are performed simultaneously.

6. A computer implemented method as recited in claim 5, wherein said bytecode instructions are platform neutral bytecode instructions.

7. A computer implemented method as recited in claim 5 wherein said compilation procedure generates optimized machine code.

8. A computer implemented method as recited in claim 5 wherein when said bytecode instructions cannot be verified, said compilation is not completed.

9. A computer implemented method for combined verification and compilation of an object class file comprising:

receiving an object class file including at least one method; and for each method in said object class file, performing a simultaneous compilation and verification of said method.

10. A computer implemented method as recited in claim 9, wherein said at least one method includes platform neutral bytecode.

11. A computer implemented method as recited in claim 10, wherein said platform neutral bytecode is JAVA bytecode.

12. A computer readable medium comprising computer executable instructions for:

receiving a plurality of platform neutral bytecodes; and

simultaneously compiling and verifying said plurality of platform neutral bytecodes.

13. A computer readable medium as recited in claim 12, wherein said plurality of platform neutral bytecodes are arranged within at least one class.

14. A computer readable medium as recited in claim 13, wherein said simultaneous compiling and verifying is performed on each class individually.

15. A computer readable medium as recited in claim 14, wherein each class is comprised of methods, and said simultaneous compiling and verifying is performed on each method individually.