US8132157B2 - Method of automatic regression testing - Google Patents

Method of automatic regression testing Download PDF

Info

Publication number
US8132157B2
US8132157B2 US12/015,751 US1575108A US8132157B2 US 8132157 B2 US8132157 B2 US 8132157B2 US 1575108 A US1575108 A US 1575108A US 8132157 B2 US8132157 B2 US 8132157B2
Authority
US
United States
Prior art keywords
version
program
binary code
error
block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US12/015,751
Other versions
US20090187788A1 (en
Inventor
Charulatha Dhuvur
Eli M. Dow
Marie R. Laser
Jassie Yu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to US12/015,751 priority Critical patent/US8132157B2/en
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DHUVUR, CHARULATHA, DOW, ELI M., LASER, MARIE R., YU, JESSIE
Publication of US20090187788A1 publication Critical patent/US20090187788A1/en
Application granted granted Critical
Publication of US8132157B2 publication Critical patent/US8132157B2/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3688Test management for test execution, e.g. scheduling of test suites

Definitions

  • IBM® is a registered trademark of International Business Machines Corporation, Armonk, N.Y., U.S.A. Other names used herein may be registered trademarks, trademarks or product names of International Business Machines Corporation or other companies.
  • This invention generally relates to regression testing. More particularly, this invention relates to a method of automatic regression testing through embedding binary code.
  • a user debugging the program performs a regression analysis or test to determine if the error(s) were caused by introduction of new changes to the program. Oftentimes the testing requires rigorous testing of all changes to the program. This testing of all changes may be complicated, or in some cases, impossible, if all the changes are not logged, accessible, and/or documented.
  • a method of automatic regression testing includes loading binary code representing a first version of a program, extracting a second version of the program embedded within the binary code of the first version of the program, executing a standalone model of the second version of the program based on the extracted second version, wherein executing includes executing a set of instructions to identify at least one error, determining if the standalone model causes the at least one error, and storing results based on the determining.
  • FIG. 1 illustrates a flowchart of a method of automatic regression testing, according to an exemplary embodiment
  • FIG. 2 illustrates a computer apparatus for automatic regression testing, according to an exemplary embodiment.
  • a method which significantly increases the simplicity of regression testing. This increase in simplicity results in better adaptability to changes in computer programs, and an increased ability to analyze errors resulting from changes in computer programs.
  • a method of automatic regression testing includes embedding binary code from a previous software release within a new software release.
  • the embedded binary code may be used in regression testing as a control point for analysis to compare the previous version of the program with the newer version.
  • all new changes may be efficiently tested and/or analyzed without iteration through data logs of changes resulting in a more thorough analysis of the program changes.
  • the testing may be automated due to the binary code being embedded, as the changes are inherent, and little interaction between the process and a computer user or administrator may be necessary during testing. This may reduce the possibility for error during testing.
  • the new software release may be linked with the embedded binary code from its previous release. If an error is detected during program execution, automatic regression testing may be performed. Alternatively, automatic regression testing may be performed at any time, for example, continuously during program execution, or automatic regression testing may be performed between software version releases. In an example where a set of instructions causes an error, the set of instructions causing the error may be run using the linked-embedded binary code from the previous software release in an automatic fashion. If the set of instructions is executed successfully with the embedded code, but causes errors using the newer program release, the error is more likely to be caused by changes in the newer version of the software. Therefore, the automatic regression test algorithm may return a result identifying that a change in the program has resulted in a program error. The automatic regression test algorithm may also prompt for user feedback for further action such as a detailed test analysis (e.g., test log or results), or other similar feedback.
  • a detailed test analysis e.g., test log or results
  • the method 100 includes loading binary code for a program at block 101 .
  • the binary code may be binary code for a program which has thrown an exception, fault, and/or other type or error, or for which automatic regression testing has been initiated by any other suitable means (e.g., through continuous testing or software release analysis).
  • the method 100 may be initiated by the exception itself (i.e., fully automated testing), may be initiated through user input (partially automated testing), may be run continuously, and proceeds automatically with testing.
  • the method 100 further includes verifying if the loaded binary (i.e., from block 101 ) is a tall binary at block 110 . For example, if there is embedded binary code or a plurality of different modules/program versions stacked or embedded within the loaded binary code, the loaded binary code may be termed a tall binary. If the loaded binary is a tall binary, the method 100 includes jumping to the newest version of the program at block 102 .
  • the method further includes extracting a previous (or embedded) program version at block 103 .
  • the previous program version is in the form of an embedded binary code linked to the newest program version of the binary code loaded at block 101 .
  • the method includes executing a standalone version of the embedded binary code at block 104 .
  • the standalone version being executed is fed any instructions which may have caused the error(s) initiating the method 100 .
  • the standalone version may be a model identical to the previous version of the program.
  • a standalone model, standalone version, or other “standalone” binary code execution is an isolated program based on binary code.
  • the standalone model is isolated in that it is executed as if it is a separate program, and may be given instructions as if it is a separate program. It follows that the standalone model also outputs results as if it is a separate program as well.
  • the method 100 further includes verifying if any exceptions (i.e., errors) have occurred in the standalone version at block 111 . If an error(s) has occurred, the method includes logging and/or storing the binary version at block 106 . This stored result may be accessed during or after testing by a user or administrator to determine which changes may have caused the error(s). Further, the method 100 includes determining if there is another stored binary code version at block 112 . For example, a plurality of previous versions of a program may be embedded and linked within the binary code loaded at bock 101 . If there is another binary code version embedded, the method includes jumping to a next binary code version at block 105 .
  • exceptions i.e., errors
  • the method 100 may iterate through blocks 103 , 104 , 111 , 106 , 112 , and 105 and function as described above if there are previous versions of binary code embedded within the binary code loaded at block 101 , and if there are errors thrown in respective standalone executions at block 111 .
  • the method 100 includes determining if the binary code loaded at block 101 includes a previous defect (i.e., a previous binary code version at any point in automatic regression testing results in an error) at block 113 . If there is a previous defect, the method 100 includes opening the defect against a version stored at block 106 , at block 107 , and ending the automatic testing algorithm at block 108 . If there is no previous defect, the method 100 includes ending testing at block 108 .
  • a previous defect i.e., a previous binary code version at any point in automatic regression testing results in an error
  • the method 100 includes determining if the binary code loaded at block 101 includes a previous defect at any point in automatic regression testing at block 113 . If there is a previous defect, the method 100 includes opening the defect against a version stored at block 106 , at block 107 , and ending the automatic testing algorithm at block 108 . If there is no previous defect, the method 100 includes ending testing at block 108 .
  • the method 100 determines that the binary code loaded at block 101 is not a tall binary, the method 100 includes executing the binary code loaded at block 101 as a standalone program at block 104 , and continues to block 108 according to the description given hereinbefore.
  • block 108 may include a notification algorithm or somewhat similar notification execution methodology.
  • the method 100 may include notifying a system administrator or user of the results of automatic regression testing at block 108 .
  • the method 100 may include writing results of automatic regression testing to a log or log file, directing results to an automatic policy manager of a computer system implementing the automatic regression testing, and/or providing a general system notification at block 108 .
  • any combination of the above examples should be includes within the scope of example embodiments, and furthermore, further example notification options or methodologies not included herein may be applicable to example embodiments, depending upon any particular implementation.
  • FIG. 2 illustrates a computer apparatus for regression testing, according to an exemplary embodiment. Therefore, portions or the entirety of the method may be executed as instructions in a processor 202 of the computer system 200 .
  • the computer system 200 includes memory 201 for storage of instructions and information, input device(s) 203 for computer communication, and display device 204 .
  • the present invention may be implemented, in software, for example, as any suitable computer program on a computer system somewhat system to computer system 200 .
  • a program in accordance with the present invention may be a computer program product causing a computer to execute the example method described herein.
  • the computer program product may include a computer-readable medium having computer program logic or code portions embodied thereon for enabling a processor (e.g., 202 ) of a computer apparatus (e.g., 200 ) to perform one or more functions in accordance with one or more of the example methodologies described above.
  • the computer program logic may thus cause the processor to perform one or more of the example methodologies, or one or more functions of a given methodology described herein.
  • the computer-readable storage medium may be a built-in medium installed inside a computer main body or removable medium arranged so that it can be separated from the computer main body.
  • Examples of the built-in medium include, but are not limited to, rewriteable non-volatile memories, such as RAMs, ROMs, flash memories, and hard disks.
  • Examples of a removable medium may include, but are not limited to, optical storage media such as CD-ROMs and DVDs; magneto-optical storage media such as MOs; magnetism storage media such as floppy disks (trademark), cassette tapes, and removable hard disks; media with a built-in rewriteable non-volatile memory such as memory cards; and media with a built-in ROM, such as ROM cassettes.
  • Such programs when recorded on computer-readable storage media, may be readily stored and distributed.
  • the storage medium as it is read by a computer, may enable the method(s) disclosed herein, in accordance with an exemplary embodiment of the present invention.

Abstract

A method of automatic regression testing includes loading binary code representing a first version of a program, extracting a second version of the program embedded within the binary code of the first version of the program, executing a standalone model of the second version of the program based on the extracted second version, wherein executing includes executing a set of instructions to identify at least one error, determining if the standalone model causes the at least one error, and storing results based on the determining.

Description

TRADEMARKS
IBM® is a registered trademark of International Business Machines Corporation, Armonk, N.Y., U.S.A. Other names used herein may be registered trademarks, trademarks or product names of International Business Machines Corporation or other companies.
BACKGROUND
1. Technical Field
This invention generally relates to regression testing. More particularly, this invention relates to a method of automatic regression testing through embedding binary code.
2. Description of Background
Generally, if an error or errors occur in a program or application, a user debugging the program performs a regression analysis or test to determine if the error(s) were caused by introduction of new changes to the program. Oftentimes the testing requires rigorous testing of all changes to the program. This testing of all changes may be complicated, or in some cases, impossible, if all the changes are not logged, accessible, and/or documented.
SUMMARY
A method of automatic regression testing includes loading binary code representing a first version of a program, extracting a second version of the program embedded within the binary code of the first version of the program, executing a standalone model of the second version of the program based on the extracted second version, wherein executing includes executing a set of instructions to identify at least one error, determining if the standalone model causes the at least one error, and storing results based on the determining.
Additional features and advantages are realized through the techniques of the exemplary embodiments described herein. Other embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed invention. For a better understanding of the invention with advantages and features, refer to the detailed description and to the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 illustrates a flowchart of a method of automatic regression testing, according to an exemplary embodiment; and
FIG. 2 illustrates a computer apparatus for automatic regression testing, according to an exemplary embodiment.
The detailed description explains an exemplary embodiment, together with advantages and features, by way of example with reference to the drawings.
DETAILED DESCRIPTION
According to an exemplary embodiment, a method is provided which significantly increases the simplicity of regression testing. This increase in simplicity results in better adaptability to changes in computer programs, and an increased ability to analyze errors resulting from changes in computer programs.
According to an exemplary embodiment, a method of automatic regression testing includes embedding binary code from a previous software release within a new software release. The embedded binary code may be used in regression testing as a control point for analysis to compare the previous version of the program with the newer version. In this manner, all new changes may be efficiently tested and/or analyzed without iteration through data logs of changes resulting in a more thorough analysis of the program changes. The testing may be automated due to the binary code being embedded, as the changes are inherent, and little interaction between the process and a computer user or administrator may be necessary during testing. This may reduce the possibility for error during testing.
According to an exemplary embodiment, the new software release may be linked with the embedded binary code from its previous release. If an error is detected during program execution, automatic regression testing may be performed. Alternatively, automatic regression testing may be performed at any time, for example, continuously during program execution, or automatic regression testing may be performed between software version releases. In an example where a set of instructions causes an error, the set of instructions causing the error may be run using the linked-embedded binary code from the previous software release in an automatic fashion. If the set of instructions is executed successfully with the embedded code, but causes errors using the newer program release, the error is more likely to be caused by changes in the newer version of the software. Therefore, the automatic regression test algorithm may return a result identifying that a change in the program has resulted in a program error. The automatic regression test algorithm may also prompt for user feedback for further action such as a detailed test analysis (e.g., test log or results), or other similar feedback.
Turning to FIG. 1, a flowchart of a method of automatic regression testing is illustrated. According to FIG. 1, the method 100 includes loading binary code for a program at block 101. The binary code may be binary code for a program which has thrown an exception, fault, and/or other type or error, or for which automatic regression testing has been initiated by any other suitable means (e.g., through continuous testing or software release analysis). The method 100 may be initiated by the exception itself (i.e., fully automated testing), may be initiated through user input (partially automated testing), may be run continuously, and proceeds automatically with testing.
The method 100 further includes verifying if the loaded binary (i.e., from block 101) is a tall binary at block 110. For example, if there is embedded binary code or a plurality of different modules/program versions stacked or embedded within the loaded binary code, the loaded binary code may be termed a tall binary. If the loaded binary is a tall binary, the method 100 includes jumping to the newest version of the program at block 102.
The method further includes extracting a previous (or embedded) program version at block 103. The previous program version is in the form of an embedded binary code linked to the newest program version of the binary code loaded at block 101. Upon loading the embedded binary code, or at substantially the same time as loading the embedded binary code, the method includes executing a standalone version of the embedded binary code at block 104. The standalone version being executed is fed any instructions which may have caused the error(s) initiating the method 100. The standalone version may be a model identical to the previous version of the program.
As used herein, a standalone model, standalone version, or other “standalone” binary code execution, is an isolated program based on binary code. The standalone model is isolated in that it is executed as if it is a separate program, and may be given instructions as if it is a separate program. It follows that the standalone model also outputs results as if it is a separate program as well.
The method 100 further includes verifying if any exceptions (i.e., errors) have occurred in the standalone version at block 111. If an error(s) has occurred, the method includes logging and/or storing the binary version at block 106. This stored result may be accessed during or after testing by a user or administrator to determine which changes may have caused the error(s). Further, the method 100 includes determining if there is another stored binary code version at block 112. For example, a plurality of previous versions of a program may be embedded and linked within the binary code loaded at bock 101. If there is another binary code version embedded, the method includes jumping to a next binary code version at block 105. Therefore, the method 100 may iterate through blocks 103, 104, 111, 106, 112, and 105 and function as described above if there are previous versions of binary code embedded within the binary code loaded at block 101, and if there are errors thrown in respective standalone executions at block 111.
If any particular standalone execution does not result in an error at block 111, the method 100 includes determining if the binary code loaded at block 101 includes a previous defect (i.e., a previous binary code version at any point in automatic regression testing results in an error) at block 113. If there is a previous defect, the method 100 includes opening the defect against a version stored at block 106, at block 107, and ending the automatic testing algorithm at block 108. If there is no previous defect, the method 100 includes ending testing at block 108.
Alternatively, if the iterative portion of the method 100 described hereinbefore “runs out” or exhausts embedded previous versions of binary code using verification at block 112, the method 100 includes determining if the binary code loaded at block 101 includes a previous defect at any point in automatic regression testing at block 113. If there is a previous defect, the method 100 includes opening the defect against a version stored at block 106, at block 107, and ending the automatic testing algorithm at block 108. If there is no previous defect, the method 100 includes ending testing at block 108.
Returning to block 110, if the method 100 determines that the binary code loaded at block 101 is not a tall binary, the method 100 includes executing the binary code loaded at block 101 as a standalone program at block 104, and continues to block 108 according to the description given hereinbefore.
It is noted that block 108 may include a notification algorithm or somewhat similar notification execution methodology. For example, upon ending execution of automatic regression testing, the method 100 may include notifying a system administrator or user of the results of automatic regression testing at block 108. Similarly, the method 100 may include writing results of automatic regression testing to a log or log file, directing results to an automatic policy manager of a computer system implementing the automatic regression testing, and/or providing a general system notification at block 108. It is further noted that any combination of the above examples should be includes within the scope of example embodiments, and furthermore, further example notification options or methodologies not included herein may be applicable to example embodiments, depending upon any particular implementation.
Furthermore, according to an exemplary embodiment, the method described hereinbefore may be implemented by a computer system or apparatus. For example, FIG. 2 illustrates a computer apparatus for regression testing, according to an exemplary embodiment. Therefore, portions or the entirety of the method may be executed as instructions in a processor 202 of the computer system 200. The computer system 200 includes memory 201 for storage of instructions and information, input device(s) 203 for computer communication, and display device 204. Thus, the present invention may be implemented, in software, for example, as any suitable computer program on a computer system somewhat system to computer system 200. For example, a program in accordance with the present invention may be a computer program product causing a computer to execute the example method described herein.
The computer program product may include a computer-readable medium having computer program logic or code portions embodied thereon for enabling a processor (e.g., 202) of a computer apparatus (e.g., 200) to perform one or more functions in accordance with one or more of the example methodologies described above. The computer program logic may thus cause the processor to perform one or more of the example methodologies, or one or more functions of a given methodology described herein.
The computer-readable storage medium may be a built-in medium installed inside a computer main body or removable medium arranged so that it can be separated from the computer main body. Examples of the built-in medium include, but are not limited to, rewriteable non-volatile memories, such as RAMs, ROMs, flash memories, and hard disks. Examples of a removable medium may include, but are not limited to, optical storage media such as CD-ROMs and DVDs; magneto-optical storage media such as MOs; magnetism storage media such as floppy disks (trademark), cassette tapes, and removable hard disks; media with a built-in rewriteable non-volatile memory such as memory cards; and media with a built-in ROM, such as ROM cassettes.
Further, such programs, when recorded on computer-readable storage media, may be readily stored and distributed. The storage medium, as it is read by a computer, may enable the method(s) disclosed herein, in accordance with an exemplary embodiment of the present invention.
With an exemplary embodiment of the present invention having thus been described, it will be obvious that the same may be varied in many ways. The description of the invention hereinbefore uses this example, including the best mode, to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims. Such variations are not to be regarded as a departure from the spirit and scope of the present invention, and all such modifications are intended to be included within the scope of the present invention as stated in the following claims.

Claims (3)

What is claimed is:
1. A method of automatic regression testing, comprising:
loading binary code representing a first version of a program;
extracting a second version of the program embedded within the binary code of the first version of the program;
executing a standalone model of the second version of the program based on the extracted second version, wherein executing includes executing a set of instructions to identify at least one error;
determining if the standalone model causes the at least one error;
storing results if the standalone model causes the at least one error; and
iterating through all previous versions of the program embedded within the binary code of the first version of the program, wherein iterating includes,
extracting a next version of the program embedded within the binary code of the first version of the program,
executing a next standalone model of the next version of the program based on the extracted next version, wherein executing includes executing the set of instructions to identify the at least one error,
determining if the next standalone model causes the at least one error, and
storing results if the next standalone model causes the at least one error.
2. The method of claim 1, further comprising:
determining if the binary code representing the first version of the program includes a plurality of embedded previous versions of the program packaged or stacked within the first version of the program.
3. The method of claim 1, further comprising:
returning the stored results if the standalone model causes the at least one error.
US12/015,751 2008-01-17 2008-01-17 Method of automatic regression testing Expired - Fee Related US8132157B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/015,751 US8132157B2 (en) 2008-01-17 2008-01-17 Method of automatic regression testing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US12/015,751 US8132157B2 (en) 2008-01-17 2008-01-17 Method of automatic regression testing

Publications (2)

Publication Number Publication Date
US20090187788A1 US20090187788A1 (en) 2009-07-23
US8132157B2 true US8132157B2 (en) 2012-03-06

Family

ID=40877393

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/015,751 Expired - Fee Related US8132157B2 (en) 2008-01-17 2008-01-17 Method of automatic regression testing

Country Status (1)

Country Link
US (1) US8132157B2 (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100299654A1 (en) * 2009-05-21 2010-11-25 Microsoft Corporation Approach for root causing regression bugs
US20120023373A1 (en) * 2010-07-23 2012-01-26 Salesforce.Com, Inc. Testing a software application used in a database system
US20120239981A1 (en) * 2011-03-15 2012-09-20 International Business Machines Corporation Method To Detect Firmware / Software Errors For Hardware Monitoring
US20120317545A1 (en) * 2011-06-10 2012-12-13 International Business Machines Corporation Systems and methods for providing feedback for software components
US8924932B2 (en) 2013-04-11 2014-12-30 International Business Machines Corporation Using stack data and source code to rank program changes
CN104468259A (en) * 2014-11-11 2015-03-25 上海新炬网络信息技术有限公司 Method for automatically testing communication service expense
US20150286557A1 (en) * 2014-04-08 2015-10-08 Oracle International Corporation Embedded instruction sets for use in testing and error simulation of computing programs
US20160062765A1 (en) * 2014-09-02 2016-03-03 International Business Machines Corporation Identifying semantic differences between source code versions
US9606901B1 (en) * 2014-08-05 2017-03-28 Amdocs Software Systems Limited System, method, and computer program for generating a detailed design of at least one telecommunications based integration testing project
US9877243B2 (en) 2014-07-16 2018-01-23 International Business Machines Corporation Determining a location of a mobile device
US10049031B2 (en) 2014-12-09 2018-08-14 International Business Machines Corporation Correlation of violating change sets in regression testing of computer software
US10761828B2 (en) 2017-01-06 2020-09-01 Microsoft Technology Licensing, Llc Deviation finder
US11036613B1 (en) 2020-03-30 2021-06-15 Bank Of America Corporation Regression analysis for software development and management using machine learning
US11144435B1 (en) 2020-03-30 2021-10-12 Bank Of America Corporation Test case generation for software development using machine learning

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090055805A1 (en) * 2007-08-24 2009-02-26 International Business Machines Corporation Method and System for Testing Software
CN103164334B (en) 2011-12-19 2016-03-30 国际商业机器公司 Detect the system and method for the breakaway poing in web application automatic test case
CN103631705B (en) * 2012-08-24 2018-01-05 百度在线网络技术(北京)有限公司 A kind of regression testing method and device for search engine
CN113641573A (en) * 2021-07-26 2021-11-12 安徽中科国创高可信软件有限公司 Revision log-based automatic testing method and system for program analysis software

Citations (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5651111A (en) * 1994-06-07 1997-07-22 Digital Equipment Corporation Method and apparatus for producing a software test system using complementary code to resolve external dependencies
US5673387A (en) 1994-05-16 1997-09-30 Lucent Technologies Inc. System and method for selecting test units to be re-run in software regression testing
US5694540A (en) 1994-12-15 1997-12-02 Lucent Technologies Inc. Automated software regression test and compilation system
US5784553A (en) * 1996-01-16 1998-07-21 Parasoft Corporation Method and system for generating a computer program test suite using dynamic symbolic execution of JAVA programs
US6305010B2 (en) * 1997-12-04 2001-10-16 Incert Software Corporation Test, protection, and repair through binary code augmentation
US20030028856A1 (en) * 2001-08-01 2003-02-06 Apuzzo Joseph T. Method and apparatus for testing a software component using an abstraction matrix
US20030046681A1 (en) * 2001-08-30 2003-03-06 International Business Machines Corporation Integrated system and method for the management of a complete end-to-end software delivery process
US6550057B1 (en) * 1999-08-31 2003-04-15 Accenture Llp Piecemeal retrieval in an information services patterns environment
US20040088602A1 (en) 2002-11-05 2004-05-06 Cohen Richard M. Automated recording and replaying of software regression tests
US6766475B2 (en) 2001-01-04 2004-07-20 International Business Machines Corporation Method and apparatus for exercising an unknown program with a graphical user interface
US6907546B1 (en) * 2000-03-27 2005-06-14 Accenture Llp Language-driven interface for an automated testing framework
US6966013B2 (en) 2001-07-21 2005-11-15 International Business Machines Corporation Method and system for performing automated regression tests in a state-dependent data processing system
US6986125B2 (en) * 2001-08-01 2006-01-10 International Business Machines Corporation Method and apparatus for testing and evaluating a software component using an abstraction matrix
US20060107121A1 (en) 2004-10-25 2006-05-18 International Business Machines Corporation Method of speeding up regression testing using prior known failures to filter current new failures when compared to known good results
US20060129992A1 (en) 2004-11-10 2006-06-15 Oberholtzer Brian K Software test and performance monitoring system
US20060225041A1 (en) 2005-04-04 2006-10-05 International Business Machines Corporation Method for testing modified user documentation software for regressions
US20070006043A1 (en) 2005-06-29 2007-01-04 Markus Pins System and method for regression tests of user interfaces
US20070006041A1 (en) 2005-06-30 2007-01-04 Frank Brunswig Analytical regression testing on a software build
US7178063B1 (en) 2003-07-22 2007-02-13 Hewlett-Packard Development Company, L.P. Method and apparatus for ordering test cases for regression testing
US20070074149A1 (en) * 2005-08-26 2007-03-29 Microsoft Corporation Automated product defects analysis and reporting
US20100153785A1 (en) * 2006-10-30 2010-06-17 The Trustees Of Columbia University In The City Of New York Methods, media, and systems for detecting an anomalous sequence of function calls
US8006204B2 (en) * 1999-02-05 2011-08-23 Tensilica, Inc. Automated processor generation system for designing a configurable processor and method for the same

Patent Citations (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5673387A (en) 1994-05-16 1997-09-30 Lucent Technologies Inc. System and method for selecting test units to be re-run in software regression testing
US5651111A (en) * 1994-06-07 1997-07-22 Digital Equipment Corporation Method and apparatus for producing a software test system using complementary code to resolve external dependencies
US5694540A (en) 1994-12-15 1997-12-02 Lucent Technologies Inc. Automated software regression test and compilation system
US5784553A (en) * 1996-01-16 1998-07-21 Parasoft Corporation Method and system for generating a computer program test suite using dynamic symbolic execution of JAVA programs
US6305010B2 (en) * 1997-12-04 2001-10-16 Incert Software Corporation Test, protection, and repair through binary code augmentation
US8006204B2 (en) * 1999-02-05 2011-08-23 Tensilica, Inc. Automated processor generation system for designing a configurable processor and method for the same
US6550057B1 (en) * 1999-08-31 2003-04-15 Accenture Llp Piecemeal retrieval in an information services patterns environment
US6907546B1 (en) * 2000-03-27 2005-06-14 Accenture Llp Language-driven interface for an automated testing framework
US6766475B2 (en) 2001-01-04 2004-07-20 International Business Machines Corporation Method and apparatus for exercising an unknown program with a graphical user interface
US6966013B2 (en) 2001-07-21 2005-11-15 International Business Machines Corporation Method and system for performing automated regression tests in a state-dependent data processing system
US20030028856A1 (en) * 2001-08-01 2003-02-06 Apuzzo Joseph T. Method and apparatus for testing a software component using an abstraction matrix
US6986125B2 (en) * 2001-08-01 2006-01-10 International Business Machines Corporation Method and apparatus for testing and evaluating a software component using an abstraction matrix
US7735080B2 (en) * 2001-08-30 2010-06-08 International Business Machines Corporation Integrated system and method for the management of a complete end-to-end software delivery process
US20030046681A1 (en) * 2001-08-30 2003-03-06 International Business Machines Corporation Integrated system and method for the management of a complete end-to-end software delivery process
US20040088602A1 (en) 2002-11-05 2004-05-06 Cohen Richard M. Automated recording and replaying of software regression tests
US7178063B1 (en) 2003-07-22 2007-02-13 Hewlett-Packard Development Company, L.P. Method and apparatus for ordering test cases for regression testing
US20060107121A1 (en) 2004-10-25 2006-05-18 International Business Machines Corporation Method of speeding up regression testing using prior known failures to filter current new failures when compared to known good results
US20060129992A1 (en) 2004-11-10 2006-06-15 Oberholtzer Brian K Software test and performance monitoring system
US20060225041A1 (en) 2005-04-04 2006-10-05 International Business Machines Corporation Method for testing modified user documentation software for regressions
US20070006043A1 (en) 2005-06-29 2007-01-04 Markus Pins System and method for regression tests of user interfaces
US20070006041A1 (en) 2005-06-30 2007-01-04 Frank Brunswig Analytical regression testing on a software build
US20070074149A1 (en) * 2005-08-26 2007-03-29 Microsoft Corporation Automated product defects analysis and reporting
US20100153785A1 (en) * 2006-10-30 2010-06-17 The Trustees Of Columbia University In The City Of New York Methods, media, and systems for detecting an anomalous sequence of function calls

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Title: Compatibility and Regression Testing of COTS-Component-Based Software, author: Mariani, L et al, source: IEEE, dated: May 20, 2007. *
Title: Regression Test Selection for Black-box Dynamic Link Library Components author: Jiang Zheng et al, dated: May 20, 2007. *

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100299654A1 (en) * 2009-05-21 2010-11-25 Microsoft Corporation Approach for root causing regression bugs
US20120023373A1 (en) * 2010-07-23 2012-01-26 Salesforce.Com, Inc. Testing a software application used in a database system
US8510602B2 (en) * 2010-07-23 2013-08-13 Salesforce.Com, Inc. Testing a software application used in a database system
US20120239981A1 (en) * 2011-03-15 2012-09-20 International Business Machines Corporation Method To Detect Firmware / Software Errors For Hardware Monitoring
US20120317545A1 (en) * 2011-06-10 2012-12-13 International Business Machines Corporation Systems and methods for providing feedback for software components
US8572553B2 (en) * 2011-06-10 2013-10-29 International Business Machines Corporation Systems and methods for providing feedback for software components
US8924932B2 (en) 2013-04-11 2014-12-30 International Business Machines Corporation Using stack data and source code to rank program changes
US10229029B2 (en) * 2014-04-08 2019-03-12 Oracle International Corporation Embedded instruction sets for use in testing and error simulation of computing programs
US20150286557A1 (en) * 2014-04-08 2015-10-08 Oracle International Corporation Embedded instruction sets for use in testing and error simulation of computing programs
US10555226B2 (en) 2014-07-16 2020-02-04 International Business Machines Corporation Determining a location of a mobile device
US9877243B2 (en) 2014-07-16 2018-01-23 International Business Machines Corporation Determining a location of a mobile device
US9606901B1 (en) * 2014-08-05 2017-03-28 Amdocs Software Systems Limited System, method, and computer program for generating a detailed design of at least one telecommunications based integration testing project
US20160062765A1 (en) * 2014-09-02 2016-03-03 International Business Machines Corporation Identifying semantic differences between source code versions
US9594553B2 (en) * 2014-09-02 2017-03-14 International Business Machines Corporation Identifying semantic differences between source code versions
CN104468259B (en) * 2014-11-11 2018-01-05 上海新炬网络信息技术股份有限公司 A kind of communication traffic rate automated testing method
CN104468259A (en) * 2014-11-11 2015-03-25 上海新炬网络信息技术有限公司 Method for automatically testing communication service expense
US10049031B2 (en) 2014-12-09 2018-08-14 International Business Machines Corporation Correlation of violating change sets in regression testing of computer software
US10761828B2 (en) 2017-01-06 2020-09-01 Microsoft Technology Licensing, Llc Deviation finder
US11036613B1 (en) 2020-03-30 2021-06-15 Bank Of America Corporation Regression analysis for software development and management using machine learning
US11144435B1 (en) 2020-03-30 2021-10-12 Bank Of America Corporation Test case generation for software development using machine learning
US11556460B2 (en) 2020-03-30 2023-01-17 Bank Of America Corporation Test case generation for software development using machine learning

Also Published As

Publication number Publication date
US20090187788A1 (en) 2009-07-23

Similar Documents

Publication Publication Date Title
US8132157B2 (en) Method of automatic regression testing
US9921948B2 (en) Software commit risk level
Hemmati et al. Prioritizing manual test cases in traditional and rapid release environments
US8839203B2 (en) Code coverage-based taint perimeter detection
US20080282230A1 (en) Product, method and system for using window authentication in testing graphical user interface applications
US20150269060A1 (en) Development tools for logging and analyzing software bugs
Hemmati et al. Prioritizing manual test cases in rapid release environments
CN103257919B (en) Inspection method and device for script programs
US10049031B2 (en) Correlation of violating change sets in regression testing of computer software
US8327334B2 (en) Replay of program executions using cross-entropy
JP7404839B2 (en) Identification of software program defect location
US9317254B1 (en) Fault tolerance model, methods, and apparatuses and their validation techniques
US20200143061A1 (en) Method and apparatus for tracking location of input data that causes binary vulnerability
CN112181800A (en) Vehicle function testing device and control method thereof
JP2015011372A (en) Debug support system, method, program, and recording medium
CN100501686C (en) Method , processor and system for processing mistake during execution of mistake processing program
US8225298B2 (en) Tool for analyzing Siebel escripts
US9842044B2 (en) Commit sensitive tests
CN113971031A (en) Software package dependency relationship checking method and device
US8291383B1 (en) Code analysis via dual branch exploration
CN107562593A (en) A kind of automated testing method and system for verifying internal memory ECC functions
US20090271663A1 (en) Providing detailed program state information for error analysis
US11520689B2 (en) System and method for automatic program repair using fast-result test cases
US11256612B2 (en) Automated testing of program code under development
US20180225165A1 (en) Configurable system wide tests

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DHUVUR, CHARULATHA;DOW, ELI M.;LASER, MARIE R.;AND OTHERS;REEL/FRAME:020387/0980

Effective date: 20080116

REMI Maintenance fee reminder mailed
LAPS Lapse for failure to pay maintenance fees
STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20160306