GitBucket
Newer
zeynalig
<?xml version="1.0" ?>
<tei>
<teiHeader>
<fileDesc xml:id="_exception-analysis-resilience-ist"/>
</teiHeader>
<text xml:lang="en">
<front> Exception Handling Analysis and Transformation Using Fault Injection: <lb/>Study of Resilience Against Unanticipated Exceptions <lb/> Benoit Cornu, Lionel Seinturier, and Martin Monperrus <lb/>University of Lille & INRIA, Lille, France <lb/>December 18, 2014 <lb/> Abstract <lb/> Context: In software, there are the error cases that <lb/>are anticipated at specification and design time, those <lb/>encountered at development and testing time, and those <lb/>that were never anticipated before happening in produc-<lb/>tion. Is it possible to learn from the anticipated er-<lb/>rors during design to analyze and improve the resilience <lb/>against the unanticipated ones in production? <lb/> Objective: In this paper, we aim at analyzing and im-<lb/>proving how software handles unanticipated exceptions. <lb/>The first objective is to set up contracts about excep-<lb/>tion handling and a way to assess them automatically. <lb/>The second one is to improve the resilience capabilities <lb/>of software by transforming the source code. <lb/> Method: We devise an algorithm, called short-circuit <lb/>testing, which injects exceptions during test suite exe-<lb/>cution so as to simulate unanticipated errors. It is a <lb/>kind of fault-injection techniques dedicated to exception-<lb/>handling. This algorithm collects data that is used for <lb/>verifying two formal contracts that capture two resilience <lb/>properties w.r.t. exceptions: the source-independence <lb/>and pure-resilience contracts. Then we propose a code <lb/>modification technique, called " catch-stretching " which <lb/>allows error-recovery code (of the form of catch blocks) <lb/>to be more resilient. <lb/> Results: Our evaluation is performed on 9 open-source <lb/>software applications and consists in analyzing 241 catch <lb/>blocks executed during test suite execution. Our results <lb/>show that 101/214 of them (47%) expose resilience prop-<lb/>erties as defined by our exception contracts and that <lb/>84/214 of them (39%) can be transformed to be more <lb/>resilient. <lb/> Conclusion: Our work shows that it is possible to rea-<lb/>son on software resilience by injecting exceptions dur-<lb/>ing test suite execution. The collected information al-<lb/>lows us to apply one source code transformation that <lb/>improves the resilience against unanticipated exceptions. <lb/>This works best if the test suite exercises the exceptional <lb/>programming language constructs in many dierent sce-<lb/>narios. <lb/> </front>
<body>1 Introduction <lb/> At Fukushima's power plant, the anticipated maximum <lb/>tsunami height was 5.6m [1]. On March 11, 2011, the <lb/>highest waves struck at 15m. In software, there are the <lb/>errors anticipated at specification and design time, those <lb/>encountered at development and testing time, and those <lb/>that happen in the production mode yet never antici-<lb/>pated, as Fukushima's tsunami. <lb/>Resilience is " the persistence of service delivery that <lb/>can justifiably be trusted, when facing changes " [14]. <lb/> " Changes may refer to unexpected failures, attacks or ac-<lb/>cidents (e.g., disasters)" [25]. In this paper, we aim at <lb/>reasoning on the ability of software to correctly handle <lb/>unanticipated errors. <lb/>We focus on the resilience against exceptions [11]. Ex-<lb/>ceptions are programming language constructs for han-<lb/>dling errors. Exceptions are implemented in most main-<lb/>stream programming languages [12] and widely used in <lb/>practice [6]. In large and complex software, it is impos-<lb/>sible to predict all error cases that will happen in the <lb/>field (real-world environments are too unpredictable and <lb/>usages too diverse). In this paper, the resilience against <lb/>exceptions is the ability to correctly handle exceptions <lb/>that were never foreseen at specification time neither en-<lb/>countered during development or testing. This is our <lb/>deep motivation: helping developers to understand and <lb/>improve the resilience of their applications against unan-<lb/>ticipated exceptions. <lb/>The key diculty behind this research agenda is the <lb/>notion of " unanticipated " : how to reason on what one <lb/>does not know or on what one has never seen? To an-<lb/>swer this question, we start by proposing one definition <lb/>of " anticipated exception " . First, we consider well-tested <lb/>software (i.e. those with a good automated test suite). <lb/>Second, we define an " anticipated exception " as an ex-<lb/>ception that is triggered during the test suite execution. <lb/>To this extent, those exceptions and the associated be-<lb/>havior (error detection, error-recovery) are specified by <lb/>the test suite (which is a pragmatic approximation of an <lb/>
<page>1 <lb/></page>
idealized specification [24]). <lb/>Then, we simulate " unanticipated exceptions " by in-<lb/>jecting exceptions at appropriate places during test suite <lb/>execution. This fault injection technique, called " short-<lb/>circuit testing " , consists of throwing exceptions at the <lb/>beginning of try-blocks, simulating the worst error case <lb/>when the complete try-block is skipped due to the occur-<lb/>rence of a severe error. Despite the injected exceptions, <lb/>a large number of test cases still passes. When this hap-<lb/>pens, it means that the software under study is able to <lb/>resist to certain unanticipated exceptions. It can be said <lb/> " resilient " according to our definition of " Resilience". <lb/>The art of injecting exceptions during test suite exe-<lb/>cution consist of 1) selecting the right places to inject <lb/>exceptions 2) choosing the right point in time for injec-<lb/>tion and 3) throwing the appropriate kind of exceptions. <lb/>With an intuitive, then formal reasoning on the nature of <lb/>resilience and exceptions, we tackle those three challenges <lb/>and define two contracts on the programming language <lb/>construct " try-catch " that capture two facets of software <lb/>resilience against unanticipated exceptions. The satisfac-<lb/>tion or violation of those contracts is assessed using the <lb/>execution data collected during short-circuit testing. <lb/>Finally, we use the knowledge on resilience obtained <lb/>with short-circuit testing to replace the caught type of a <lb/>catch block by one of its super-type. This source code <lb/>transformation, called " catch stretching " is considered <lb/>correct if the test suite continues to pass. By enabling <lb/>catch blocks to correctly handle more types of exception <lb/>(w.r.t. the specification), the code is more capable of <lb/>handling unanticipated exceptions. <lb/>Our approach helps developers to be aware of what <lb/>part of their code is resilient, and to automatically <lb/>recommend modifications of catch blocks that im-<lb/>prove the software resilience. <lb/>Our technique is novel. There are techniques to pro-<lb/>vide information about the test suite with respect to ex-<lb/>ceptions or to improve the test suite ([5, 8, 10, 26]). Our <lb/>contribution is on analyzing and improving the applica-<lb/>tive code itself (the test suite is just a means). Other pa-<lb/>pers make static analyses of exception handling ([22, 23]). <lb/>Our contribution is a dynamic technique which uses a <lb/>new kind of fault injection. <lb/>We evaluate our approach by analyzing the resilience <lb/>of 9 well-tested open-source applications written in Java. <lb/>In this dataset, we analyze the resilience capabilities of <lb/>241 try-catch blocks and show that 92 of them satisfy <lb/>at least one resilience contract and 24 try-catch blocks <lb/>violate a resilience property. <lb/>To sum up, our contributions are: <lb/> • A definition and formalization of two contracts on <lb/>try-catch blocks, <lb/> • An algorithm and four predicates to verify whether <lb/>a try-catch satisfies those contracts, <lb/> • A source code transformation to improve the re-<lb/>silience against exceptions, <lb/> • An empirical evaluation on 9 open sources applica-<lb/>tions with one test suite each showing that there <lb/>exists resilient try-catch blocks in practice. <lb/> 2 Background <lb/> In our work, we use the distinction of Avizienis, Laprie <lb/>and Randell [2] between faults, errors and failures. How-<lb/>ever, we also consider the common usage, which con-<lb/>sists of " fault-injection " and " error-handling " whereas <lb/>it might sometimes be more appropriate to say " error-<lb/>injection " or " fault-handling " . In our paper, for sake of <lb/>understandability, we prefer the common usage and use <lb/>well-known expressions such as " fault-injection " or " fault <lb/>model " . <lb/> 2.1 Background on Exceptions <lb/> Exceptions are programming language constructs for <lb/>handling errors [11]. Exceptions can be thrown and <lb/>caught. When one throws an exception, this means that <lb/>something has gone wrong and this cancels the nominal <lb/>behavior of the application: the program will not follow <lb/>the normal control-flow and will not execute the next <lb/>instructions. Instead, the program will "jump" to the <lb/>nearest matching catch block. In the worst case, there is <lb/>no matching catch block in the stack and the exception <lb/>reaches the main entry point of the program and conse-<lb/>quently, stops its execution (i.e. crashes the program). <lb/>When an exception is thrown then caught, it is equiva-<lb/>lent to a direct jump from the throw location to the catch <lb/>location: in the execution state, only the call stack has <lb/>changed, but not the heap 1 . For a practical presentation <lb/>of exceptions in mainstream programming languages, we <lb/>refer to any introductory textbook, e.g. [20]. Avizienis <lb/>et al. [2] do not mention exceptions. We consider excep-<lb/>tions as errors and we use the term error in the paper as <lb/>much as possible. <lb/> 2.2 Definition of Resilience <lb/> We embrace the definition of " software resilience" by La-<lb/>prie as interpreted by Trivedi et al.: <lb/> Definition Resilience is " the persistence of service deliv-<lb/>ery that can justifiably be trusted, when facing changes " <lb/> [14]. " Changes may refer to unexpected failures, attacks <lb/>or accidents (e.g., disasters)" [25]. <lb/>
<note place="footnote"> 1 the heap may change if the programming language contains a <lb/>finalization mechanism (e.g. in Module-2+ [19]) <lb/></note>
<page> 2 <lb/></page>
Along with Trivedi et al., we interpret the idea of " un-<lb/>expected " events with the notion of " design envelope" <lb/>[25], a known term in safety critical system design. The <lb/>design envelope defines all the anticipated states of a <lb/>software system. It defines the boundary between antic-<lb/>ipated and unanticipated runtime states. The design en-<lb/>velope contains both correct states and incorrect states, <lb/>the latter resulting from the anticipation of misusages <lb/>and attacks. According to that, " resilience deals with <lb/>conditions that are outside the design envelope" [25]. <lb/>Along this line, we consider that the main dierence be-<lb/>tween software resilience and software robustness is that <lb/>software robustness deals with anticipated kinds of errors <lb/>(i.e. inside the " design envelope " ). <lb/>In this paper, we focus on the resilience in the context <lb/>of software that uses exceptions. We interpret and re-<lb/>fine this general definition in the context of mainstream <lb/>exception handling. <lb/> Definition Resilience against exceptions is the software <lb/>system's ability to reenter a correct state when an unan-<lb/>ticipated exception occurs. <lb/> 2.3 Specifications and Test Suites <lb/> A test suite is a collection of test cases where each test <lb/>case contains a set of assertions [4]. The assertions spec-<lb/>ify what the software is meant to do (i.e. it defines the <lb/>design envelope). Hence, in the rest of this paper, we <lb/>consider the test suites as specifications 2 . For instance, <lb/> " assert(3, division(15,5)) " specifies that the result of the <lb/>division of 15 by 5 should be 3. <lb/>A test suite may also encode what a software pack-<lb/>age does outside standard usage (error defined in the <lb/>design envelope). For instance, one may specify that <lb/> " division(15,0) " should throw an exception "Division by <lb/>zero not possible". Hence, the exceptions that are thrown <lb/>during test suite execution are the anticipated errors. If <lb/>an exception is triggered by the test and caught later on <lb/>in the application code, the assertions specify that the <lb/>exception-handling code has worked as expected. <lb/>Our definition of resilience relates to exceptions that <lb/>are not specified, that are outside the design envelope. <lb/>We consider test suites as approximation of the design <lb/>envelope. Consequently, in this paper, the considered <lb/>resilience consists in handling unanticipated exceptions <lb/>where we define unanticipated exceptions as exceptions <lb/>that are not triggered during test suite execution. <lb/>
<note place="footnote">2 Conversely, when we use the term " specification " , we refer to <lb/>the test suite (even if they are an approximation of an idealized <lb/>specification [24]) <lb/></note>
3 Automatic Analysis and In-<lb/>crease of Software Resilience <lb/> We define in Section 3.1 two exception contracts applica-<lb/>ble to try-catch blocks. We then describe an algorithm <lb/>(see Section 3.2) and formal predicates (see Section 3.3) <lb/>to verify those contracts according to a test suite. Finally <lb/>we present the concept of catch stretching, a technique <lb/>to improve the resilience of software applications against <lb/>exceptions (see Section 3.4). The insight behind our ap-<lb/>proach is that we can use test suites as an oracle for the <lb/>resilience capabilities against unanticipated errors. <lb/> 3.1 Definition of Two Contracts for Ex-<lb/>ception Handling <lb/> We now present two novel contracts for exception-<lb/>handling programming constructs. We use the term <lb/> " contract " in its generic acceptation: a property of a <lb/>piece of code that contributes to reuse, maintainabil-<lb/>ity, correctness or another quality attribute. For in-<lb/>stance, the " hashCode/equals " contract 3 is a property <lb/>on a pair of methods. Our definition is broader in scope <lb/>than Meyer's "contracts" [17] which refer to precondi-<lb/>tions, postconditions and invariants contracts. <lb/>We focus on contracts on the programming language <lb/>construct try and catch blocks, which we refer to as " try-<lb/>catch " . A try-catch is composed of one try block and <lb/>one catch block. Note that a try with multiple catch <lb/>blocks is considered as a set of pairs consisting of the <lb/>try block and one of its catch blocks. This means that <lb/>a try with n catch blocks is considered as n try-catch <lb/>blocks. This concept is generalized in most mainstream <lb/>languages, sometimes using dierent names (for instance, <lb/>a catch block is called an " except " block in Python). In <lb/>this paper, we ignore the concept of " finally " block [12] <lb/>which is more language specific and much less used in <lb/>practice [6]. <lb/> 3.1.1 Source Independence Contract <lb/>Motivation When an harmful exception occurs during <lb/>testing or production, a developer has two possibilities. <lb/>One way is to avoid the exception to be thrown by fixing <lb/>its root cause (e.g. by inserting a not null check to avoid <lb/>a null pointer exception). The other way is to write a <lb/>try block surrounding the code that throws the excep-<lb/>tion. The catch block ending the try block defines the <lb/>recovery mechanism to be applied when this exception <lb/>occurs. The catch block responsibility is to recover from <lb/>the particular encountered exception. By construction, <lb/>the same recovery would be applied if another exception <lb/>of the same type occurs within the scope of the try block <lb/>at a dierent location. <lb/>
<note place="footnote"> 3 http://docs.oracle.com/javase/7/docs/api/java/lang/ <lb/> Object.html#hashCode() <lb/></note>
<page> 3 <lb/></page>
Listing 1: An Example of Source-<lb/>Independent Try-Catch Block. <lb/> try{ <lb/> String arg = getArgument(); <lb/>String key = format(arg); <lb/>return getProperty(key , isCacheActivated <lb/>); <lb/>}catch(MissingPropertyException e){ <lb/>return "missing property"; <lb/>} <lb/> xx <lb/>Listing 2: An Example of Source-<lb/>Dependent Try-Catch Block. <lb/> boolean isCacheActivated = false; <lb/>try{ <lb/>isCacheActivated = getCacheAvailability <lb/>(); <lb/>return getProperty(key , <lb/>isCacheActivated); <lb/>}catch(MissingPropertyException e){ <lb/>if( isCacheActivated ){ <lb/>return "missing property"; <lb/>}else{ <lb/>throw new CacheDisableException(); <lb/>} } <lb/> xx <lb/>Listing 3: An Example of Purely-<lb/>Resilient Try-Catch Block. <lb/> try{ <lb/>return getPropertyFromCache(key); <lb/>}catch(MissingPropertyException e){ <lb/>return getPropertyFromFile(key); <lb/>} <lb/> This motivates the source-independence contract: the <lb/>normal recovery behavior of the catch block must work <lb/>for the foreseen exceptions; but beyond that, it should <lb/>also work for exceptions that have not been encountered <lb/>but may arise in a near future. <lb/>We define a novel exception contract that we called <lb/> " source-independence " as follows: <lb/> Definition A try-catch is source-independent if the <lb/>catch block proceeds equivalently, whatever the source <lb/>of the caught exception is in the try block. <lb/>For now, we loosely define " proceeds equivalently " : if <lb/>the system is still in error, it means that the error kind <lb/>is the same; if the system has recovered, it means that <lb/>the available functionalities are the same. <lb/>For example, Listing 1 shows a try-catch that satisfies <lb/> the source-independence contract. If a value is missing in <lb/>the application, an exception is thrown and the method <lb/>returns a default value " missing property". The code of <lb/>the catch block (only one return statement) clearly does <lb/>not depend on the application state. The exception can <lb/>be thrown by any of the 3 statements in the try, and the <lb/>result will still be the same. <lb/>On the contrary, Listing 2 shows a try-catch that vi-<lb/>olates the source-independence contract. Indeed, the <lb/>result of the catch process depends on the value of is-<lb/>CacheActivated. If the first statement fails (throws an <lb/>exception), the variable isCacheActivated is false, then <lb/>an exception is thrown. If the first statement passes but <lb/>the second one fails, isCacheActivated can be true, then <lb/>the value missing property is returned. The result of <lb/>the execution of the catch depends on the state of the <lb/>program when the catch begins (here it depends on the <lb/>value of the isCacheActivated variable). In case of fail-<lb/>ure, a developer cannot know if she will have to work <lb/>with a default return value or with an exception. This <lb/>catch is indeed source-dependent. <lb/>We will present a formal definition of this contract and <lb/>an algorithm to verify it in Section 3.3. We will show that <lb/>both source-independent and source-dependent catch <lb/>blocks exist in practice in Section 4. <lb/> Discussion How can it happen that developers write <lb/>source-dependent catch blocks? Developers discover <lb/>some exception risks at the first run-time occurrence of <lb/>an exception at a particular location. In this case, the <lb/>developer adds a try-catch block and puts the exception <lb/>raising code in the try body. Often, the try body con-<lb/>tains more code than the problematic statement in order <lb/>to avoid variable scope and initialization problems. How-<lb/>ever, while implementing the catch block, the developer <lb/>still assumes that the exception can only be thrown by <lb/>the problematic statement, and refers to variables that <lb/>were set in previous statements in the try block. In <lb/>other words, the catch block is dependent on the applica-<lb/>tion state at the problematic statement. If the exception <lb/>comes from the problematic statement, the catch block <lb/>works, if not, it fails to provide the expected recovery <lb/>behavior. <lb/>The source independence contract shows that if an <lb/>unanticipated exception happens the catch block would <lb/>still be able to recover the application state. This means <lb/>that the try-catch is able to handle unanticipated excep-<lb/>tions, hence it is resilient w.r.t. the chosen definition of <lb/>resilience (see Section 2.2). <lb/> 3.1.2 Pure Resilience Contract <lb/>Motivation In general, when an error occurs, it is more <lb/>desirable to recover from this error than to stop or crash. <lb/>A good recovery consists in returning the expected result <lb/>despite the error and in continuing the program execu-<lb/>tion. <lb/>One way to obtain the expected result under error is <lb/>to be able to do the same task in a way that, for the <lb/>same input, does not lead to an error but to the expected <lb/>result. Such an alternative is sometimes called " plan B " . <lb/>In terms of exception, recovering from an exception with <lb/>a plan B means that the corresponding catch contains the <lb/>code of this plan B. The plan B performed by the catch <lb/>is an alternative to the " plan A" which is implemented in <lb/>the try block. Hence, the contract of the try-catch block <lb/>(and not only the catch or only the try) is to correctly <lb/>perform a task T under consideration whether or not an <lb/>exception occurs. We refer to this contract as the " pure <lb/>resilience" contract. <lb/>A pure resilience contract applies to try-catch blocks. <lb/>We define it as follows: <lb/> Definition A try-catch is purely resilient if the system <lb/>
<page>4 <lb/></page>
state is equivalent at the end of the try-catch execution <lb/>whether or not an exception occurs in the try block. <lb/>By system state equivalence, we mean that the eects <lb/>of the plan A on the system are similar to those of plan B <lb/>from a given observation perspective. If the observation <lb/>perspective is a returned value, the value from plan A <lb/>is semantically equivalent to the value of plan B (e.g. <lb/>satisfies an " equals " predicate method in Java). <lb/>For example, Listing 3 shows a purely resilient try-<lb/>catch where a value is required, the program tries to <lb/>access this value in the cache. If the program does not <lb/>find this value, it retrieves it from a file. We will present <lb/>a formal definition of this contract and an algorithm to <lb/>verify it in Section 3.3. <lb/> Usage There are dierent use cases of purely resilient <lb/>try-catch blocks. We have presented the use case of <lb/>caching for pure resilience in Listing 3. One can use <lb/>purely resilient try-catch blocks for performance reasons: <lb/>a catch block can be a functionally equivalent yet slower <lb/>alternative. The ecient and more risky implementa-<lb/>tion of the try block is tried first, and in case of an ex-<lb/>ception, the catch block takes over to produce a correct <lb/>result. Optionality is another reason for pure resilience. <lb/>Whether or not an exception occurs during the execution <lb/>of an optional feature in a try block, the program state is <lb/>valid and allows the execution to proceed normally after <lb/>the execution of the try-block. <lb/> Discussion The dierence between source-<lb/>independence and pure resilience is as follows. Source-<lb/>independence means that under error the try-catch has <lb/>always the same observable behavior. In contrast, pure <lb/>resilience means that in nominal mode and under error <lb/> the try-catch block has always the same observable <lb/>behavior. This shows that pure resilience subsumes <lb/>source-independence: by construction, purely resilient <lb/>catch blocks are source-independent. The pure resilience <lb/>contract is a loose translation of the concept of recovery <lb/>block [13] in mainstream programming languages. A <lb/>purely resilient try-catch is able to handle unanticipated <lb/>exceptions in any situation while still performing the <lb/>expected operation. Hence it is resilient w.r.t. the <lb/>chosen definition of resilience (see Section 2.2). <lb/>Although the " pure resilience" contract is strong, we <lb/>will show in Section 4 that we observe purely resilient <lb/>try-catch blocks in reality, without any dedicated search: <lb/>the dataset under consideration has been set up inde-<lb/>pendently of this concern. The source independence and <lb/>pure resilience contracts are not meant to be mandatory. <lb/>The try-catch blocks can satisfy one, both, or none. We <lb/>only argue that satisfying them is better from the view-<lb/>point of resilience, according to our technical definition <lb/>of resilience given in Section 2.2. <lb/> 3.2 The Short-circuit Testing Algorithm <lb/> We now present a technique, called " short-circuit test-<lb/>ing " , which allows one to find source-independent and <lb/>purely-resilient try-catch blocks. <lb/> Definition Short-circuit testing consists of dynamically <lb/>injecting exceptions during the test suite execution in <lb/>order to analyze the resilience of try-catch blocks. <lb/>The system under test is instrumented so that the ef-<lb/>fects of injected exceptions are logged. This data is next <lb/>analyzed to verify whether a try-catch block satisfies or <lb/>violates the two contracts aforementioned. The injected <lb/>exceptions represent unanticipated situations. <lb/>According to our definition of resilience and test suites <lb/>as specification, everything that is in the test suite is an-<lb/>ticipated. To identify unanticipated exceptions, we need <lb/>to artificially create runtime cases that are not in the <lb/>test suite. This is exactly the key point of short-circuit <lb/>testing: it complements the test suite with unanticipated <lb/>scenarios. Short-circuit testing allows us to study the re-<lb/>silience of try-catch blocks in unanticipated scenarios. <lb/>We call this technique " short-circuit testing " because <lb/>it ressembles electrical short-circuits: when an exception <lb/>is injected, the code of the try block is somehow short-<lb/>circuited. The name of software short-circuit is also used <lb/>in the Hystrix resilience library 4 . <lb/>What can we say about a try-catch when a test passes <lb/>while injecting an exception in it? We use the test suite <lb/>as an oracle of execution correctness: if a test case passes <lb/>under injection, the new behavior triggered by the in-<lb/>jected exception is in accordance with the specification. <lb/>Otherwise, if the test case fails, the new behavior is de-<lb/>tected as incorrect by the test suite. <lb/> 3.2.1 Algorithm <lb/> Our algorithm for short-circuit testing is given in Fig-<lb/>ure 1. Our algorithm needs an application A and its test <lb/>suite T S. <lb/> First, a static analysis extracts the list of existing try-<lb/>catch blocks. For instance, the system extracts that <lb/>method foo() contains one try with two associated catch <lb/>blocks: they form two try-catch blocks (see Section 3.1). <lb/>In addition, we also need to know which test cases specify <lb/>which try-catch blocks, i.e. the correspondence between <lb/>test cases and try-catch blocks: a test case is said to <lb/>specify a try-catch if it uses it. To perform this, the <lb/>algorithm collects data about the standard run of the <lb/>test suite under consideration. For instance, the system <lb/>learns that the try block in method foo() is executed on <lb/>the execution of test #1 and #5. The standard run of <lb/>short-circuit testing also collects fine-grain data about <lb/>the occurrences of exceptions in try-catch blocks during <lb/>the test suite execution (see Section 3.3.1). <lb/>
<note place="footnote"> 4 see https://github.com/Netflix/Hystrix <lb/></note>
<page> 5 <lb/></page>
Input: An Application A, a test suite T S specifying the behavior of A. <lb/> Output: a matrix M (try-catch times test cases, the cells represent test success or failure. <lb/> begin <lb/> try_catch_list Ω static_analysis(A) <lb/> Û retrieve all the try-catch of the application <lb/> standard_behavior Ω standard_run(T S) <lb/> Û get test colors and try-catch behaviors <lb/> for t oe try_catch_list <lb/> Û For each try-catch tc in the application <lb/> do <lb/> prepare_injection(tc) <lb/> Û prepare the try-catch tc by setting an injector <lb/> Û which will throw an exception of the type caught by tc <lb/> Û at the beginning of each execution of the try tc <lb/>as Ω get_test_using(tc, standard_behavior) <lb/> Û retrieve all tests in the test suite T S <lb/> Û using the try of the try-catch tc <lb/> for a oe as <lb/> Û For all test a which use the current try <lb/> do <lb/> pass Ω run_test_with_injection(a) <lb/> Û get the result of the test under injection <lb/> M [tc, a] = pass <lb/> Û store the result of the test a under injection in tc <lb/> return M <lb/> Figure 1: The Short-Circuit Testing Algorithm. Exception injection is used to collect data about the behavior <lb/>of catch blocks. <lb/>Then, the algorithm loops over the try-catch pairs (re-<lb/>call that a try with n catch blocks is split into n concep-<lb/>tual pairs of try/catch). For each try-catch pair, the set <lb/>of test cases using t, called as, is extracted in the moni-<lb/>toring data of the standard run. The algorithm executes <lb/>each one of these tests while injecting an exception at <lb/>the beginning of the try under analysis. This simulates <lb/>the worst-case exception, worst-case in the sense that it <lb/>discards the whole code of the try block. The type of the <lb/>injected exception is the statically declared type of the <lb/>catch block. For instance, if the catch block catches " Ar-<lb/>rayIndexOutOfBoundsException " , an instance of " Ar-<lb/>rayIndexOutOfBoundsException " is thrown. <lb/>Consequently, if the number of catch blocks corre-<lb/>sponding to the executed try block is N, there is one <lb/>static analysis, one full run of the test suite and N runs <lb/>of as. In our example, the system runs its analysis, and <lb/>executes the full test suite once. Then it runs tests #1 <lb/>and #5 with fault injection twice. The first time the in-<lb/>jected exception goes in the first catch block, the second <lb/>time, it goes, thanks to typing, in the second catch block. <lb/> 3.2.2 Output of Short-circuit Testing <lb/> The output of our short-circuit testing algorithm is a ma-<lb/>trix M which represents the result of each test case under <lb/>injection (for each try-catch). M is a matrix of boolean <lb/>values where each row represents a try-catch block, and <lb/>each column represents a test case. A cell in the matrix <lb/>indicates whether the test case passes with exception in-<lb/>jection in the corresponding try-catch. This matrix is <lb/>used to evaluate the exception contract predicates de-<lb/>scribed next in Section 3.3. <lb/>Short-circuit testing is performed with source code <lb/>transformations. Monitoring and fault injection code is <lb/>added to the application under analysis. Listing 4 illus-<lb/>trates how this is implemented. The injected code is able <lb/>to throw an exception in a context dependent manner. <lb/>The injector is driven by an exception injection controller <lb/>at runtime. <lb/> 3.3 Resilience Predicates <lb/> We now describe four predicates that are evaluated on <lb/>each row of the matrix to assess whether: the try-catch is <lb/>source-independent (contract satisfaction), the try-catch <lb/>is source-dependent (contract violation), the try-catch is <lb/>purely-resilient (contract satisfaction), the try-catch is <lb/>not purely-resilient (contract violation). <lb/>As hinted here, there is no one single predicate p for <lb/>which contract[x] = p[x] and ¬contract[x] = ¬p[x]. For <lb/>both contracts, there are some cases where the short-<lb/>circuit testing procedure yields not enough data to decide <lb/>whether the contract is satisfied or violated. The princi-<lb/>ple of the excluded third (principium tertii exclusi) does <lb/>not apply in our case. <lb/> 3.3.1 Definition of Try-catch Usages <lb/> Our exception contracts are defined on top of the notion <lb/>of " try-catch usages". A try-catch usage refers to the <lb/>execution behavior of try-catch blocks with respect to <lb/>exceptions. We define three kinds of try-catch usages as <lb/>follows: (1) No exception is thrown during the execution <lb/>of the try block (called " pink usage") and the test case <lb/>passes; (2) An exception is thrown during the execution <lb/>of the try block and this exception is caught by the catch <lb/>(called " white usage") and the test case passes; (3) An <lb/>
<page>6 <lb/></page>
exception is thrown during the execution of the try block <lb/>but this exception is not caught by the catch (called " blue <lb/>usage" – this exception may be caught later or expected <lb/>by the test case if it specifies an error case). <lb/>In these usages, the principle of the excluded third <lb/>(principium tertii exclusi) applies: a try-catch usage is <lb/>either pink or white or blue. Note that a single try-catch <lb/>can be executed multiple times, with dierent try-catch <lb/>usages, even in one single test. This information is used <lb/>later in this Section to verify the contracts. <lb/> 3.3.2 Source Independence Predicate <lb/> The decision problem is formulated as: given a try-catch <lb/>and a test suite, does the source-independence contract <lb/>hold? The decision procedure relies on two predicates. <lb/> Predicate #1 (source_independent[x]): Satisfac-<lb/>tion of the source independence contract: A try-<lb/>catch x is source independent if and only if for all <lb/>test cases that execute the corresponding catch block <lb/>(white_usage), it still passes when one throws an ex-<lb/>ception at the worst-case location in the corresponding <lb/>try block. <lb/>Formally, this reads as: <lb/> source_independent[x] = 'a oe A x |'u a oe usages_in(x, a)| <lb/> (is_white_usage[u a ] =∆ pass_with_injection[a, x]) <lb/> (1) <lb/>In this formula, x refers to a try-catch (a try and its <lb/>corresponding catch block), A x is the set of all tests ex-<lb/>ecuting x (passing in the try block), u is a try-catch us-<lb/>age, i.e. a particular execution of a given try-catch block, <lb/> usages_in(x, a) returns the runtime usages of try-catch <lb/> x in the test case a, is_white_usage[u] evaluates to true <lb/>if and only if an exception is thrown in the try block and <lb/>the catch intercepts it, pass_with_injection evaluates <lb/>to true if and only if the test case t passes with exception <lb/>injection in try-catch x. <lb/> Predicate #2 (source_dependent[x]):Violation of <lb/>the source independence contract: A try-catch x <lb/> is not source independent if there exists a test case that <lb/>executes the catch block (white_usage) which fails when <lb/>one throws an exception at a particular location in the <lb/>try block. <lb/>This is translated as: <lb/> source_dependent[x] = ÷a oe A x |'u a oe usages_in(x, a)| <lb/> (is_white_usage[u a ] · ¬pass_with_injection[a, x]) <lb/> (2) <lb/> Pathological <lb/>cases: <lb/> By <lb/>construction <lb/> source_dependent[x] <lb/> and <lb/> source_independent[x] <lb/> cannot be evaluated to true at the same time (the <lb/>decision procedure is sound). If source_ <lb/>dependent[x] and source_independent[x] are both <lb/>evaluated to false, it means that the procedure yields <lb/>not enough data to decide whether the contract is <lb/>satisfied or violated. <lb/> 1 <lb/> try{ <lb/> 2 <lb/> // injected code <lb/> 3 <lb/> if(Controller.isCurrentTryCatchWithInjection()) <lb/> 4 <lb/> if(Controller.currentInjectedExceptionType() == <lb/>Type01Exception.class ){ <lb/> 5 <lb/> throw new Type01Exception(); <lb/> 6 <lb/> }else if(Controller.currentInjectedExceptionType() == <lb/>Type02Exception.class ){ <lb/> 7 <lb/> throw new Type02Exception(); <lb/> 8 <lb/> } <lb/> 9 <lb/>10 <lb/> ... //normal try body <lb/> 11 <lb/> ... <lb/> 12 <lb/> } catch (Type01Exception t1e) { <lb/> 13 <lb/> ... //normal catch body <lb/> 14 <lb/> } catch (Type02Exception t2e) { <lb/> 15 <lb/> ... //normal catch body <lb/> 16 <lb/> } <lb/> Listing 4: Short-circuit testing is performed with source <lb/>code injection. The injected code is able to throw an <lb/>exception in a context dependent manner. The injector <lb/>can be driven at runtime. <lb/> 3.3.3 Pure Resilience Predicate <lb/> The decision problem is formulated as: given a try-catch <lb/>and a test suite, does the pure-resilience contract hold? <lb/>The decision procedure relies on two predicates. <lb/> Predicate #3 (resilient[x]): Satisfaction of the <lb/>pure resilience contract A try-catch x is purely re-<lb/>silient if it is covered by at least one pink usage and <lb/>all test cases that executes the try block pass when one <lb/>throws an exception at the worst-case location in the <lb/>corresponding try block. In other words, this predicate <lb/>holds when all tests pass even if one completely discards <lb/>the execution of the try block. <lb/>Loosely speaking, a purely resilient catch block is a <lb/> " perfect plan B". <lb/>This is translated as: <lb/> resilient[x] = ('a oe A x |pass_with_injection[a, x]) <lb/> ·(÷a oe A x |÷u a oe usages_in(x, a)|is_pink_usage[u a ]) <lb/> (3) <lb/>where is_pink_usage[u] evaluates to true if and only <lb/>if no exception is thrown in the try block. <lb/> Predicate #4 (not_resilient[x]): Violation of the <lb/>pure resilience contract A try-catch x is not purely <lb/>resilient if there exists a failing test case when one throws <lb/>the exception at a particular location in the correspond-<lb/>ing try block. <lb/>This predicate reads as: <lb/> not_resilient[x] = ÷a oe A c |¬pass_with_injection[a, x] <lb/> (4) <lb/> Pathological cases By construction resilient[x] and <lb/> not_ <lb/>resilient[x] cannot be evaluated to true at the same time <lb/>(the decision procedure is sound). Once again, if they <lb/>are both evaluated to false, it means that the procedure <lb/>yields not enough data to decide whether the contract is <lb/>satisfied or violated. <lb/>
<page>7 <lb/></page>
# <lb/> ex-<lb/>ecuted <lb/>try-catch <lb/># purely <lb/>resilient <lb/>try-catch <lb/># source-<lb/>independent <lb/>try-catch <lb/># source-<lb/>dependent <lb/>try-catch <lb/>Unknown <lb/>w.r.t. <lb/>resilience <lb/>Unknown <lb/>w.r.t. <lb/>source <lb/>indepen-<lb/>dence <lb/># <lb/>Stretch-<lb/>able <lb/>try-catch <lb/>commons-lang <lb/>49 <lb/>3/49 <lb/>18/49 <lb/>5/49 <lb/>1/49 <lb/>26/49 <lb/>16/18 <lb/>commons-codec <lb/>14 <lb/>0/14 <lb/>12/14 <lb/>0/14 <lb/>0/14 <lb/>2/14 <lb/>12/12 <lb/>joda time <lb/>18 <lb/>0/18 <lb/>4/18 <lb/>0/18 <lb/>2/18 <lb/>14/18 <lb/>4/4 <lb/>spojo core <lb/>1 <lb/>1/1 <lb/>1/1 <lb/>0/1 <lb/>0/1 <lb/>0/1 <lb/>1/1 <lb/>sonar core <lb/>10 <lb/>0/10 <lb/>9/10 <lb/>1/10 <lb/>1/10 <lb/>0/10 <lb/>7/9 <lb/>sonar plugin <lb/>6 <lb/>0/6 <lb/>3/6 <lb/>0/6 <lb/>0/6 <lb/>3/6 <lb/>3/3 <lb/>jbehave core <lb/>42 <lb/>2/42 <lb/>7/42 <lb/>2/42 <lb/>9/42 <lb/>33/42 <lb/>7/7 <lb/>shindig-java-gadgets 80 <lb/>2/80 <lb/>30/80 <lb/>12/80 <lb/>21/80 <lb/>38/80 <lb/>26/30 <lb/>shindig-common <lb/>21 <lb/>1/21 <lb/>8/21 <lb/>4/21 <lb/>4/21 <lb/>9/21 <lb/>8/8 <lb/>total <lb/>241 <lb/>9 <lb/>92 <lb/>24 <lb/>38 <lb/>125 <lb/>84/92 <lb/>Table 1: The number of source independent , purely resilient and stretchable catch blocks found with short-circuit <lb/>testing. Our approach provides developers with new insights on the resilience of their software. <lb/> 3.4 Improving Software Resilience with <lb/>Catch Stretching <lb/> We have defined two formal criteria of software resilience <lb/>and an algorithm to verify them (Section 3.3). How to <lb/>put this knowledge in action? <lb/>For both contracts, one can improve the test suite it-<lb/>self. As discussed above, some catch blocks are never <lb/>executed and others are not suciently executed to be <lb/>able to infer their resilience properties (the pathological <lb/>cases of Section 3.3). The automated refactoring of the <lb/>test suite is outside the scope of this paper. <lb/> 3.4.1 Definition of Catch Stretching <lb/> We now aim at improving the resilience against unantici-<lb/>pated exceptions, those exceptions that are not specified <lb/>in the test suite and even not foreseen by the developers. <lb/>According to our definition of resilience, this means im-<lb/>proving the capability of the software under analysis to <lb/>correctly handle unanticipated exceptions. <lb/>One solution to force the contract satisfaction is to <lb/>reduce the size of try blocks, so that they only contain <lb/>statements involving exceptions during test suite execu-<lb/>tion. This is a trivial solution, it does not improve re-<lb/>silience. It does actually the opposite: less exceptions <lb/>can be caught. <lb/>The other solution is to transform the catch blocks <lb/>so that they catch more exceptions than before. This <lb/>is what we call " catch stretching " : replacing the <lb/>type of the caught exceptions. For instance, replac-<lb/>ing catch(FileNotFoundException e) by catch(IO-<lb/>Exception e). The extreme of catch stretching is to <lb/>parametrize the catch with the most generic type of ex-<lb/>ceptions (e.g. Throwable in Java, Exception in .NET). <lb/>In other words, In situations where unanticipated ex-<lb/>ceptions appear, the control flow would be broken, and <lb/>the program would likely crash. This is what we consider <lb/>as " incorrect " . After catch stretching, the program would <lb/>not crash, and this is what we consider as " better " . <lb/>Let us now examine why this simple transformation <lb/>catch is meaningful with respect to unanticipated excep-<lb/>tions. <lb/>We claim that all source-independent catch blocks are <lb/>candidates to be stretched. This encompasses purely-<lb/>resilient try-catch blocks as well since by construction <lb/>they are also source-independent (see Section 3.1). The <lb/>reasoning is as follows. <lb/> 3.4.2 Catch Stretching Under Short-Circuit <lb/>Testing <lb/> By stretching source independent catch-blocks, the re-<lb/>sult is equivalent under short-circuit testing. In the <lb/>original case, injected exceptions are of type X and <lb/>caught by catch(X e). In the stretched case, injected <lb/>exceptions are of generic type Exception and caught by <lb/> catch(Exception e). In both cases, the input state of <lb/>the try block is the same (as set by the test case), and <lb/>the input state of the catch block is the same (since no <lb/>code has been executed in the try block due to fault in-<lb/>jection). Consequently, the output state of the try-catch <lb/>is exactly the same. Under short-circuit testing, catch <lb/>stretching yields strictly equivalent results. <lb/> 3.4.3 Catch Stretching and Test Suite Specifica-<lb/>tion <lb/> Let us now consider a standard run of the test suite and <lb/>a source-independent try-catch. In standard mode, with <lb/>the original code, there are two cases: either all the ex-<lb/>ceptions thrown in the try block under consideration are <lb/>caught by the catch (case A), or at least one exception <lb/>traverses the try block without being caught because it <lb/>is of an uncaught type (case B). In both cases, we refer <lb/>to exceptions normally triggered by the test suite, not <lb/>injected ones. <lb/>
<page>8 <lb/></page>
In the first case, catch stretching does not change the <lb/>behavior of the application under test: all exceptions <lb/>that were caught in this catch block in the original ver-<lb/>sion are still caught in the stretched catch block. In other <lb/>words, the stretched catch is still correct according to the <lb/>specification. And it is able to catch many more unan-<lb/>ticipated exceptions: it corresponds to our definition of <lb/>resilience. On those source-independent try-catch of case <lb/>(A), catch stretching improves the resilience of the appli-<lb/>cation. <lb/> We now study the second case (case B): there is at <lb/>least one test case in which the try-catch x under anal-<lb/>ysis is traversed by an uncaught exception. There are <lb/>again two possibilities: this uncaught exception bubbles <lb/>to the test case, which expects the exception (it speci-<lb/>fies that an exception must be thrown and asserts that <lb/>it is actually thrown). If this happens, we don't apply <lb/>catch stretching. Indeed, it is specified that the excep-<lb/>tion must bubble, and to respect the specifications we <lb/>must not modify the try-catch behavior. The other pos-<lb/>sibility is that the uncaught exception in try-catch x is <lb/>caught by another try-catch block y later in the stack. <lb/>When stretching try-catch x, one replaces the recovery <lb/>code executed by try-catch y by executing the recovery <lb/>code of try-catch x. However, it may happen that the <lb/>recovery code of x is dierent from the recovery code of <lb/> y, and that consequently, the test case that was passing <lb/>with the execution of the catch of y (the original mode) <lb/>fails with the execution of the catch x. <lb/> To overcome this issue, we propose to again use the test <lb/>suite as the correctness oracle. For source-independent <lb/>try-catch blocks of case B, one stretches the catch to " Ex-<lb/>ception " , one then runs the test suite, and if all tests still <lb/>pass, we keep the stretched version. As for case A, the <lb/>stretching enables to handle more unanticipated excep-<lb/>tions while remaining correct with respect to the speci-<lb/>fication. Stretching source-independent try-catch blocks <lb/>of both case A and case B improves the resilience. <lb/> 3.4.4 Summary <lb/> To sum up, improving software resilience with catch <lb/>stretching consists of: First, stretching all source-<lb/>independent try-catch blocks of case A. Second, for each <lb/>source-independent try-catch blocks of case B, running <lb/>the test suite after stretching to check that the transfor-<lb/>mation has produced correct code according to the spec-<lb/>ification. Third, running the test suite with all stretched <lb/>catch blocks to check whether there is no strange inter-<lb/>play between all exceptions. <lb/>We will show in Section 4.3 that most (91%) of source-<lb/>independent try-catch blocks can be safely stretched ac-<lb/>cording to the specification. <lb/> 4 Empirical Evaluation <lb/> We have presented two exception contracts: pure re-<lb/>silience and source independence (Section 3.1). We now <lb/>evaluate those contracts from an empirical point of view. <lb/>Can we find real world try-catch blocks for which the <lb/>corresponding test suite enables us to prove their source <lb/>independence? Their pure resilience capability? Or to <lb/>prove that they violate of those exception contracts. <lb/> 4.1 Dataset <lb/> In this paper, we analyze the specification of error-<lb/>handling in the test suites of 9 Java open-source projects: <lb/>Apache commons-lang, Apache commons-code, joda-<lb/>time, Spojo core, Sonar core, Sonar Plugin, JBehave <lb/>Core, Shindig Java Gadgets and Shinding Common. The <lb/>selection criteria are as follows. First, the test suite has <lb/>to be in the top 50 of most tested exceptions accord-<lb/>ing to the SonarSource Nemo ranking 5 . SonarSource <lb/>is the organization behind the software quality assess-<lb/>ment platform " Sonar " . The Nemo platform is their show <lb/>case, where open-source software is continuously tested <lb/>and analyzed. Second, the test suite has to be runnable <lb/>within low overhead in terms of dependencies and exe-<lb/>cution requirements. <lb/>The line coverage of the test suites under study has a <lb/>median of 81%, a minimum of 50% and a maximum of <lb/>94%. This dataset contains a total of 767 catch blocks. <lb/> 4.2 Relevance of Contracts <lb/> The experimental protocol is as follows. We run the <lb/>short-circuit testing algorithm described in Section 3.2 <lb/>on the 9 reference test suites described before. As seen <lb/>in Section 3.2, short-circuit testing runs the test suite n <lb/> times, where n is the number of executed catch blocks <lb/>in the application. In total, we have thus 241 executions <lb/>over the 9 test suites of our dataset. <lb/>Table 1 presents the results of this experiment. For <lb/>each project of the dataset and its associated test suite, it <lb/>gives the number of executed catch blocks during the test <lb/>suite execution, purely resilient try-catch blocks, source-<lb/>independent try-catch blocks, and the number of try-<lb/>catch blocks for which runtime information is not suf-<lb/>ficient to assess the truthfulness of our two exception <lb/>contracts. <lb/> 4.2.1 Source Independence <lb/> Our approach is able to demonstrate that 92 try-catch <lb/>blocks (sum of the fourth column of Table 1) are source-<lb/>independent (to the extent of the testing data). This is <lb/>worth noticing that with no explicit ways for specifying <lb/>them and no tool support for verifying them, some de-<lb/>velopers still write catch blocks satisfying this contract. <lb/>
<note place="footnote"> 5 See http://nemo.sonarsource.org <lb/></note>
<page> 9 <lb/></page>
This shows that our contracts are not purely theoretical: <lb/>they reflect properties of error-handling code that can be <lb/>found in real software. <lb/>Beyond this, the developers not only write some <lb/>source-independent catch blocks, they also write test <lb/>suites that provide enough information to decide <lb/>with short-circuit testing whether the catch is source-<lb/>independent or not. <lb/>Our approach also identifies 24 try-catch blocks that <lb/>are source-dependent, i.e. that violate the source-<lb/>independence predicate. Our approach makes the de-<lb/>velopers aware that some catch blocks are not indepen-<lb/>dent of the source of the exception: the catch block im-<lb/>plicitly assumes a state resulting from the execution of <lb/>several statements at the beginning of the try. Within <lb/>the development process, this is a warning. The devel-<lb/>opers can then fix the try or the catch block if they <lb/>think that this catch block should be source indepen-<lb/>dent or choose to keep them source-dependent, in total <lb/>awareness. It is out of the scope of this paper to auto-<lb/>matically refactor source-dependent try-catch blocks as <lb/>source-independent. <lb/>For instance, a source-dependent catch block of the <lb/>test suite of sonar-core is shown in Listing 5. Here the <lb/>"key" statement is the if (started == false) (line 6). In-<lb/>deed, if the call to super.start() throws an exception be-<lb/>fore the variable started is set to true (started = true <lb/> line 15), an exception is thrown (line 7). On the con-<lb/>trary, if the same DatabaseException occurs after line <lb/>15, the catch block applies some recovery by setting de-<lb/>fault value (setEntityManagerFactory). Often, source-<lb/>dependent catch blocks contain if/then constructs. To <lb/>sum-up, short-circuit testing catches assumptions made <lb/>by the developers, and uncover causality eects between <lb/>the code executed within the try block and the code of <lb/>the catch block. <lb/>Finally, our approach highlights that, for 24 catch <lb/>blocks (fifth column of Table 1), there are not enough <lb/>tests to decide whether the source-independence con-<lb/>tract holds. This also increases the developer aware-<lb/>ness. This signals to the developers that the test suite is <lb/>not good enough with respect to assessing this contract. <lb/>This knowledge is directly actionable: for assessing the <lb/>contracts, the developer has to write new tests or refac-<lb/>tor existing ones. In particular, as discussed above, if <lb/>the same test case executes several times the same catch <lb/>block, this may introduce noise to validate the contract <lb/>or to prove its violation. In this case, the refactoring <lb/>consists of splitting the test case so that the try/catch <lb/>block under investigation is executed only once. <lb/> 4.2.2 Pure Resilience <lb/> We now examine the pure-resilience contracts. In our <lb/>experiment, we have found 9 purely resilient try-catch <lb/>blocks in our dataset. The distribution by application is <lb/>shown in the third column of Table 1. <lb/> 1 <lb/> public class MemoryDatabaseColector extends <lb/>AbstractDatabaseColector { <lb/> 2 <lb/> public void start(){ <lb/> 3 <lb/> try{ <lb/> 4 <lb/> super.start(); // code below <lb/> 5 <lb/> }catch (DatabaseException ex) { <lb/> 6 <lb/> if (started==false) // this is the source≠dependence <lb/> 7 <lb/> throw ex; <lb/> 8 <lb/> setEntityManagerFactory(); <lb/> 9 <lb/> }}} <lb/> 10 <lb/>11 <lb/> public void start(){ <lb/> 12 <lb/> ... <lb/> 13 <lb/> // depending on the execution of the following statement <lb/> 14 <lb/> // the catch block of the caller has a dierent behavior <lb/> 15 <lb/> started = true; <lb/> 16 <lb/> ...}} <lb/> Listing 5: A Source-Dependent Try-Catch Found in <lb/>Sonar-core using Short Circuit Testing. <lb/>Listing 6 shows a purely resilient try-catch block found <lb/>in project spojo-core using short-circuit testing. The <lb/>code has been slightly modified for sake of readabil-<lb/>ity. The task of the try-catch block is to return an <lb/>instantiable Collection class which is compatible with <lb/>the class of a prototype object. The plan A con-<lb/>sists of checking that the class of the prototype ob-<lb/>ject has an accessible constructor (simply by calling <lb/> getDeclaredConstructor). If there is no such construc-<lb/>tor, the method call throws an exception. In this case, <lb/>the catch block comes to the rescue and chooses from a <lb/>list of known instantiable collection classes one that is <lb/>compatible with the type of the prototype object. Ac-<lb/>cording to the test suite, the try-catch is purely resilient: <lb/>always executing plan B yields passing test cases. <lb/>The pure resilience is much stronger than the source <lb/>independence contract. While the former states that <lb/>the catch has the same behavior wherever the excep-<lb/>tion comes from, the latter states that the correctness <lb/>as specified by the test suite is not impacted in presence <lb/>of unanticipated exceptions. Consequently, it is normal <lb/>to observe far fewer try-catch blocks verifying the pure <lb/>resilience contract compared to the source-independent <lb/>contract. Despite the strength of the contract, this con-<lb/>tract also covers a reality: perfect alternatives, ideal <lb/>plans B exist in real code. <lb/>One also sees that there are some try-catch blocks <lb/>for which there is not enough execution data to assess <lb/>whether they are purely resilient or not. This happens <lb/>when a try-catch is only executed in white try-catch us-<lb/>ages and in no pink try-catch usage. By short-circuiting <lb/>the white try-catch usages (those with internally caught <lb/>exceptions), one proves it source-independence, but we <lb/>also need to short-circuit a nominal pink usage of this <lb/>try-catch to assess that plan B (of the catch block) works <lb/>instead of plan A (of the try block). This fact is surpris-<lb/>ing: this shows that some try-catch blocks are only spec-<lb/>ified in error mode (where exceptions are thrown) and <lb/>not in nominal mode (with the try completing with no <lb/>thrown exception). This also increases the awareness of <lb/>
<page>10 <lb/></page>
1 <lb/> // task of try≠catch: <lb/> 2 <lb/> // given a prototype object <lb/> 3 <lb/> Class clazz = prototype.getClass(); <lb/> 4 <lb/> // return a Collection class that has an accessible constructor <lb/> 5 <lb/> // which is compatible with the prototypes class <lb/> 6 <lb/> try { <lb/> 7 <lb/> // plan A: returns the prototypes class if a constructor exists <lb/> 8 <lb/> prototype.getDeclaredConstructor(); <lb/> 9 <lb/> return clazz; <lb/> 10 <lb/> } catch (NoSuchMethodException e) { <lb/> 11 <lb/> // plan B: returns a known instantiable collection <lb/> 12 <lb/> // which is compatible with the prototypes class <lb/> 13 <lb/> if (LinkedList.class.isAssignableFrom(clazz)) { <lb/> 14 <lb/> return LinkedList.class; <lb/> 15 <lb/> } else if (List.class.isAssignableFrom(clazz)) { <lb/> 16 <lb/> return ArrayList.class; <lb/> 17 <lb/> } else if (SortedSet.class.isAssignableFrom(clazz)) { <lb/> 18 <lb/> return TreeSet.class; <lb/> 19 <lb/> } else { <lb/> 20 <lb/> return LinkedHashSet.class; <lb/> 21 <lb/> } <lb/> 22 <lb/> } <lb/> Listing 6: A Purely-Resilient Try-Catch Found in spojo-<lb/>core (see SpojoUtils.java) <lb/>the developers: for those catch blocks, test cases should <lb/>be written to specify the nominal usage. <lb/> 4.3 Catch Stretching <lb/> We look at whether, among the 92 source-independent <lb/>try-catch blocks of our dataset, we can find stretchable <lb/>ones (stretchable in the sense of Section 3.4, i.e. for which <lb/>the caught exception can be set to " Exception " ). We use <lb/>source code transformation and the algorithm described <lb/>in Section 3.4. <lb/>The last column of Table 1 gives the number of <lb/>stretchable try-catch blocks out of the number of source-<lb/>independent try-catch blocks. For instance, in commons-<lb/>lang, we have found 18 candidates source-independent <lb/>try-catch blocks. Sixteen (16/18) of them can be safely <lb/>stretched: all test cases pass after stretching. <lb/>Table 1 indicates two results. First, most (91%) of the <lb/>source-independent try-catch blocks can be stretched to <lb/>catch all exceptions. In this case, the resulting trans-<lb/>formed code is able to catch more unanticipated excep-<lb/>tions while remaining correct with respect to the speci-<lb/>fication. <lb/>Second, there are also try-catch blocks for which catch <lb/>stretching does not work. As explained in Section 3.4, <lb/>this corresponds to the the case where the stretching re-<lb/>sults in hiding correct recovery code (w.r.t. to the speci-<lb/>fication), with new one (the code of the stretched catch) <lb/>that proves unable to recover from a traversing excep-<lb/>tion. <lb/>In our dataset, we encounter all cases discussed in <lb/>Section 3.4. For instance in joda-time, all four source-<lb/>independent try-catch blocks represent are never tra-<lb/>versed by an exception – case A of Section 3.4.3. (for <lb/>instance the one at line 560 of class ZoneInfoCompiler). <lb/>We have shown that analytically, they can safely be <lb/>stretched. We have run the test suite after stretching, <lb/>all tests pass. <lb/>We have observed the two variations of case B (try-<lb/>catch blocks traversed by exceptions in the original code). <lb/>For instance, in sonar-core, by stretching a NonUni-<lb/>queResultException catch to the most generic exception <lb/>type, an IllegalStateException is caught. However, this <lb/>is an incorrect transformation that results in one failing <lb/>test case. <lb/>Finally, we discuss the last and most interest-<lb/>ing case. <lb/>In commons-lang, the try-catch at line <lb/>826 of class ClassUtils can only catch a ClassNot-<lb/>FoundException but is traversed by a NullPointerEx-<lb/>ception during the execution of the test ClassU-<lb/>tilsTest.testGetClassInvalidArguments. By stretching <lb/>ClassNotFoundException to the most generic exception <lb/>type, the NullPointerException is caught: the catch <lb/>block execution replaces another catch block upper in the <lb/>stack. Although the stretching modifies the test case exe-<lb/>cution, the test suite passes, this means that the stretch-<lb/>ing is correct with respect to the test suite. <lb/> 4.4 Summary <lb/> To sum up, this empirical evaluation has shown that the <lb/>short-circuit testing approach of exception contracts en-<lb/>ables to increase the knowledge one has on a piece of soft-<lb/>ware. First, it indicates source-independent and purely <lb/>resilient try-catch blocks. This knowledge is actionable: <lb/>those catch blocks can be safely stretched to catch any <lb/>type of exceptions. Second, it indicates source-dependent <lb/>try-catch blocks. This knowledge is actionable: it says <lb/>that the error-handling should be refactored so as to <lb/>resist to unanticipated errors. Third, it indicates " un-<lb/>known " try-catch blocks. This knowledge is actionable: <lb/>it says that the test suite should be extended and/or <lb/>refactored to support automated analysis of exception-<lb/>handling. <lb/> 5 Discussion <lb/> There are two important points to be discussed with re-<lb/>spect to the short circuit testing algorithm: where the <lb/>exception is injected and the special case of statically <lb/>verified checked exceptions in Java (and other languages <lb/>such as .NET). We discuss at the end of the section the <lb/>threats to the validity of our empirical results. <lb/> 5.1 Injection Location <lb/> In short-circuit testing we inject worst-case exceptions <lb/>(see Section 3.2), where " worst-case " means that we in-<lb/>ject exceptions at the beginning of the try block. Thus, <lb/>the injected exception discards the whole code of the try <lb/>block. <lb/>Another possibility would be to inject exceptions at <lb/>dierent locations in the try block (for instance, before <lb/>
<page>11 <lb/></page>
line 1 of the try block, after line 1, after line 2, etc.). <lb/>Let us call this algorithm " fine-grain injection". It would <lb/>have the following property. First, it would take longer; <lb/>instead of executing each test once with worst-case injec-<lb/>tion, this algorithm would execute each test for each pos-<lb/>sible injection location. Second, if the satisfaction or vi-<lb/>olation of the contracts is undecidable with short-circuit <lb/>testing, it would still be undecidable with fine-grain in-<lb/>jection because the verifiability of contracts depends on <lb/>the test suite coverage of try-catch only and not on the <lb/>location (see section 3.3). Also, this algorithm could not <lb/>invalidate the source dependence of catch blocks (if a <lb/>test case fails with short-circuit, it would still fail with <lb/>fine-grain injection which encompasses the worst case). <lb/>The most interesting property of fine-grain injection is <lb/>that it could show that some catch blocks that are char-<lb/>acterized as source independent by short-circuit testing <lb/>are actually source dependent. Let us show that it is <lb/>theoretically possible yet unlikely. <lb/>Let us consider the code in Listing 7 and a test case <lb/>which asserts that this method returns 0 in a specific er-<lb/>ror scenario. In this error scenario, the exception comes <lb/>from the call to myMethod(). In this case, when enter-<lb/>ing the catch block, a=true because the last assigment <lb/>to a is a=bar() and bar returns true. Consequently <lb/>the catch block returns null and the test passes. With <lb/>short-circuit testing an exception is thrown at injection <lb/> location #1 when a is still equals to true (assigned just <lb/>before the try block). As a result the catch block still <lb/>returns null as expected by the test, which passes. The <lb/>behavior of the catch block is the same as the expected <lb/>one, so short-circuit testing assesses that this catch block <lb/>is source independent. The fine-grain injection algorithm <lb/>would identify 2 additionnal injection locations injection <lb/>location #2 and #3. It would execute the test 3 times, <lb/>each time with injection at a dierent location. In the <lb/>first run, it works just as short-circuit testing (injection <lb/>in injection location #1 ). In the second run an excep-<lb/>tion is injected at injection location #2, thus a = false <lb/> because the last assigment to a is a=foo() and foo re-<lb/>turns false. Consequently the catch block returns ≠1 <lb/> instead of 0. The behavior of the catch is dierent under <lb/>injection, the test case fails (≠1 returned instead of 0). <lb/> Because of a failing test case under injection, the catch <lb/>block under test would be proven source dependent (as <lb/>said by Predicate #2 in Section 3.3.2). <lb/>This example shows that there may be a sequence of <lb/>program states that are either recoverable or unrecover-<lb/>able. The state switching occurs not only between the <lb/>statements of the try block but also within the code ex-<lb/>ecuted in the called methods. Consequently, a meaning-<lb/>ful fine-grain injection would not only happen at each <lb/>statement of the try under analysis but also between the <lb/>statements of the called methods, in a recursive manner. <lb/>In other words, for worst-case injection, there is only <lb/>one injection required for reasoning on the resilience, in <lb/> 1 <lb/> boolean a = true; <lb/> 2 <lb/> try { <lb/> 3 <lb/> // injection location #1 <lb/> 4 <lb/> a = foo();//returns false <lb/> 5 <lb/> // injection location #2 <lb/> 6 <lb/> a = bar();//returns true <lb/> 7 <lb/> // injection location #3 <lb/> 8 <lb/> return myMethod();//exception thrown <lb/> 9 <lb/> } catch (Exception e) { <lb/> 10 <lb/> if (a){ <lb/> 11 <lb/> return 0; <lb/> 12 <lb/> }else{ <lb/> 13 <lb/> return ≠1; <lb/> 14 <lb/> } <lb/> 15 <lb/> } <lb/> Listing 7: An example of try-catch block categorized <lb/>dierently by worst-case exception injection (short-<lb/>circuit testing) and fine-grain exception injection <lb/>fine grain injection, there is a myriad of injected excep-<lb/>tions for the same try block (it is common to have one <lb/>thousand or more recursively executed statements within <lb/>the same try block execution 6 ). Hence, we recommend <lb/>worst-case injection, which is sound for the detection of <lb/>contract violations and has a predictable and aordable <lb/>computational cost (one test suite run by analyzed try-<lb/>catch). <lb/>The same reasoning applies to single-statement try <lb/>blocks (a try block with only one line of code). It is <lb/>not equivalent to receive a thrown exception from this <lb/>single statement and to inject a worst-case exception at <lb/>the beginning of the try block. Within the recursively <lb/>called methods and executed statements, there may still <lb/>be a sequence of recoverable and unrecoverable states. <lb/>Short-circuit testing of single-statement try blocks en-<lb/>sures that the catch block makes no assumption on what <lb/>is internally done during the execution of the statement. <lb/>Indeed, the real-world code of Listing 5 shows an exam-<lb/>ple of a single-statement try block, which is identified as <lb/>source-dependent thanks to short circuit testing. <lb/> 5.2 Checked and Unchecked <lb/> Short-circuit testing is independent of the programming <lb/>language and is applicable to the common exception <lb/>models (C++ , Python, Ruby, . . . ). However, our ex-<lb/>periments are done in Java, where there are two kinds of <lb/>exceptions : checked and unchecked. Unchecked excep-<lb/>tion are standard exceptions, similar to other languages, <lb/>and may occur anywhere in the code. On the contrary, <lb/>checked exceptions are subject to explicit declaration and <lb/>static verification. Consequently, the developers are al-<lb/>ways aware of all possible locations where a checked ex-<lb/>ception may occur (otherwise the code does not compile). <lb/>
<note place="footnote"> 6 Note that we do not take in consideration that an exception <lb/>may happen within a call to the standard library. In theory, all <lb/>intermediate states during a library call (and inside a native call) <lb/>are also candidate to exception injection, which is really expensive <lb/>to compute. <lb/></note>
<page> 12 <lb/></page>
1 <lb/> try { <lb/> 2 <lb/> parametriseStep(); // <≠≠ set the expected value to <lb/>StepAsString <lb/> 3 <lb/> return successful(stepAsString).withParameterValues( <lb/>parametrisedStep); <lb/> 4 <lb/> } catch (InvocationTargetException e) { <lb/> 5 <lb/> return failed(stepAsString, new UUIDExceptionWrapper( <lb/>stepAsString, failureCause)).withParameterValues( <lb/>parametrisedStep); <lb/> 6 <lb/> } <lb/> Listing 8: Violation of the Source Independence Contract <lb/>in jbehave-core (StepCreator.java). <lb/>This statically verified exceptional behavior has an im-<lb/>pact on short-circuit testing. <lb/>Short-circuit testing injects the exception at the begin-<lb/>ning of the try block. For checked exceptions, it results in <lb/>a statically and dynamically valid thrown exception (we <lb/>are still in the scope of the try block). From the view-<lb/>point of the programmer it is artificial: the programmer <lb/>knows that this exception can only happen as of the first <lb/>statement that declares throwing this checked exception. <lb/>In other words, she is sure that it is impossible that this <lb/>exception comes from the beginning of the try. However, <lb/>even if the exception itself is artificial, it enables us to <lb/>assess the resilience contracts which allows us to perform <lb/>catch stretching (see Section 3.4). <lb/>With catch stretching, in the original case, injected ex-<lb/>ceptions are of type X and caught by catch(X e). In <lb/>the stretched version, the caught exception is of the most <lb/>generic type Exception. Hence, the stretched version of <lb/>a catch block rescuing a checked exception is also able to <lb/>rescue an unchecked exception. Contrary to the injected <lb/>checked exception, those unchecked exceptions are not <lb/>impossible, they may happen in unanticipated scenar-<lb/>ios. Consequently, short-circuit testing is also valuable <lb/>for checked exceptions because it indirectly allows im-<lb/>proving the resilience against unanticipated unchecked <lb/>exceptions. <lb/> 5.3 Threats to Validity <lb/> The main threat to the construct validity lies in bugs in <lb/>our implementation of short-circuit testing. To mitigate <lb/>this threat, over the course of our experiments, we have <lb/>regularly thoroughly analyzed try-catch blocks for which <lb/>satisfaction or violation of one of the contracts was iden-<lb/>tified. The analysis consists of reading the try-catch, the <lb/>surrounding code and the test cases that execute it. In <lb/>the end, this gives us good confidence that there is no <lb/>serious bug in the experimental code. <lb/>The internal validity of our experiment is threatened <lb/>if the test case behavior we observe is not causally con-<lb/>nected to the exception that are thrown. Since we use <lb/>mature determistic software (a Java Virtual Machine) <lb/>and a deterministic dataset, this is actually unlikely that <lb/>spurious exceptions mix up our results. <lb/>Finally, concerning the external validity, we have <lb/>shown that in open-source Java software, there ex-<lb/>ists source-dependent, source-independent and purely-<lb/>resilient try-catch blocks. This may be due to the pro-<lb/>gramming language or the dataset. We believe that <lb/>it is unlikely since our contracts relate to language <lb/>and domain-independent concepts (contracts, applica-<lb/>tion state, recovery). <lb/> 6 Related work <lb/> Segal et al. [21, 3] invented Fiat, an early validation sys-<lb/>tem based on fault injection. Their fault model simulates <lb/>hardware fault (bit changes in memory). Kao et al. [15] <lb/>have described " Fine " , a fault injection system for Unix <lb/>kernels. It simulates both hardware and operating sys-<lb/>tem software faults. In comparison, we inject high-level <lb/>software faults (exceptions) in a modern platform (Java). <lb/>Bieman et al. [5] added assertions in software that can be <lb/>handled with an " assertion violation " injector. The test <lb/>driver enumerates dierent state changes that violate the <lb/>assertion. Their technique improves branch coverage, es-<lb/>pecially on error recovery code. This is dierent from <lb/>our work since: we do not manually add any information <lb/>in the system under study (tests or application). Fu et <lb/>al. [8] described a fault injector for exceptions similar to <lb/>ours in order to improve catch coverage. In comparison <lb/>to [5] and [8], we do not aim at improving the coverage <lb/>but at identifying the try-catch blocks satisfying excep-<lb/>tion contracts. <lb/>Sinha [22] analyzed the eect of exception handling <lb/>constructs (throw and catch) on dierent static analyses. <lb/>In contrast, we use dynamic information for reasoning <lb/>on the exception handling code. The same authors de-<lb/>scribed [23] a complete tool chain to help programmers <lb/>working with exceptions. The information we provide <lb/>(the list of source-independent, purely-resilient try-catch <lb/>blocks and so forth) is dierent, complementary and may <lb/>be subject to be integrated in such a tool. <lb/>Candea et al. [7] used exception injection to capture <lb/>the error-related dependencies between artifacts of an <lb/>application. They inject checked exceptions as well as <lb/>6 runtime, unchecked exceptions. We also use exception <lb/>injection but for a dierent goal: verifying the exception <lb/>contracts. <lb/>Ohe et al. [18] described an exception monitoring sys-<lb/>tem that resembles ours. Beyond the monitoring system <lb/>we also provide a strategy, two contracts and four pred-<lb/>icates to verify two exception contracts. <lb/>Ghosh and Kelly [10] did a special kind of mutation <lb/>testing for improving test suites. Their fault model com-<lb/>prises " abend " faults: abnormal ending of catch blocks. <lb/>It is similar to short-circuiting. We use the term " short-<lb/>circuit " since it is a precise metaphor of what happens. <lb/>In comparison, the term " abend " encompasses many <lb/>more kinds of faults. In our paper, we claim that the <lb/>new observed behavior resulting from short-circuit test-<lb/>
<page>13 <lb/></page>
ing should not be considered as mutants to be killed. <lb/>Actually we claim that short-circuiting should remain <lb/>undetected for sake of source independence and pure re-<lb/>silience: if short-circuiting is detected, it means that at <lb/>least one try-catch is not source-independent. Conse-<lb/>quently, if one values and agrees with the source inde-<lb/>pendence contract, short-circuiting should remain unde-<lb/>tected. <lb/>Fu and Ryder [9] presented a static analysis for re-<lb/>vealing the exception chains (exception encapsulated in <lb/>one another). In contrast, our approach is a dynamic <lb/>analysis. We do not focus on exception chains, we pro-<lb/>pose an analysis of source-independence and pure re-<lb/>silience. Mercadal [16] presented an approach to man-<lb/>age error-handling in a specific domain (pervasive com-<lb/>puting). This is forward engineering. On the contrary, <lb/>we reason on arbitrary legacy Java code, we identify re-<lb/>silient locations and provide techniques to improve their <lb/>resilience. <lb/>Zhang and Elbaum [26] have recently presented an ap-<lb/>proach that amplifies test to validate exception handling. <lb/>Their work has been a key source of inspiration for ours. <lb/>Short-circuit testing is a kind of test amplification. While <lb/>the technique is the same, the problem domain we ex-<lb/>plore is really dierent. They focus on exceptions related <lb/>to external resources. We focus on any kind of exceptions <lb/>in order to verify resilience contracts. <lb/> 7 Conclusion <lb/> In this paper, we have explored the concept of software <lb/>resilience against unanticipated exceptions. We have for-<lb/>malized two resilience properties: source-independence <lb/>and pure-resilience. We have devised an algorithm, <lb/>called short-circuit testing, to verify them. Finally, we <lb/>have proposed a source code transformation called " catch <lb/>stretching " that improves the ability of the application <lb/>under analysis to handle unanticipated exceptions. Our <lb/>empirical evaluation on 9 open-source applications show <lb/>that those contracts characterize the exceptional behav-<lb/>ior of real code: there exist try-catch blocks that satisfy <lb/>and violate the contracts. <lb/>Our future work consists in further exploring how to <lb/>improve the resilience of software applications: the scope <lb/>of try blocks can be automatically adapted while still <lb/>satisfying the test suite; the purely resilient catch blocks <lb/>could probably be used elsewhere because they have a <lb/>real recovery power; the resilience oracle has not to be <lb/>only a test suite, but for example metamorphic relations <lb/>or production traces. <lb/></body>
<listBibl> References <lb/> [1] CNSC Fukushima Task Force Report. Technical Re-<lb/>port INFO-0824, Canadian Nuclear Safety Commis-<lb/>sion, 2011. <lb/>[2] A. Avizienis, J.-C. Laprie, B. Randell, and <lb/>C. Landwehr. Basic concepts and taxonomy of de-<lb/>pendable and secure computing. IEEE Transactions <lb/>on Dependable and Secure Computing, 1(1):11–33, <lb/>2004. <lb/>[3] J. H. Barton, E. W. Czeck, Z. Z. Segall, and D. P. <lb/>Siewiorek. Fault injection experiments using fiat. <lb/> IEEE Transactions on Computers, 39(4):575–582, <lb/>1990. <lb/>[4] B. Beizer. Software testing techniques. Dreamtech <lb/>Press, 2003. <lb/>[5] J. M. Bieman, D. Dreilinger, and L. Lin. Using <lb/>fault injection to increase software test coverage. In <lb/> Seventh International Symposium on Software Reli-<lb/>ability Engineering, pages 166–174. IEEE, 1996. <lb/>[6] B. Cabral and P. Marques. Exception handling: A <lb/>field study in Java and .Net. In Proceedings of the <lb/>European Conference on Object-Oriented Program-<lb/>ming, pages 151–175. Springer, 2007. <lb/>[7] G. Candea, M. Delgado, M. Chen, and A. Fox. Au-<lb/>tomatic failure-path inference: A generic introspec-<lb/>tion technique for internet applications. In Proceed-<lb/>ings of the 3rd IEEE Workshop on Internet Appli-<lb/>cations, pages 132–141. IEEE, 2003. <lb/>[8] C. Fu, R. P. Martin, K. Nagaraja, T. D. Nguyen, <lb/>B. G. Ryder, and D. Wonnacott. Compiler-directed <lb/>program-fault coverage for highly available java in-<lb/>ternet services. In Proceedings of the International <lb/>Conference on Dependable Systems and Networks, <lb/> 2003. <lb/>[9] C. Fu and B. G. Ryder. Exception-chain analysis: <lb/>Revealing exception handling architecture in java <lb/>server applications. In Proceedings of the 29th Inter-<lb/>national Conference on Software Engineering, 2007. <lb/>[10] S. Ghosh and J. L. Kelly. Bytecode fault injection <lb/>for java software. Journal of Systems and Software, <lb/> 81(11):2034–2043, 2008. <lb/>[11] J. B. Goodenough. Exception handling: Issues and <lb/>a proposed notation. Commun. ACM, 18(12):683– <lb/>696, 1975. <lb/>[12] J. Gosling, B. Joy, G. Steele, and G. Bracha. Java <lb/>Language Specification. Addison-Wesley, 3rd edi-<lb/>tion, 2005. <lb/>[13] J. J. Horning, H. C. Lauer, P. M. Melliar-Smith, and <lb/>B. Randell. A program structure for error detection <lb/>and recovery. Springer, 1974. <lb/>[14] J.-C. Laprie. From dependability to resilience. In <lb/> Proceedings of the International Conference on De-<lb/>pendable Systems and Networks, 2008. <lb/>
<page>14 <lb/></page>
[15] W. lun Kao, R. K. Iyer, and D. Tang. Fine: A <lb/>fault injection and monitoring environment for trac-<lb/>ing the unix system behavior under faults. IEEE <lb/> Trans. Software Eng., 19(11):1105–1118, 1993. <lb/> [16] J. Mercadal, Q. Enard, C. Consel, and N. Loriant. <lb/>A domain-specific approach to architecturing error <lb/>handling in pervasive computing. In Proceedings <lb/>of the Conference on Object Oriented Programming <lb/>Systems Languages and Applications, 2010. <lb/>[17] B. Meyer. Applying design by contract. Computer, <lb/> 25(10):40–51, 1992. <lb/>[18] H. Ohe and B.-M. Chang. An exception mon-<lb/>itoring system for java. <lb/>In Rapid Integration <lb/>of Software Engineering Techniques, pages 71–81. <lb/>Springer, 2005. <lb/>[19] P. Rovner. Extending modula-2 to build large, inte-<lb/>grated systems. Software, IEEE, 3(6):46–57, 1986. <lb/>[20] W. Savitch. JAVA An Introduction to Problem Solv-<lb/>ing and Programming. Pearson, 2012. <lb/>[21] Z. Segall, D. Vrsalovic, D. Siewiorek, D. Yaskin, <lb/>J. Kownacki, J. Barton, R. Dancey, A. Robinson, <lb/>and T. Lin. Fiat-fault injection based automated <lb/>testing environment. In Proceedings of the Eigh-<lb/>teenth International Symposium on Fault-Tolerant <lb/>Computing, pages 102–107. IEEE, 1988. <lb/>[22] S. Sinha and M. J. Harrold. Analysis and testing of <lb/>programs with exception handling constructs. IEEE <lb/>Transactions on Software Engineering, 26(9):849– <lb/>871, 2000. <lb/>[23] S. Sinha, A. Orso, and M. J. Harrold. Automated <lb/>support for development, maintenance, and testing <lb/>in the presence of implicit flow control. In Proceed-<lb/>ings of the 26th International Conference on Soft-<lb/>ware Engineering, pages 336–345. IEEE, 2004. <lb/>[24] M. Staats, M. W. Whalen, and M. P. E. Heim-<lb/>dahl. Programs, tests, and oracles: the foundations <lb/>of testing revisited. In Proceedings of the Interna-<lb/>tional Conference on Software Engineering, pages <lb/>391–400. IEEE, 2011. <lb/>[25] K. S. Trivedi, D. S. Kim, and R. Ghosh. Resilience <lb/>in computer systems and networks. In Proceedings <lb/>of the 2009 International Conference on Computer-<lb/>Aided Design, ICCAD '09, pages 74–77, New York, <lb/>NY, USA, 2009. ACM. <lb/>[26] P. Zhang and S. Elbaum. Amplifying tests to vali-<lb/>date exception handling code. In Proceedings of the <lb/>International Conference on Software Engineering, <lb/> pages 595–605. IEEE Press, 2012. <lb/></listBibl>
<page>15 </page>
</text>
</tei>