## Chapter 8: Testing and Debugging

## Chapter Goals

- To learn how to design test harnesses for testing components of your programs in isolation
- To understand the principles of test case selection and evaluation
- To be able to use assertions to document program assumptions
- To become familiar with the debugger
- To learn strategies for effective debugging

## Unit Tests

- In a unit test, a function or procedure is compiled outside the program in which it will be used, together with a test harness that feeds arguments to it.
- Once you have confidence that the function is working correctly, you can plug it into your program.
- Test data can be entered manually (see `sqrtest1.cpp`), in a loop (see `sqrtest2.cpp`), or using random number generation (see `sqrtest3.cpp`).

## Unit Tests (`sqrtest1.cpp`)

## Unit Tests (`sqrtest2.cpp`)

## Unit Tests (`sqrtest3.cpp`)

## Selecting Test Cases

- Positive tests consist of legitimate inputs that you expect the program handles correctly.
- Boundary cases are also legitimate inputs that you expect the program to handle in a trivial way.
- Negative cases are inputs that you expect the program to fail on.
- Keeping test data in a file is smart because you can use it to review every version of the program.
- When you find a bug, you should record the data that caused the bug, and retest your program with that data after you have tried to fix the bug.
- The phenomenon of fixing a bug, only to have it reappear after further program modifications is called cycling.
- The process of testing against a set of past failures is called regression testing.

## Test Case Evaluations

- Once you have determined what inputs are needed to test the program, you need to decide if the outputs are correct.
- Sometimes you can verify output by calculating the correct values by hand.
- Sometimes a computation does a lot of work, and it is not practical to do the computation manually.

  >     double squareroot(double a)
  >     {
  >        if (a == 0) return 0;

  >        double xnew = a;
  >        double xold;

  >        do
  >        {
  >           xold = xnew;
  >           xnew = (xold + a / xold) / 2;
  >        }
  >        while (not approx_equal(xnew, xold));

  >        return xnew;
  >     }

- We could write a test program that verifies that the output values fulfill certain properties.
- Here we test the `squareroot()` function by comparing the square of the returned number with the original input. (See `sqrtest4.cpp`).
- Another method is to use a slower but reliable procedure (called an oracle) to verify the results. (In this case, we will use `pow()` as an oracle). (See `sqrtest5.cpp`)

## Test Case Evaluations (`sqrtest4.cpp`)

## Test Case Evaluations (`sqrtest5.cpp`)

## Assertions

- Functions often contain implicit assumptions (denominators should be nonzero, salaries should not be negative).
- Such illegal values can creep into a program due to an input or processing error - with distressing regularity!
- Assertions provide a valuable sanity check.

  >     void raise_salary(Employee& e, double by)
  >     {
  >        assert(e.get_salary() >= 0 );
  >        assert(by >= -100);
  >        double new_salary = e.get_salary() * (1 + by / 100);
  >        e.set_salary(new_salary);
  >     }

- If an assertion is not satisfied, the program terminates with a useful error message showing the line number and the code of the failed assertion:

  >     assertion failed in file finclac.cpp line 61: by >= -100

## Program Traces

- To get a printout of the program flow, you can insert trace messages into the beginning and end of every procedure.
- It is also useful to print the input parameters when a procedure is entered and to print return values when a function is exited.

  >     string int_name(int n)
  >     {
  >        cout << "Entering digit_name. n = " << n << "\n";
  >        ...
  >        cout << "Exiting digit name. Return value = "
  >           << s << "\n";
  >        return s;
  >     }

- To get a proper trace, you must locate each function exit point.
- Sample output for `n` = 12305.

  >     Inside int_name. Thousands.
  >     Entering int_name. n = 12
  >     Inside int_name. Teens.
  >     Entering teen_name. n = 12
  >     Exiting teen_name. Return value = twelve
  >     Exiting digit_name. Return value = twelve
  >     Inside int_name. Hundreds.
  >     Entering digit_name. n = 3
  >     Exiting digit_name. Return value = three
  >     Inside int_name. Ones.
  >     Entering digit_name. n = 5
  >     Exiting digit_name. Return value = five
  >     Exiting int_name. Return value = twelve thousand three hundred five

## Program Traces (Problems with Trace Messages)

- Program traces can be time-consuming to find out which trace message to insert.
- If you insert too many messages, you produce a flurry of output that is hard to analyze.
- If you insert to few, you many not have enough information to spot the cause of the error.
- When you are done with the program, you need to remove all the trace messages.
- When you find a new error, you need to stick the print statement back in.
- Many profession programmers use a debugger, not trace messages, to locate errors in their code.

## The Debugger

- Modern development environments contain special programs, called debuggers, that help you locate bugs by letting you follow the execution of a program.
- You can stop and restart your program and see the contents of variables whenever your program is temporarily stopped.
- Using a debugger is not cost free - it takes time to set up and carry out an effective debugging session.
- Larger programs are harder to debug without using a debugger, so learn how to use one is time well spent.

## The Debugger

- Debuggers vary wildly from one system to another.
- If you use an integrated development environment, which contains an editor, compiler, and a debugger, finding and starting the debugger is usually very easy.
- On a UNIX system, you must manually build a debug version of your program and invoke the debugger.
- You can go a long way with just three commands:
  - Run until this line.
  - Step to next line.
  - Inspect Variable.
- The exact names for these commands will vary from debugger to debugger, but all debuggers support them.

## The Debugger

- The "run until this line" command is the most important.
- Select a line with the mouse or cursor, then hit a key or select a menu command to run the program to the select line.
- It's possible that the program terminates normally without reaching that line, but that could be informative.

## The Debugger

- The "step to next line" command executes the current line and stops at the next program line.
- Once the program has stopped, you can look at the current values of variables.
- Some debuggers require you select the variable the choose a command such as "inspect variable."
- Some debuggers require you to type the name of the variable into a dialogue box.
- Some variables automatically show the values of all variables.

## The Debugger

- Many debuggers also have "step over" and "step into" commands.
- The "step over" command advances to the next program line:
- Example: If the current line is

  >     r = future_value(balance, p n);
  >     cout << setw(10) << r;

  "step over" advances the program to

  >     r = future_value(balance, p n);
  >     cout << setw(10) << r;

- The "step into" command advances into function calls.
- Example: if the current line is

  >     r = future_value(balance, p n);
  >     cout << setw(10) << r;

  "step into" advances the program to

  >     double future_value(double initial_balance,
  >        double p, int n)
  >     {
  >        double b = initial_balance * pow((1 + p /100), n);
  >        return b;
  >     }

## The Debugger

- You should not "step into" system functions like `setw`.
- Some debuggers have a "run to end of function" or "step out" command to get out of a function.
- Most debuggers can show you a call stack: a listing of all currently pending function calls.
- My selecting the function in the middle of the call stack, you can jump to the code line containing that function call.

## The Debugger

- All debuggers support a navigational approach by inserting breakpoints in the code.
- When the program reaches any breakpoint, execution stops.
- Breakpoints are particularly useful when you know at which point your program starts doing the wrong thing.
- Some debuggers let you set conditional breakpoints - the program stops only when a certain condition, such as `n == 0`, is met.

## The Debugger

- When inspecting an object variable, all the fields are displayed.

- With some variables you must "open up" the object, usually by clicking on a tree node. Here, we have opened up the `name` field of the `harry` object..

## Strategies

- Reproduce the error.
  - What numbers did you enter?
  - Where did you click the mouse?
  - Run the same program again; type in exactly the same input.
- Divide and Conquer
  - Step over features using debugger, but don't step into them.
  - Locate the last procedure called before failure.
  - Step into that procedure, and repeat.

## Strategies

- Know what your program should do.
  - Does your program do what's expected?
  - Before you inspect a variable, decide what value is should have.
    - If value is correct, look elsewhere for the bug.
    - If value is incorrect, double check your computation, then try to find out why your program comes up with a different value.
  - Look for off by one errors.
  - Look for computation errors (is the formula typed in correctly).
  - Make "sign calculations" when the numbers get nasty. ("Should this number be positive or negative?")

## Strategies

- Avoid quick fixes - they tend to create problems elsewhere.
- Develop a thorough understanding of the problem and the program before you try to fix it.
- Sometimes fix and fix only causes the problem to move around. Possibly the program logic is incorrect (does the program need redesign?).

## Debugger Limitations

- Break points in recursive procedures interrupt the program each time through the procedure.
- When using the debugger on a recursive procedure, watch the call stack carefully.
- Sometimes the compiler generates faster code by keeping a variable in a processor register rather than reserving a memory location for it.
  - The debugger cannot find that variable or displays a wrong value for it.
  - You can try turning off all compiler optimizations and recompile.
  - You can open a special register window that shows all processor registers (advanced).
- Some errors show up when you run the program normally, but go away under the debugger.
  - Usually an uninitialized variable is the cause.
  - Inspect all variables manually and check that they are initialized.
  - Insert print statements if you are desperate.