Methods for Effective Java Unit Testing

Joint Meeting with
Los Angeles ACM Chapter and LA SIGPLAN

Wednesday, November 8, 2000

Methods for Effective Java Unit Testing

Software distributed with defects is common and widely accepted. Companies issue license agreements indicating that software is purchased as-is, without guaranteeing it will perform as intended. Yet, the quality of software available to end-users can be improved. For this to happen, a revolution must occur in the software development process that implements a set of consistent and conscious programming techniques. Such techniques will also decrease the amount of time spent developing and debugging software. This presentation focuses on Java and offers several development guidelines that, if adopted, will result in a higher quality code that is easier to maintain and debug. Some of these practices include coding standard enforcement, white-box testing and black-box testing. Dr. Kolawa will also perform a demonstration of Jtest, ParaSoft's award-winning automatic unit testing tool for Java.

Dr. Adam Kolawa is the CEO of ParaSoft Corporation, a leading provider of
software productivity solutions. Dr. Kolawa holds a Ph.D. in theoretical physics from the California Institute of Technology. Dr. Kolawa has extensive experience in both programming and managing programmers. He has written several successful commercial software products, and has headed the development of a large number and variety of software projects, including software development tools, Web development and management tools, data mining tools and more. Heading a company that sells products to programmers and managers has given Dr. Kolawa the unique opportunity to interact with software developers and managers at multiple levels. Dr. Kolawa established his ability to clearly communicate the expertise he has gained from his education and experience in the many papers on physics and computer science that appeared in such publications as the Caltech Concurrent Computing Project Memo and the Proceedings of the Hypercube Conference. Two of Dr. Kolawa's papers are included in Parallel Computing Works!. Ed. Geoffrey Fox. Dr. Kolawa also writes opinion pieces and technical articles for various industry publications such as Software Development, SD Times, and Java Report. He has also spoken at many conferences including STAR East '99, JavaOne, Linux Expo, Sigs Conference for Java, Software Development East, SGI Expo, Quality Week, IEEE conferences and has also conducted national seminars on Strategies for Effective Software Development.

LA ACM Chapter November Meeting held Wednesday, November 8, 2000. This was a Joint Meeting of the Los Angeles Chapter of ACM and LA ACM SIGPLAN.

The presentation was "Methods for Effective Java Unit Testing" by Dr. Adam Kolawa, CEO and founder of the ParaSoft Corporation.

Dr. Kolawa started out by explaining that he would focus on Java and testing Java to show how you can use unit testing to improve the quality of the software your are writing. The car industry faced similar problems and in one year a car model had more cars being recalled than were being produced. American producers tried to cure the problem by having a team of mechanics at the end of the assembly line correcting the faults on each car that was produced. The Japanese faced the same problem of too many defects occurring during the assembly process. They tracked down where the defects were occurring during the production process and changed their processes to prevent producing defects. This idea can be transferred to software. Currently the common procedure is to produce software, test it, and deliver it. Dr. Kolawa referred to the process as "Extreme Programming" where testing is moved to very early in the development cycle and you test it as you build it. He believes that in the near future the job of Quality Assurance manager will be replaced by Development Quality manager.

One key item is error prevention because errors are expensive. One important method of preventing errors is use of defined coding standards, which are not necessarily appreciated by those who are required to use them. Unit testing helps to move testing early in the development process. The software industry can benefit by what has happened to the car industry in the last 15 years as quality of cars has improved drastically. Especially in Europe, manufacturers have assigned their products to take part in rallies where cars are driven under extreme conditions. Then they tore them down to see how their parts had held up. They found it was possible to build special test equipment and test parts such as shocks to find problems and make fixes before the parts were assembled into cars. It doesn't replace driving the cars, but it shifts the emphasis from individual defective parts to how well the parts integrate into the completely assembled car.

Unit testing is testing the smallest possible unit of an application that in an object-oriented program is testing a class or function as soon as it is completed. This can be very cost effective. On two large Hewlett-Packard projects the average construction defect cost 25% to 50% as much to fix as the average design error during the initial coding process. When fixes were delayed until later the total cost to fix construction errors increased up to twice as much as fixing design defects. Unit testing finds many errors early when they are less costly to fix. It is easier to track down potential errors and achieve 100% coverage of program code at this level. One problem has been that it has been considered too costly and requiring too much time to accomplish thorough unit testing. Unit testing requires writing special code and building a "scaffolding" to run the class test. Stubs have to be built to return values that are unavailable from external resources during the unit test phase. All of this requires extensive debugging which is sometimes harder to do for the "scaffolding" than for the particular unit under test. Test cases should be created both to determine that outputs meet specifications for specific inputs (black box testing) and that the software doesn't collapse or fail when it is given a wide range of inputs (white box testing). This can become very expensive if the entire process is done manually for each separate class.

In order to be practical, unit testing has to be automated. Approaches to automation include semi-automation where some steps are still done manually, and complete automation. Automation can use one of Parsoft's Java tools named JTEST to create stubs and test cases that will execute 100% of the code, but not every path. "Design By Contract" is an important way of seeing this is done correctly as the developers of the external systems specify what interface inputs are required and the outputs they will provide. Also it is necessary to persuade programmers to use good coding standards so that problems can be avoided. One problem with coding standards is the number of different ones available. For Java there are 174 existing coding standards and more are coming. If you are coding in C++, which is a more difficult language to use than Java, then you are badly in need of software tools.

You have to be able to write specifications that describe the object. "Extreme Programming" is very good if you have smart people, but if the design is not flexible enough you can run into trouble. One big advantage of unit testing is that you can test the most complicated things first to make certain the basic design concept is practical. When this is completed it is usually much simpler to complete the rest. Semi-automatic testing automates running test cases and error reporting. The drawback is that developers still need to create scaffolding and stubs, call functions, and design all test cases. Total automation automates everything from building scaffolding and stubs to creating and running test cases. ParaSoft decided they needed this type of solution. They tried to find one available, but couldn't find it so they built one. Currently they have developed automatic unit testing technologies for C/C++ and Java.

ParaSoft believes that programmers are a valuable commodity so burning out programmers is a bad policy, and that if you don't organize your development process bad things will happen. Developers need "play time" and if you give it to them they will improve and it will help you.

Dr. Kolawa made some additional remarks on semi-automation and total automation. Semi-automation requires having a test suite you can run every night. Test cases that cause failures are the interesting ones. If you are developing your own test cases he recommends using the Linux operating system rather than Microsoft. You can build good test software on Linux machines and use it to test Microsoft NT programs running on Windows NT. For total automation you need to have good tools with good interfaces including a batch interface. More than a GUI process is required. GUI can be used during the day for individual test cases, but it is important to be able to run a batch set of test cases overnight. He said that managers seem to believe that $3000 to $5000 test tools are expensive, but the cost of extra programmer time is much more expensive.

He concluded that technologies that automate unit testing are now available. Developers serious about quality should review their development process and determine how to integrate automatic unit testing into it if they wish to be successful.

For an extensive amount of information on Java testing and other items of interest on the ParaSoft Corporation go to:

http://www.parasoft.com

If you have any interest in automating unit tests for Java and other widely used languages I would recommend going to this site. It provides a great deal of information on their JTEST testing tool in much greater detail than Dr. Kolawa was able to provide in the time he had available at our meeting. Indeed, if you missed the meeting you can certainly fulfill your requirements for information on Java unit testing. However, missing the meeting means you missed his insights on the programming process that cannot be completely provided in this DATA-LINK report. To get the full benefit of those, you had to be there.

This was third meeting of the LA Chapter year and was attended by about 25 persons. It was an interesting and educational meeting.

Mike Walsh, LA ACM Membership Chair

The Los Angeles Chapter normally meets the first Wednesday of each month at the Ramada Hotel, 6333 Bristol Parkway, Culver City. The program begins at 8 PM. From the San Diego Freeway (405) take the Sepulveda/Centinela exit southbound or the Slauson/Sepulveda exit northbound.

To make a reservation, call or e-mail John Radbill, (818) 353-8077, and indicate your choice of entree, by Sunday before the dinner meeting.
There is no charge or reservation required to attend the program. Parking is FREE!

Thursday, Nov. 16, 6:30 Social, 7:30 Program
Leonard H. Goldenson Theater Academy of Television Arts & Sciences
5230 Lankershim Blvd., North Hollywood, CA 91601

If you couldn't make it to SIGGRAPH 2000 in New Orleans, you'll have another opportunity to see the 'Electronic Theater 2000' this evening.

Also: B.Y.O.V. - Bring Your Own Video for a juried screening of LA SIGGRAPH chapter members' work.

The Academy is located in North Hollywood two blocks east of the 170 Freeway (Magnolia Avenue exit) in the complex on the northeast corner of Lankershim and Magnolia. There is a $2 fee in the Academy parking structure.

The Academy of Television Arts & Sciences is a five minute walk south of the new North Hollywood Metro Red Line subway station at Lankershim and Chandler.