I am a student of software-engineering. In one way, I am speaking to you also as a representative of a large community of students of this subject. Software Engineering is a subject of a practical nature. It can't be learned by learning theories and methods without appreciating the scale, or at least the nature, of the problems of software engineering. One must get his hands dirty in encountering the practical problems, and if possible, to solve them in his own right. On the other hand, it's an oversimplification to say that taking a plunge into real-life industrial scale problem right after graduation can take the place of academic understanding. At best, it often creates cynical software engineers who have given in to the maxim that software-development is inherently a misery. They talk theory only to convince auditors that they deserve CMM Level 5 certification.
The real lessons of software-engineering are to be obtained in the academic environment which give an ample exposure to both theoretical and practical aspect of this difficult subject. Whether this environment is created in the universities or in the industry training rooms is besides the question. The key lies in the orientation and content of the course and the attitude of the instructor and the student alike.
Though my association with software engineering as a practice is now many years old, my introduction to it as an academic subject is new. In fact, I started studying software testing formally quite recently. Immediately after I got introduced to the very basic ideas of testing, the first thing I started craving for was to convince myself: of the fact that elaborate testing is indeed required. I knew that industrial projects are complex and are in dire need of automation, not just in testing, but in all stages of SDLC. I had seen for myself, that the amount of automation achieved in testing in most projects is dismal. I needed to see the utility at a scale where I could comprehend the need for testing, and test the automated methods that could be fruitfully applied at that level, at the same time, without getting overawed by the scale where these should actually be employed.
I started on my own to design a automatic testing system for a small software system I had built. The SUT was called mobilewp, a small emulation of the t9 dictionary in mobile phones. It displayed on the console the list of prefixes of candidate words that could be formed with the given set of keypad inputs. The system was 2000 lines of C code. It took me nearly a man-week to finish the implementation including the design of the test automation system.
The test harness was a very simple one. I wrote a bunch of test-cases, about 100 of them, manually. I wrote a small shell-script that invoked the mobilewp program with the input that I would provide it with. It would then display the output the system produced in response to that input. The result would then be saved in a designated file in a designated location as the expected output for that input. Of course, here the simplifying assumptions are that the system is correctly implemented at the time of creation; and that it's possible to find out whether that's indeed the case by manually inspecting the output to a given test-input. The second assumption was indeed true. The first one was also practically true with a bit of care taken in inspecting the outputs during the test-case generation. The criterion I used for test-case generation was `intuition!'
Then, I created another shell-script that played the role of the actual test-harness. Given a list of test-cases, it would pick those test-cases one by one from the prespecified location where the sample inputs were stored. It would feed them to the mobilewp program and dump the outputs into a predesignated output directory. All that done, it would finally diff between the expected outputs and the corresponding actual outputs. If no difference was found, the test-case passed. Else, it would be verdicted as failed.
A pretty trivial system it was. The test data was generated by -- as I mentioned -- mere intuition. The initial check done on what was the expected output too was done manually. The test verdict was passed by mere diff. However, there were some very good things about it. It took me less than a day's effort to write all the test cases. Though their generation was informal, it happened alongside the system development. This gave them an intuitive penetration that's possible only while the system is being developed; and the developer has the best idea at that time what's expected of the system, and what's right. The automatic execution of all those test cases takes just about a minute. At a point the test suite thus created was quite complete. During further few days' of development, the process of incorporating new features was comparably less painstaking than otherwise. It would take just a minute to run all the test cases automatically after incorporation of every new feature. Almost invariably at the first run a test case would fail. The set of failing test cases would easily give an insight as to what had gone wrong. It was quite easy! It worked! And it took just that getting hands dirty to get included in the league of supporters of automation of software-testing. It's not enough to show that it's required to do good testing; it's equally important to show that it pays to do so.
As a parting note, I would just like to point out that the above exercise qualifies as a black-box type of testing. The test data generation was manual. The test execution and evaluation of test results was automatic. The specification was not formal. It wasn't even informal. In fact it was implicit residing only in the mind.
Related blogs:
No comments:
Post a Comment