COMPUTER SYSTEMS RESEARCH
Software Testing 4 - Requirements & Specifications
3rd Quarter 2006
- Also see the Testing Report worksheet to attach your results

  • Resources:

    Defining requirements and the specifications for verifying these requirements for your program

    1. Need for requirements and specifications
      	"Defining requirements to establish specifications is the first step in the development
      	of a system. However, in many situations, not enough care is taken in establishing 
      	correct requirements up front. This causes problems when ambiguities in requirements 
      	surface later in the life cycle, and more time and money is spent in fixing these 
      	ambiguities. 
      	
      	Therefore, it is necessary that requirements are established in a systematic way 
      	to ensure their accuracy and completeness, but this is not always an easy task. 
      	This difficulty in establishing good requirements often makes it more of an art than
      	a science. 
      	
      	The difficulty arises from the fact that establishing requirements is a tough abstraction 
      	problem and often the implementation gets mixed with the requirements. In addition, 
      	it requires people with both communication and technical skills. 
      	
      	As requirements are often weak about what a system should not do, this poses potential 
      	problems in the development of dependable systems, where these requirements are 
      	necessary to ensure that the system does not enter an undefined state."
      
      	Requirements and Specifications
      
      
    2. Static analysis techniques
      	"The most important characteristic of static analysis techniques is that the 
      	testing does not necessitate the execution of the program (Hausen84, 325). 
      	``Essential functions of static analysis are checking whether representations and 
      	descriptions of software are consistent, noncontradictory or unambiguous'' (Hausen84, 325). 
      	
      	It aims at correct descriptions, specifications and representations of software 
      	systems and is therefore a precondition to any further testing exercise.
      	
      	Static analysis covers the lexical analysis of the program syntax and investigates 
      	and checks the structure and usage of the individual statements (Sneed87, 10.3-3). 
      	
      	There are principally three different possibilities of program testing (Sneed87, 10.3-3), 
      
      	   1. program internally, to check completeness and consistency
      	   2. considering pre-defined rules
      	   3. comparing the program with its specification or documentation "
      
      	   Glass box testing - static analysis
      
    3. Distinction between requirements and specifications
      	"There is a distinct difference between requirements and specifications. 
      	A requirement is a condition needed by a user to solve a problem or achieve an objective. 
      	A specification is a document that specifies, in a complete, precise, verifiable manner, 
      	the requirements, design, behavior, or other characteristics of a system, and often, 
      	the procedures for determining whether these provisions have been satisfied. 
      	
      	For example, a requirement for a car could be that the maximum speed to be at least 
      	120mph. The specification for this requirement would include technical information 
      	about specific design aspects. 
      	
      	Another term that is commonly seen is requirements specification, which is a 
      	document that specifies the requirements for a system or component. 
      	It includes functional requirements, performance requirements, interface requirements, 
      	design requirements, and developement standards."
      
      	Requirements and specifications
      
    4. Step one: Establish correct requirements
      	"The first step toward developing accurate and complete specifications is to 
      	establish correct requirements. As easy as this sounds, establishing correct requirements 
      	is extremely difficult and is more of an art than a science. 
      	
      	There are different steps one can take toward establishing correct requirements. 
      	Although some of the suggestions sound fairly obvious, actually puttting them into 
      	practice may not be as easy as it sounds. 
      	
      	The first step is to negotiate a common understanding. There is a quote by John von Neumann 
      	that states "There's no sense being exact about something if you don't even know 
      	what you're talking about." [Gause89] Communication between the designer and customer 
      	is vital. There is no point in trying to establish exact specifications if the 
      	designers and customers cannot even agree on what the requirements are.
      
      	Requirements and specifications
      
    5. How to begin establishing correct requirements - Asking questions
      	Often the problem one has in establishing correct requirements is how to get started. 
      	One of the most important things in getting started is to ask questions. 
      	Context-free questions are high-level questions that are posed early in a project 
      	to obtain information about global properties of the design problem and potential solutions. 
      	Examples of context-free questions include who is the client?, what is the reason 
      	for solving this problem?, what environment is this product likely to encounter?, 
      	and what is the trade-off between time and value?. 
      	
      	These questions force both sides, designer and customer, to look at the higher issues.
      	Also, since these questions are appropriate for any project, they can be prepared in 
      	advance. 
      	
      	Another important point is to get the right people involved. There is no point in 
      	discussing requirements if the appropriate people are not involved in the discussion. 
      	Related to getting the right people involved is making meetings work. Having effective 
      	meetings is not as easy as it sounds. However, since they play a central role in 
      	establishing requirements it is essential to know how to make meetings work. There 
      	are important points to keep in mind when creating effective meetings, which include 
      	creating a culture of safety for all participants, keeping the meeting to an 
      	appropriate size, and other points. [Gause89]
      
    6. How to begin establishing correct requirements - Ideas, brainstorming
      	Exploring the possibilities is another important step toward generating correct 
      	requirements. Ideas are essential in establishing correct requirements, so it is 
      	important that people can get together and generate ideas. Every project will 
      	also encounter conflicts. Conflicts can occur from personality clashes, people that 
      	cannot get along, intergroup prejudice such as those between technical people and 
      	marketing people, and level differences. It is important that a facilitator is 
      	present to help resolve conflicts.
      
    7. How to begin establishing correct requirements - establish specific expectations of the project.
      	In establishing requirements, it is important to specifically establish the functions, 
      	attributes, constraints, preferences, and expectations of the product. Usually in 
      	the process of gaining information, functions are the first ones to be defined. 
      	
      	Functions describe what the product is going to accomplish. It is also important to 
      	determine the attributes of a product. Attributes are characteristics desired by 
      	the client, and while 2 products can have similar functions, they can have competely 
      	different attributes. After all the attributes have been clarified and attached to 
      	functions, we must determine the constraints on each of the atrributes. 
      	
      	Preferences, which is a desirable but optional condition placed on an attribute, 
      	can also be defined in addition to its constraints. Finally, we must determine what 
      	the client's expectations are. This will largely determine the success of the product.
      
    8. Establishing correct requirements - testing
      	Testing is the final step on the road to establishing correct requirements. There 
      	are several testing methods used, as listed below. [Gause89] 
      
      	*  Ambiguity poll - Used to estimate the ambiguity in a requirement. This involves 
      	asking questions such as how fast?, how big?, how expensive?, and then determining 
      	if there is ambiguity between the high and low values.
      	
          	* Technical review - A testing tool for indicating the progress of the requirements 
      	work. It can be formal or informal and generally only deals with technical issues. 
      	Technical reviews are necessary because it is not possible to produce error-free 
      	requirements and usually it is difficult for the producers to see their own mistakes.
          	
      	* User satisfaction test - A test used on a regular basis to determine if a customer 
      	will be satisifed with a product.
          
          
          	* Black box test cases - Constructed primarily to test the completeness, accuracy, 
      	clarity, and conciseness of the requirements.
      	
      	* Existing products - Useful in determining the desirable and undesirable 
      	characteristics of a new product.
      
      
    9. Is your project a "system"?
      	Systems exist everywhere in the universe we live in. The universe can be considered 
      	a system, and so can an atom. A system is very loosely defined and can be considered 
      	as any of the following definitions. [Blanchard90]
      
      	* A combination of elements forming a complex or unitary whole (i.e. river system 
      	or transportation system)
      	
          	* A set of correlated members (i.e. system of currency)
      	
          	* An ordered and comprehensive assemblage of facts, principles, or doctrines in a 
      	particular field of knowledge (i.e. system of philosophy)
          	
      	* A coordinated body of methods, a complex scheme, or a plan of procedure (i.e. 
      	system of organization and management)
          	
      	* Any regular or special method of plan of procedure (i.e. a system of marking)
      
      
    10. A system's operational requirements
      	A system's operational requirements should include the following elements. [Blanchard90]
      
      	* Mission definition - Identification of the primary operating mission of the 
      	system in addition to alternative and secondary missions.
              
      	* Performance and physical parameters - Definition of the operating characteristics 
      	or functions of the system.
              
      	* Use requirements - Anticipation of the use of the system.
              
      	* Operational deployment or distribution - Identification of transportation and 
      	mobility requirements. Includes quantity of equipment, personnel, etc. and 
      	geographical location.
              
      	* Operational life cycle - Anticipation of the time that the system will be in 
      	operational use.
              
      	* Effectiveness factors - Numbers specified for system requirements. Includes 
      	cost-system effectiveness, mean time between maintenance(MTBM), failure rate, 
      	maintenance downtime, etc.
              
      	* Environment - Definition of the environment in which the system is expected to operate.