Saturday, June 5, 2010

Methodology - Agile Testing Approach

What does Agile Means:
  • Being Agile means creating an environment where change is embraced
  • Changes in requirements, system knowledge, and /or organization are considered part of normal software development
  • Agile is about focused collaboration between customers and developers
  • The main goal is clean code that works, at a minimum investment
Why Agile: Reference: [AgileManifesto01] “Individuals and interactions over processes and tools Working software over comprehensive documentation. Customer collaboration over contract negotiation. Responding to change over following a plan. That is, while there is value in the items on. the right, we value the items on the left more”
 
When does Agile Makes sense:
  • Rapidly develop and deliver software that
  • meets business needs in the face of an
  • unpredictable future
  • Ability to change your mind
  • Early availability of business value
  • Option to cancel at any time yet still have value
  • Handle uncertainty
  • Visibility to & feedback from the customer 
When does Agile makes no sense:
  • Uncertain and/or unstable success criteria
  • Uncertain and/or unstable requirements
  • Immediate access to business expertise
  • High value in early release of partial functionality
  • Project can be “self-contained”
  When does Less Agility makes sense:
  • Knowable, stable project goals and requirements
  • Like version 3 of a well-established application
  • Customer can't absorb frequent releases
  • High effort of data conversion, user training, …
  • Team is too large, dispersed, or high turnover
  • Poor team dynamics
  • Lack of customer representation and/or poor
  • customer relationship
  • Dependence on external supplier's
  • Long-term maintainability is important
  • Life- or mission-critical products
There's more than one way to be Agile: 
  • Extreme Programming
  • Scrum
  • Crystal
  • Agile Modeling
  • Dynamic Systems Development Method
  • Feature-Driven Development
  • Adaptive Software Development
Is Agile Appropriate:
  • §Learning objective
  • §Practice evaluating projects to see why they would or
  • §wouldn’t be good candidates for agile development
  • §Procedure
  • §Read the Project Profiles in the Labs section
  • §Discuss the projects in your groups
  • §What characteristics of the projects make them good
  • §candidates to be run as agile projects?
  • §What characteristics make them not so good
  • §candidates to be agile projects?
  • §Extra Credit: Analyze your current project's 
Test A Little Code A Little:
  • §You don’t need to write all the test cases first
    •1. You have to create of one test that the current code won’t pass
    •2. Write code to pass the test
    •3. Go back to step 1
  • §If it is hard to write a test there may be a design issue
  • §Each round of test & code includes refactoring of the previous code
Frequent Build and Test:
  • §Completely build the product and run it through a series of tests to verify basic operation
  • §“Daily build and smoke test”
  • §“Continuous integration”
  • §Works well with large volume projects
  • §Lowering integration risk
  • §Exposing big integration errors quickly
  • §Tracking project progress
  • §Extra effort/overhead
 How Frequent:
  • §Not more than a couple of hours between builds
  • §Build should take 10 minutes
  • §Asynchronous builds most common
  • §Check stuff in, the automatic build process
  • §notices, sends out email of result
  • §Synchronous builds give team a chance to reflect
  • §Builds scheduled at certain times for all programmers
Multiple Builds:
  • §Different builds accomplish different goals
  • §Short, small builds on the test first approach
  • §Overnight builds to test full integration
  • §Weekly run more complete tests
  • §Automation here is key
  • §Agile sees this as a worthy area to spend
     
 Test Coverage Criteria:
  • §How do I have confidence that my set of test cases is “good enough”?
  • §Input and output domain coverage
  • §Boundary value testing
  • Decision (Branch) coverage
  • Agile Testing good for Agile development projects;  can work on conventional projects too.
  • Don’t implement Agile testing as a whole, but adapt its most valuable concepts
  • Testers should be active members of the team.
  • Testers often hesitate to speak up during team meetings, share ideas, interact with developers.
  • Agile only works when there is lots of both formal and informal communication within a team.
Ongoing Conversation concept.
§With developers
§With CSRs
§Business Analysts.

Testers and developers are working together.
§Share office
§Testers assigned to help particular developers
§Test plan evolves  through a series of short “drop-in” meetings
§Benefits of pairing with developers:
     - developers gain insight into potential errors
     - testers gain insight into constraints and opportunities.
     - together can advance automated testing.
     - Grey box:  - understand relationship btw parts of the systems. 
                         - understand bugs:  what were the root causes?
                         - what needs to be retested?
                    
Test Status is reported via simple to read charts that answer specific question “What part of code can we stop worrying about?.”
Stand up reports.

Conversation with a customer is as important as with developers.
–Remember that customer are trying to figure out what they need, want and are getting, in large part, by trying out the  working code.
–Get feedback from CSRs -> create tests
–Participate in usability testing
Remember: “ …thinking does not take place inside our heads … Most of our mental activities need external resources.”
 
Keep in mind that sometimes design docs can be flawed.
Documents can’t be an adequate representation of the working code.
View them as:  interesting texts, partly fictional, often useful
Active communication is the key to a successful test plan

First, Waste No Time. Evaluate risky areas and test them first
     -  We gain an understanding of the product we're testing to the point where we can imagine what kinds of problems are more likely to happen and what problems would have more impact if they happened. Then we put most of our effort into testing for those problems. Agile Testing is concerned with uncovering the most important information as soon as possible.
Mission. What's more important? Writing test cases or doing testing?
     -  In Agile Testing we don't start with a task ("write test cases"), we start with a mission. Our mission may be "find important problems fast". If so, then writing test cases may not be the best approach to the test process. If, on the other hand, our mission is "please the FDA auditors", then we not only will have to write test cases, we'll have to write certain kinds of test cases and present them in a specifically approved format.  In our case it’s both.
 
Skills.
      -  Knowing the product lets you perform “out of the box” testing
Exploration.   
     -  Use exploratory testing. We prefer to let the next test we do be influenced by the last test we did. This is a good thing, because we're not imprisoned by pre-conceived notions about what we should test, but let ourselves develop better ideas as we go.
Teaming.   
     -  Agile Testers cheat. That is, they do things that our former elementary school teachers would consider cheating: we figure things out in advance where possible, we borrow other people's work, we use every resource and tool we can find. Example:  Pair testing: two heads, one computer. This idea has proven valuable in the practice of Extreme Programming, and it works for testing just as well. In our experience of normal testing, testers usually work quietly and alone, rather than hunting for bugs like a pack of rapid wolves. Again the message of this presentation is not  to try to adapt Agile testing as a whole, but extract and implement the most useful techniques of Agile testing.
Some of the stuff is being done already.
    -  Testers have stand up meetings
    -  Some Interaction with CSRs (manager, team leads)
What we don’t do yet:
    -  Closer interaction with developers.
    -  Closer interaction with CSRs. (primary testers)
    -  Robust testing, “out-of-the-box” thinking. (for primary testers)
    -  Grey box:  understanding of the system.
    -  Automation:  combine unit tests and our automation tests.
    - Testing in pairs. 

Increase quality of the product.  Find bugs faster.
Get more exposure for QA
Get better understanding of components.
Interact with developers more, which is always good.
Get better understanding of the working code.
Deliver marketable product earlier