V-Concept of Testing - Points
to Ponder on Agile
Impact – Agile does NOT have a cost/schedule
advantage over a typical waterfall model, assuming
that the requirements remain stable over the duration
of the project. Where Agile makes a difference
Iterative development addresses the impact of
changing requirements far better than waterfall.
Reduces the risk of severe defects during customer
review by having multiple iterations and getting
continuous feedback from the customer.
Continuous integration and validation mean that
defect fixes and minor requirement changes during
development can be accommodated within the iteration.
2. Iterative Development
– Split the dev phase into multiple iterations,
depending on the total effort and risk of application
Given our resource situation, iterations should
ideally not be more than 8-12 weeks
The first iteration while focusing on the framework
and infrastructure components should include at
least some UI intensive scenarios, to facilitate
early feedback from the client.
Test-driven design and programming – Agile
depends heavily on the premise that the developed
code is well-tested. The programmers should write
code that is easily testable from an object oriented
Automated and Complete Unit Testing – Code
coming out of the developer’s workshop should
have undergone complete unit testing amounting
to 100% of achievable code coverage (who certifies
this?). Emphasis is also put on an automated unit
test framework, so that multiple iterations can
be executed with minimal additional effort. Whenever
a new code unit is tested, test previously released
units as a sort of regression.
Pair Programming – Can be explored based
on cost & availability of skilled resources.
4. Continuous Integration
Frequent incremental integration as opposed to
big bang integration.
i. Eliminates high risk integration issues which
may lie hidden till the end in a big bang integration
ii. Quick turn around time for integration defects
– dev or testing is not held up for long
Unit Tested Code – Only 100% unit tested
code qualifies for continuous integration. Any
integration issues should NOT be due to code defects
missed during unit testing
Frequency – Varies depending on the maturity
of team, complexity of requirements, % of UI component
in application etc. In our scenario, the recommended
target would be at least two integrations a week,
after the initial build on the integration box.
Validation – Each of the cone integrations
shall be smoke tested, and milestones shall be
– A small team of developers will work closely
with QA for managing the continuous integration
process, and for test support. The responsibilities
of the team will be
Controlling the contents of each integration
Sorting out integration issues
Fixing defects found during testing.
Agile relies heavily on “nipping defects
in the bud”.
Agile does not define a single, intensive integration
testing phase, for the simple reason that there
is no “integration phase”.
How do we validate the continuously integrated
build then? – Using the system test plan
instead, to test minor milestones
i. Milestones – Define milestones that
comprise of a small number of related functionalities,
which can be tested independently. Ideally the
cone integrations and milestones should be planned
in such a way that every three or four cone
integrations should add up to a milestone.
ii. The system test plan for a milestone should
be prepared and reviewed at all levels before
the milestone is reached.
iii. At the end of 2 or 3 milestones, do a full
regression of the previous milestones.
Defect fixes – Fixes for defects encountered
during the system testing should be fixed during
the integration for the next milestone, before
a lot more functionality gets added on top of
the buggy code.
This ensures that by the time the application
(or the iteration) reaches QA, the number of defects
7. QA Involvement
– QA Team gets involved from requirements
An experienced QA resource shall be involved
during initial requirements gathering and take
control of requirement management from the QA
From the end of the initial requirements phase,
the QA plan preparation will go hand in hand
with design. The QA plan should be granular
in the extreme, containing all of the system
or integration test scenarios in detail.
QA will be responsible for validation the milestones
during continuous integration.
The dev team should aim for at least 50% mix
of J2EE resources with quality experience in
executing web projects, and usage of tools identified.
The QA team should have 50% of its resources
well experienced in requirements handling, preparation
and management of QA plan from requirements/design,
and Agile Testing processes.