Which test case should not be automated?

These types of

test cases should not be automated

1. One Time testing. (Non Repeatable)
2. Ad-hoc Testing (As soon As possible results required)
3. Usability Testing.

Which test case should be automated?

Following

test case should be automated

• Test case need to be run for every build of the application
  (Sanity Testing)
  
• Test that use multiple data values for the same actions.
  (Data Driven Testing)
  
• Critical Application functionality.
• Business functionality which won’t be change in short time period.
• Performance testing
  (Load & Stress)
  

What are disadvantages of automated testing tools?

Automated tools have following disadvantages.

Usability Testing:
Usability of software can not be automated so automated tools cannot rate the usability of an application.

Cost:

 Usually these are licensed so automated tools are quite expensive

Programming knowledge required:
 To customize test scripts according to test requirement.
Test maintenance is costly: In case of playback methods.
Even though a minor change occurs in the GUI, the test script has to be rerecorded or replaced by a new test script.

What-are-benefits-or-advantages-of-automated-testing?

 

Following are the benefits for using automated tools

Fast:

Human be slow but Automated Testing Tool runs tests significantly faster than human users.

Reliable:

Tests perform precisely the same operations each time they are run, thereby eliminating human error.

Repeatable:

You can test how the software reacts under repeated execution of the same operations.

Programmable:

You can program sophisticated tests that bring out hidden information from the application.

Reusability:

You can reuse the Automated Test scripts, user defined functions etc.

How-is-Statistical-Analysis-is-in-TPPM-Activity?

Statistical Analysis

• Perform Problem Root Cause Analysis.

• Mention root cause from different main cause categories

• Identify different solution

• Select best solution

• Mention C&PA to avoid problem in future

How-is-Actual-Measurements-Entry- in-TPPM-Activity?

Take measurement of following

attributes/Variables and enter in Testing PPM

• Actual size of code (LOC/AFPs)

• Actual Defects Identified by each test case

• Actual efforts expended to execute each test case

How-estimation-in-TPPM-(-testing-process-performance-model- activity?

In testing PPM first of all estimate following things step by step

• Size of each Test Case (over all size and lenght of the every  test case is estimated)
• Defects of each Test Case(defects of the test cases are estimated)
• Efforts to execute each test case (Efforts are also are part of the estimates)

What-is-Testing-Process-Performance-Model-(TPPM)-activity?

Testing PPM activity

Testing PPM activity can be logically divided into following Sub-Activities.

• Estimations-here estimations are done
• Actual Measurement Entry - here we come with actual measures
• Statistical Analysis - we are doing analysis for the estimations and measure in this activity

What-is-the-sequence-of-testing-process-performance-model-TPPM?

Sequence of TPPM

• Testing Process Performance model is prepared for each Release testing activity of Project/Product .
• After test designing of each release, TPPM estimation is completed.
• While  test execution, TPPM is updated on daily basis or after some defined interval based on release test execution time.

What-is-testing-process-performance-model-or-TPPM?

Testing Process Performance Model

Testing Process Performance model is to predict the important attributes of testing process and monitor the efficiency and performance of testing process by tracking and analyzing the actual data.

What-is-use-of-process-performance-models-PPMs?

Use Of PPMs

Process Performance models are used to estimate the project activities for different critical sub processes which are important to achieve the project and business objectives.

Mainly PPM are to :

• Predict
• Track
• Manipulate

What-are-PPM-(process-performance-models)?

PPM

Process Performance Models are used to estimate and predict results to be achieved by following a process.

There are following process performance models.

• Effort Estimation Model
• Defect Estimation Model
• Time and Effort PPM
• Release Management PPM
• Patch Release PPM
• Testing PPM

How analysis and reporting after data collection?

There are a few steps for analysis and reporting.

• Periodically review the actual results
• Apply quantitative and statistical techniques
• Determine and document actions needed to address the deficiencies
• The results of data analysis are reviewed with affected groups & individuals

• Project Summary
• Analysis Reports

How data is collected for metrics and measurements?

There are some ways for metrics and measurements for data

• Measurements are collected as per their definition at defined points as mentioned in Organization Metrics File
• Data is collected and compiled by using Project Summary, Help Desk, eRMS, Analysis Reports, Peer Review Analysis Reports etc
• Verification of the measurement data is done independently to check completeness, integrity, accuracy by SQA & SEG for the data that they provide to each other
• Once the data is collected will be assessed independently by QE
• Project may consider additional metrics to better manage their project
• Data should be available for use only by appropriate groups and personnel.

What are the business objective of metrics and measurements?

These are business objectives

• Project must be signed-off on time with maximum 15% schedule variance.
• Estimation Accuracy (EA) of Effort, & Cost must be within 0.85 to 1.15.
• 100 % of signed off requirements should be implemented in final delivery.
• After system sign-off re-work effort on bugs must not exceed 5% of total effort on development per year.
• At least 80% of resource and time utilization must be ensured for each resource.
• Achieve at least 20% profit on average for each project.
• Employee turnover should not be more than 8% during a year.
• Exploration of two technologies during a year, and one of them must be used.
• At least 90% of Critical/Major bug requests should be serviced as per agreement during a year (as per PM’s discretion that change request is Critical/Major).
• At least 10% of the total leads should be materialized by Business Development during the year.
• Employees satisfaction index (ESI)
• Conduct at least one market survey per year

What are the business goals of metrics and measurements?

We may say that these are its main goals.

• Continuously enhance the quality of our products & services
• Increase customer satisfaction leading to repeat business
• Attract, develop & retain competent & motivated resource base by ensuring highest level of employee satisfaction
• Be ready to meet market expectations by being proactive and having in-depth understanding of market requirements
• To be the world leader in providing - - - - software solutions
• Become a preferred vendor of - - - - software solutions by strengthening customer base in existing markets and aggressively exploring new markets
• To become a profitable organization

What are the attributes for effective software metrics?

 

Attributes effective software metrics

• Simple and computable
• Consistent and objective
• Consistent in its use of units and dimensions
• Programming language independent
• An effective mechanism for high-quality feedback

What are the activities that are suggesting measurement principles?

Activities are

• Formulation
• Collection
• Analysis
• Interpretation
• Feedback

Principles for Formulation

• The objectives of measurement should be established before data collection begins.
• Each technical metric should be defined in an unambiguous manner.
• Metrics should be derived based on a theory that is valid for the domain of application.
• Metrics should be tailored to best accommodate specific products and processes.

Principles for Collection and Analysis

• Whenever possible, data collection and analysis should be automated.
• Valid statistical techniques should be applied to establish relationships between internal product attributes and external quality characteristics.
• Interpretative guidelines and recommendations should be established for each metric.

What is the relationship of Metrics and Measurement and Quality?

Here are the points to be noted

• Software requirements are the foundation from which quality is measured. Lack of conformance to requirements is lack of quality.
• Specified standards define a set of development criteria that guide the manner in which software is engineered. If the criteria are not followed, lack of quality will almost surely result.
• There is a set of implicit requirements that often goes unmentioned (e.g., the desire for ease of use). If software conforms to its explicit requirements but fails to meet implicit requirements, software quality is suspect.

So there is relationship. As we have measurement correct we will have solid requirements and on that at the quality will be fine

How do ensure that Metrics and mesasurements are correct?

There are two things to do for ensuring that these are correct.

• Establish the objectives of measurement before data collection begins, defining each technical metric in an unambiguous manner.
• Define a few metrics and then use them to gain insight into the quality of a software engineering work product.

What are the main steps of metric and measurement?

Here are the main steps.

• Derive the software measures and metrics that are appropriate for the representation of software.
• Data required to derive the formulated metrics are collected.
• Once computed, appropriate metrics are analyzed on pre-established guidelines and past data.
• The results of the analysis are interpreted to gain insight into the quality of the software, and the results of the interpretation lead to modification of work products arising out of analysis, design, code, or test.

What is software metric?

Software Metric

Software metric is measure of some property of a piece of software or its specifications.

It provides with quantitative information to make the process better by analyzing that information.