If Fail, skip the rest of the steps?

Suppose we have a product that has 3 items and 4 functions. Each functions are applicable to each items. So, it will be like:

Item 1 – Feature W
Item 1 – Feature X
Item 1 – Feature Y
Item 1 – Feature Z
Item 2 – Feature W
Item 2 – Feature X
…and so on…

How should the test cases be written? We can have 1 big test case for an item, like this:

Test Case 1 – Features in Item 1

1. Test Feature W in Item 1
2. Test Feature X in Item 1
3. and so on…

Or 1 big test case for each function, like this:

Test case 1 – Feature W

1. Test Feature W in Item 1
2. Test Feature W in Item 2
3. and so on…

Then, suddenly a bug is discovered for Feature W. What happens next is, if we use Approach A, all test cases for Item 1, 2, and 3 will fail, because all test cases consists of testing Feature W. But if we use Approach B, only 1 test case, i.e. test case for Feature W only will fail.

But, how do we know either it’s an item, or a feature that will fail?

Also, to add salts onto the wound, there’s a possible tendency for testers to rush up testing, will lead them to skip the testing of the rest of the test cases that are not failing. For instance, if a tester found out that a bug can fail a third step of the test case, he/she may just skip executing the rest of the steps and just failed the test case. That’s why it’s no good to rush our testers to complete their tests quick :P

Of course, some may argue, maybe it’s better to use Approach C instead:

Test Case 1 – Feature W in Item 1
Test Case 2 – Feature X in Item 1
Test Case 3 – Feature Y in Item 1

That will translate to much higher number of test cases, i.e. number of features x numbers of item!

So, the remedy? Use Approach C if you want.

Or, what we can do is, make sure that the testers still execute the rest of the test steps, although the first part of the test is failing due to the discovered bugs.

How do we know the tester still execute the rest of the test steps? Relatively, we can judge from the execution time. That is, if your team have it recorded. If the execution time is much much shorter, than an average execution time of that same test, that could be something fishy.

Another positive reinforcement would be to motivate the testers themselves, about how major a bug that they could miss if they just fail a test because of a previous bug, for the sake of completing their tasks for the day.

That also means, the management should not be too pushy about us completing our tests faster.

Fast, cheap, good. These three attributes don't co-exist together. You can pick any two of them, but not all three.

What does a Tester do, actually?

What does a Test Engineer or QA person do actually? It's not as popular as software developers or coders. I myself never heard of this job description before deciding what courses to take during my college years. It's only until several months before I saw openings in J*bStreet when doing my job-hunting. Even when I decided to take up the job with the Company M, I'm still not clear about the job. At that time, like most of the graduates, the aim is 'to get some experience first'.

[Yeah, we can talk a lot about the importance of pursuing your dreams, make sure the job fits yourself, dream job, satisfaction, yada yada yada. Yeah right, tell that to employers nowadays. Fresh graduates look for experience, not satisfaction.]

But, as my days went on at the Company M, it's not such a bad decision after all. I got to know a different world, that actually revolves around software development, but not the development itself. Testing has never occurs to my mind as a career. Ask the students nowadays, how many of them actually know what does a tester do? Or actually even heard of it? I myself having hard time explaining to others.

[Come to think of it, I'll tell them, "I find other people's fault for a living...Heheh...]

Software development life cycle, or in its short form, SDLC, simply put, are stages to develop software, first, by designing the software, then you specify requirements about it, then you write it, and at the end, you test it before it got released to the customers.

Those were the days when not that many companies really consider the last stage of the life cycle as important - testing. Developers do the testing themselves, or even worse, no testing at all. No quality check, na da.

It could be because of several reasons actually. The major one is that the lack of fund to get a dedicated resources to test. It's not just the human, it also includes the testing tools, and the times needed. Why, just be satisfied with some tests that developers already run.

It could also be that the testing stages are deemed not to be worthy of it. I got to know about a company, a big one, that don't really do proper quality check on their products, their excuse being that the product market's lifetime is very short. By the time the consumers realized that their products have certain defects, at that time an average consumer may already buy the next version of that product. So, the management calculated the ROI, that it's not worth it to spend capital on testing, rather just wait for the customers to buy the updated version. No wonder the software in their products suck!

After I did my time in Company M, and moved on, I started seeing a bigger picture. Testing is not just applicable to the software or hardware actually. It's actually exist in different working worlds. It could have different names, with different job description, but in the end, they converge to similar concepts. Tester, Validation Engineer, Quality Assurance, Quality Control, it even goes on to Audits, MQA, S*RIM, J*KIM, and so on.

If we really think about it, there are always some sorts of checking going around in our daily life. As you type on your laptop, who qualify it that it will not explode in front of you? As you drive to your work today, who make sure that the car passes certain ratings that it is safe to be use? As you eat at your favorite mamak stall, which department certify that the ingredients are halal (if you're Muslim) or the hygiene level is A or B?

There are always parties that will watch for errors, look for faults, and hunt for bugs.

Notepad++

I have to admit I’m quite left behind in technology and tools up-to-date. So, I just knew about Notepad++ when I join this Company E. Good things about Notepad++:

- colourful syntax that follows the language of the file

- multiple tabs of files can be opened

- last saved memory, i.e. we can just close the Notepad while working on a file, and it’s there when we re-open it. No annoying auto-recovery stuff.

- And so on….

What I like most, are the ability to open in a different views, i.e. Move to Other View. This is damn helpful when I need to see two documents side by side.

And to add, Plug-ins > Compare, allows me to compare the documents viewed side-by-side and quickly identify the differences in colours. Talk about Documentation Test!

Importance of recording setup time

QA Desk, 6.18 pm.

I’m still at my desk, adding up some test cases to the test plan, for the new component that we need to integrate. The whole day is pretty much spent for setting up the environment. Come to think of it, can we say that the majority of times are spent prior to the test to setup the environment the testing itself?

When I was in Company M, in our Test Management System (which is developed in-house by Company M itself), we need to enter execution time, investigation time, and setup time for certain test cases. Then at the end of the period, let say after the project finishes, post-mortem perhaps, the team lead will go into the system and start mining the data to see if there is any spike on setup time, execution time, or investigation time. From there, improvements can be suggested, like:

- Having a dedicated environment
Saves time to setup each time the same test is needed, especially for new hires. However, it will constrain the resources to be used up for other activities. The dedicated environment can sometimes be sacrificed to be dissembled for that purpose. My personal case, is when we built up some environments, they only stay intact for awhile, until the stringent time comes, that everybody starts dismantling them for other uses.

- Clear written test cases
How many times do we face problems when the test cases are not clearly mention some settings or some features need to be enabled for the test? What can be done is that the tester should be encouraged to update the test cases every time they found something useful to be added

Like what some people said, if you can’t measure it, you can’t improve it.

In Company E, it is not required yet to record the setup time. Would be good eh, if I suggest this to my boss?

hello world

Test "Hello"
Test "World"