Geek4Eva

Earlier today jut exchanged few messages with Michael Bolton over Twitter. Here is a quick tip from him on when to automate the testing for a ‘broken link’.

@michaelbolton	I told @mhesusser about a broken link on his blog. He fixed it, and tried it. I tried it too. Do we still need an automated check for that? about 8 hours ago from TweetDeck
@Geek4Eva_	@michaelbolton Yes if it is expected to be an ongoing issue. about 9 hours ago from TweetDeck in reply to michaelbolton
@michaelbolton	@Geek4Eva_ And if not? about 8 hours ago from TweetDeck in reply to Geek4Eva_
@Geek4Eva_	@michaelbolton Tricky question. I’d say yes – as thats what regression tests are for. about 8 hours ago from TweetDeck in reply to michaelbolton
@Geek4Eva_	@michaelbolton … To prove that already working stuff is not broken. about 8 hours ago from TweetDeck
@michaelbolton	Here’s what (I say) to ask about in terms of the automated check for broken link issue:1) risk 2) cost 3) value 4) time 5) skill 6) priority about 7 hours ago from TweetDeck

Haven’t really had a first-hand experience of Test Driven Development, though have come across some real promoters of it.

The two most common push back that I hear from developers or their managers for not being able to implement this are:

1 – It takes a while to implement as in training developers and them into the habit.
2 – The extra cost that is involved in creating those Tests

Image Source

I’d say both these factors are quite subjective and it’d be difficult to estimate the cost for them.

But Miško Hevery, an Agile coach and developer at Google has gone ahead and tried to estimate the Cost of Testing. And he has estimated an overhead of 10% for writing tests for any piece of code.

So a naive answer is that writing test carries a 10% tax. But, we pay taxes in order to get something in return. Here is what I get for 10% which pays me back:

* When I implement a feature I don’t have to start up the whole application and click several pages until I get to page to verify that a feature works. In this case it means that I don’t have to refreshing the browser, waiting for it to load a dataset and then typing some test data and manually asserting that I got what I expected. This is immediate payback in time saved!
* Regression is almost nil. Whenever you are adding new feature you are running the risk of breaking something other then what you are working on immediately (since you are not working on it you are not actively testing it). At least once a day I have a what the @#$% moment when a change suddenly breaks a test at the opposite end of the codebase which I did not expect, and I count my lucky stars. This is worth a lot of time spent when you discover that a feature you thought was working no longer is, and by this time you have forgotten how the feature is implemented.
* Cognitive load is greatly reduced since I don’t have to keep all of the assumptions about the software in my head, this makes it really easy to switch tasks or to come back to a task after a meeting, good night sleep or a weekend.
* I can refactor the code at will, keeping it from becoming stagnant, and hard to understand. This is a huge problem on large projects, where the code works, but it is really ugly and everyone is afraid to touch it. This is worth money tomorrow to keep you going.

Sounds promising! I am quite keen to discuss this with my development manager – what are their plans on this?

Here’s a quick and simple overview of the main differences between Waterfall Development, Iterative Waterfall Development, Scrum/Agile Development and Lean by Agile101.net

Waterfall Development

‘Waterfall Development’ is another name for the more traditional approach to software development.

It’s called ‘waterfall’ as this type of development is often planned using a Gantt chart – you complete one phase (e.g. planning) before moving on to the next phase (e.g. development).

In Waterfall approaches, you will rarely aim to re-visit a ‘phase’ once it’s completed. As such, you better get whatever you’re doing right the first time!

This approach is highly risky, often more costly and generally less efficient than more Agile approaches.

The main issues with this approach include:

• You don’t realise any value until the end of the project (when you deploy) (See: Self-Funding Projects, a Benefit of Agile Software Development)
• You leave the testing until the end, which means you’re leaving issue discovery until late in the day
• You don’t seek approval from the stakeholders until late in the day – their requirements might have changed
• You’re heavily reliant upon a plan, which you can/will often follow to the detriment of the end result
• You’re heavily reliant upon a project manager driving the way – the power of one

Iterative Waterfall Development

This approach carries less risk than a traditional Waterfall approach but is still far more risky and less efficient than a more Agile approaches.

The focus is on delivering a sprint of work as opposed to a series of valuable/shippable features.

The most commonly occurring issue in this type of scenario (in my experience) is bottle necking.

For example, you deliver loads of code a little bit behind schedule (?) and you leave it until the last minute to test everything. One issue takes longer than expected to resolve, you miss your sprint deadline and you deliver nothing.

Another common symptom of this type of approach is over-commitment.  It’s really difficult to estimate the total effort associated with a particular User Story/Feature when approaching delivery in this phased way.

You’re more or less forced to estimate each phase separately (e.g. estimate development separately to testing in this instance) – this doesn’t work as the phases are not separate, they’re totally intertwined.

For example, if you find an issue with the test, you must return to development. The whole team must remain focused on delivering the end goal, not the separate phases.

It’s also worth noting that velocity and burn downs are far less (if at all) useful in this type of environment – you don’t benefit from early-warning-signs as you don’t find out whether you’re on track until the end of the sprint.

Scrum Development

This approach carries far less risk than Waterfall approaches.

We focus on delivering fully-tested, independent, valuable, small features. As such, we diversify our risk – if one feature goes wrong, it should not impact another feature.

With that said, we still plan our work in iterations and we will still release at the end of each iteration.

Lean Development

Lean is very similar to Scrum in the sense that we focus on features as opposed to groups of features – however Lean takes this one step further again.

In Lean Development, you select, plan develop, test and deploy one feature (in its simplest form) before you select, plan, develop, test and deploy the next feature. By doing this, you further isolate risk to a feature-level.

In these environments, you aim to eliminate ‘waste’ wherever possible – you therefore do nothing until you know it’s necessary or relevant.

Michael Bolton is one of the well respected and experienced software testing guru around. And he has posted this series of Testing vs Checking articles on his blog. In one of the introductory posts he has mentioned:

Image Source

Checking Is Confirmation

Checking is something that we do with the motivation of confirming existing beliefs. Checking is a process of confirmation, verification, and validation.

Testing Is Exploration and Learning

Testing is something that we do with the motivation of finding new information. Testing is a process of exploration, discovery, investigation, and learning. When we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated, we’re testing

Checks Are Machine-Decidable; Tests Require Sapience

A check provides a binary result—true or false, yes or no. Checking is all about asking and answering the question “Does this assertion pass or fail?” Such simple assertions tend to be machine-decidable and are, in and of themselves, value-neutral.

A test has an open-ended result. Testing is about asking and answering the question “Is there a problem here?” That kind of decision requires the application of many human observations combined with many value judgements.

Testing Is Not Quality Assurance, But Checking Might Be

You can assure the quality of something over which you have control; that is, you can provide some level of assurance to some degree that it fulfills some requirement, and you can accept responsiblity if it does not fulfill that requirement. If you don’t have authority to change something, you can’t assure its quality, although you can evaluate it and report on what you’ve found. (See pages 6 and 7 of this paper, in which Cem Kaner explains the distinction between testing and quality assurance and cites Johanna Rothman’s excellent set of questions that help to make the distinction.) Testing is not quality assurance, but acts in service to it; we supply information to programmers and managers who have the authority to make decisions about the project.

Checking, when done by a programmer, is mostly a quality assurance practice. When an programmer writes code, he checks his work. He might do this by running it directly and observing the results, or observing the behaviour of the code under the debugger, but often he writes a set of routines that exercise the code and perform some assertions on it. We call these unit “tests”, but they’re really checks, since the idea is to confirm existing knowledge. In this context, finding new information would be considered a surprise, and typically an unpleasant one. A failing check prompts the programmer to change the code to make it work the way he expects. That’s the quality assurance angle: a programmer helps to assure the quality of his work by checking it.

Testing, the search for new information, is not a quality assurance practice per se. Instead, testing informs quality assurance. Testing, to paraphrase Jerry Weinberg, is gathering information with the intention of informing a decision, or as James Bach says, “questioning a product in order to evaluate it.” Evaluation of a product doesn’t assure its quality, but it can inform decisions that will have an impact on quality. Testing might involve a good deal of checking; I’ll discuss that at more length below.

Checkers Require Specifications; Testers Do Not

A tester, as Jerry Weinberg said, is “someone who knows that things can be different”. As testers, it’s our job to discover information; often that information is in terms of inconsistencies between what people think and what’s true in reality. (Cem Kaner’s definition of testing covers this nicely: “testing is an empirical, technical investigation of a product, done on behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek.”)

As a result there are some really ctitical comments made by readers in there.

Personally I liked the initial idea but I need to ponder a bit more on it to come up with my opinion about it.

I haven’t been able to read all of it as yet, but if you are interested than here are the links to complete Testing vs. Checking Series
Testing vs. Checking
Transpection and the Three Elements of Checking
Pass vs. Fail vs. Is There a Problem Here?
Elements of Testing and Checking
Testing, Checking, and Changing the Language
Tests vs. Checks: The Motive for Distinguishing

Enjoy!!

Here are couple of exampes of what the testers at my workplace like to do:

1 - This one is to remind developers to not spread bugs:

2 - This image is from the testing area which has similar bugs all over the place. To remind passerbys of what testers do – Find Bugs:

I have seen some really blunt testers in my IT career. Blunt as in, if they have a problem they’ll make sure they let people know of it and that too in a most undiplomatic way.

Image Source

Some of the commonly heard statements are:

• “Why can’t we ever get the application in time into testing?”
• “Why can’t he (developer) just fix this without breaking something else?”
• “We will not continue testing unless fixes this P1 issue.”

…and some really ruder than these.

What makes me wonder is:
– are testers born like that?
– is being blunt one of their personality traits? and
– do they really like behaving like this?

One of the reason I think for doing this is to get heard.

Whatever the reason for doing this is but one thing is for sure that this creates an uncomfortable working environment for everyone – developers, analysts, testers, managers and anyone else involved. There are other better ways of making yourself heard. And one of them is to earn that credibility. Once you’ve achieved that, it is relatively easy to make your point and it does not put anyone in a tensed situation.

So what do you think? Does this happen at your work place? Do you think being direct is the only way to get heard?