By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,089 Members | 2,418 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,089 IT Pros & Developers. It's quick & easy.

Acceptance test spike example

P: n/a
I'm posting this message for 2 reasons.

First, I'm still pretty new and shakey to the whole Acceptance Testing thing,
and I'm hoping for some feedback on whether I'm on the right track. Second,
although all the Agile literature talks about the importance of doing
Acceptance Testing, there's very little in any of the books or out on the Web
that helps with how to do it. If I am on the right track, this will be one
more helpful item folks can find on a Google search.

The code below is basically a spike I wrote in a few hours last night to prove
to myself and my team that it would be feasible to quickly create a simple,
useful Acceptance Testing harness for our learning project.

First, I wrote an example test script I would hope to be able to execute...

===============
?recipeListCount:0
!new
?name:"New Recipe"
name:"PB&J"
?name:"PB&J"
!save
!close
?recipeListCount:1
!goToListItem 1
!openListItem
?name:"PB&J"
===============

In this script, ? means test a value, ! means execute an action, and a line
that starts with neither ? or ! is a value assignment.

Next, here's the Python code I wrote to try to run the script. This was just
proof of concept, so at this point, it just runs a single test script with the
script file name hard coded. Also, for simplicty, the skeleton of the
hypothetical application model object is, for now, in-line in the AT system
code.

===============
import string
class RecipeOrgModel:
recipeListCount = 0
name = "New Recipe"

def new(self):
pass

def save(self):
pass

def close(self):
pass

def goToListItem(self,itemNum):
pass

def openListItem(self):
pass
class ParsedStatement:
name = None
value = ""

def __init__(self,statementText,nameValuesDelim):
delimPos = string.find(statementText,nameValuesDelim)
if delimPos==-1:
self.name = statementText
else:
self.name = statementText[0:delimPos]
self.value = statementText[delimPos+1:]
class TestSession:
fileName=""
lines = None
model = RecipeOrgModel()

def __init__(self,fileName):
self.fileName = fileName

def run(self):
self.loadLines(self.fileName)

for line in self.lines:
if not self.processLine(line): break

def loadLines(self, fileName):
file = open(fileName)
lines = file.readlines()
file.close

lines = map(string.strip,lines)
self.lines = lines

def processLine(self,line):
print(line)

if line[0]=='?':
return(
self.TestValue(line[1:])
)
elif line[0]=='!':
self.DoAction(line[1:])
return 1
else:
self.AssignValue(line)
return 1

def TestValue(self,statement):
parsed = ParsedStatement(statement,":")
valueExpr = "self.model." + parsed.name
valueIs = eval(valueExpr)
valueExpected = eval(parsed.value)
hasExpectedVal = (valueExpected == valueIs)
if not hasExpectedVal:
print(" - Expected " + str(parsed.value) + ", but got " +
str(valueIs))
return(hasExpectedVal)

def AssignValue(self,statement):
parsed = ParsedStatement(statement,":")
exec("self.model." + parsed.name + "=" + parsed.value)

def DoAction(self,statement):
parsed = ParsedStatement(statement," ")
methodCall = "self.model." + parsed.name +"(" + parsed.value + ")"
exec(methodCall)
session = TestSession("test1.txt")
session.run()
===============

Note how the powerful, context-aware exec() and eval() procedures really help
simplify the code.

Finally, here's the output I get when I run this with the example script above
saved in a file called test1.txt in the current directory.

===============
?recipeListCount:0
!new
?name:"New Recipe"
name:"PB&J"
?name:"PB&J"
!save
!close
?recipeListCount:1
- Expected 1, but got 0
===============

And this result is pretty much what we expect to see - cool.
Jul 19 '05 #1
Share this Question
Share on Google+
6 Replies


P: n/a
Steve Jorgensen wrote:
I'm posting this message for 2 reasons.

First, I'm still pretty new and shakey to the whole Acceptance Testing thing,
and I'm hoping for some feedback on whether I'm on the right track. Second,
although all the Agile literature talks about the importance of doing
Acceptance Testing, there's very little in any of the books or out on the Web
that helps with how to do it. If I am on the right track, this will be one
more helpful item folks can find on a Google search.

The code below is basically a spike I wrote in a few hours last night to prove
to myself and my team that it would be feasible to quickly create a simple,
useful Acceptance Testing harness for our learning project.

First, I wrote an example test script I would hope to be able to execute...

===============
?recipeListCount:0
!new
?name:"New Recipe"
name:"PB&J"
?name:"PB&J"
!save
!close
?recipeListCount:1
!goToListItem 1
!openListItem
?name:"PB&J"
===============


Snipped.

Its best to aim the Acceptance Test at your Customers level of
expertise, not at the developers. As a Customer is the person best
placed to tell us What they want. Yes Our team people (Devs or testers)
can help write the actual tests, but the Customer should still be able
to read them. After all, the tests are Executable Requirements.

There's two flavors of Customer oriented styles I've come across.

The first uses a Domain Specific Language...eg using your values above..

---------------------------------
IsTotalNumberOfRecipes 0

ClickNewRecipeButton
RecipeNameToUse "PB&J"
SaveNewRecipe
CloseDialog

IsTotalNumberOfRecipes 1

IsThereOneRecipeCalled "PB&J"
---------------------------------

All commands are the first full word. No need for special tokens to
decide what the command is supposed to do, as each Command is the name
of a class to use.

You can use reflection to dynamically create the Command instances or
the name can be used by a factory as an id.

The Commands themselves know what they are supposed to do, not the
Script parsing engine.

The second approach is using FIT/Fitnesse - google these for exact
description. They take information in Table form. This allows the
customer to use any software to create the test scripts, so long as it
can create tables: Word processors, Spread Sheets, etc.

Andrew
Jul 19 '05 #2

P: n/a

"Andrew McDonagh" <ne**@andrewcdonagh.f2s.com> wrote in message
news:d9**********@news.freedom2surf.net...
Steve Jorgensen wrote:
I'm posting this message for 2 reasons.

First, I'm still pretty new and shakey to the whole Acceptance Testing
thing,
and I'm hoping for some feedback on whether I'm on the right track.
Second,
although all the Agile literature talks about the importance of doing
Acceptance Testing, there's very little in any of the books or out on the
Web
that helps with how to do it. If I am on the right track, this will be
one
more helpful item folks can find on a Google search.

The code below is basically a spike I wrote in a few hours last night to
prove
to myself and my team that it would be feasible to quickly create a
simple,
useful Acceptance Testing harness for our learning project.

First, I wrote an example test script I would hope to be able to
execute...

===============
?recipeListCount:0
!new
?name:"New Recipe"
name:"PB&J"
?name:"PB&J"
!save
!close
?recipeListCount:1
!goToListItem 1
!openListItem
?name:"PB&J"
===============

Snipped.

Its best to aim the Acceptance Test at your Customers level of expertise,
not at the developers. As a Customer is the person best placed to tell us
What they want. Yes Our team people (Devs or testers) can help write the
actual tests, but the Customer should still be able to read them. After
all, the tests are Executable Requirements.

There's two flavors of Customer oriented styles I've come across.

The first uses a Domain Specific Language...eg using your values above..

---------------------------------
IsTotalNumberOfRecipes 0

ClickNewRecipeButton
RecipeNameToUse "PB&J"
SaveNewRecipe
CloseDialog

IsTotalNumberOfRecipes 1

IsThereOneRecipeCalled "PB&J"
---------------------------------

All commands are the first full word. No need for special tokens to decide
what the command is supposed to do, as each Command is the name of a class
to use.

You can use reflection to dynamically create the Command instances or the
name can be used by a factory as an id.

The Commands themselves know what they are supposed to do, not the Script
parsing engine.

The second approach is using FIT/Fitnesse - google these for exact
description. They take information in Table form. This allows the customer
to use any software to create the test scripts, so long as it can create
tables: Word processors, Spread Sheets, etc.


The Python version of FIT (currently at 0.7a2) can be found in the
file section of either the FitNesse Yahoo group or the ExtremeProgramming
Yahoo group. One of these days I'm going to get it on PyPi.

It's full featured, works both in batch and with FitNesse, and
includes most of FitLibrary.

There's a book coming out on FIT (FIT for Developing Software)
by Rick Mugridge and Ward Cunningham. It was supposed to be out
a week ago Friday, but B&N says it was delayed.

John Roth
PyFit

Andrew


Jul 19 '05 #3

P: n/a

"Steve Jorgensen" <no****@nospam.nospam> wrote in message
news:mh********************************@4ax.com...
Note how the powerful, context-aware exec() and eval() procedures really
help
simplify the code.
A stylistic note: I believe that most or all of your eval/exec uses could
be done with getattr and setattr instead, which are in the language for
precisely those situations in which the name of an attribute is in a
runtime string. To my mind, this would be simpler and better style.
valueExpr = "self.model." + parsed.name
valueIs = eval(valueExpr)
I believe this is valueIs = getattr(self.model, parsed.name)
methodCall = "self.model." + parsed.name +"(" + parsed.value + ")"
exec(methodCall)

I believe this is getattr(self.model, parsed.name)(parsed.value).

exec("self.model." + parsed.name + "=" + parsed.value)


I believe this is setattr(self.model, parsed.name, parsed.value).

and so on.

Terry J. Reedy

Jul 19 '05 #4

P: n/a
On Sun, 26 Jun 2005 22:42:40 -0400, "Terry Reedy" <tj*****@udel.edu> wrote:

"Steve Jorgensen" <no****@nospam.nospam> wrote in message
news:mh********************************@4ax.com.. .
Note how the powerful, context-aware exec() and eval() procedures really
help
simplify the code.


A stylistic note: I believe that most or all of your eval/exec uses could
be done with getattr and setattr instead, which are in the language for
precisely those situations in which the name of an attribute is in a
runtime string. To my mind, this would be simpler and better style.
valueExpr = "self.model." + parsed.name
valueIs = eval(valueExpr)


I believe this is valueIs = getattr(self.model, parsed.name)
methodCall = "self.model." + parsed.name +"(" + parsed.value + ")"
exec(methodCall)

I believe this is getattr(self.model, parsed.name)(parsed.value).

exec("self.model." + parsed.name + "=" + parsed.value)


I believe this is setattr(self.model, parsed.name, parsed.value).

and so on.


Thanks. I'm new to Python, so I'll take all the style advice I can get.

- Steve J.
Jul 19 '05 #5

P: n/a
On Sun, 26 Jun 2005 16:10:05 -0700, Steve Jorgensen
<no****@nospam.nospam> wrote:
I'm posting this message for 2 reasons.

First, I'm still pretty new and shakey to the whole Acceptance Testing thing,
and I'm hoping for some feedback on whether I'm on the right track. Second,
although all the Agile literature talks about the importance of doing
Acceptance Testing, there's very little in any of the books or out on the Web
that helps with how to do it. If I am on the right track, this will be one
more helpful item folks can find on a Google search.


Check out "Fit For Software Development". It should be published now.

-----
Robert C. Martin (Uncle Bob) | email: un******@objectmentor.com
Object Mentor Inc. | blog: www.butunclebob.com
The Agile Transition Experts | web: www.objectmentor.com
800-338-6716
"The aim of science is not to open the door to infinite wisdom,
but to set a limit to infinite error."
-- Bertolt Brecht, Life of Galileo
Jul 19 '05 #6

P: n/a
On Sun, 26 Jun 2005 16:10:05 -0700, Steve Jorgensen <no****@nospam.nospam>
wrote:
I'm posting this message for 2 reasons.

First, I'm still pretty new and shakey to the whole Acceptance Testing thing,
and I'm hoping for some feedback on whether I'm on the right track. Second,
although all the Agile literature talks about the importance of doing
Acceptance Testing, there's very little in any of the books or out on the Web
that helps with how to do it. If I am on the right track, this will be one
more helpful item folks can find on a Google search.

The code below is basically a spike I wrote in a few hours last night to prove
to myself and my team that it would be feasible to quickly create a simple,
useful Acceptance Testing harness for our learning project.

....

Here's an updated script and code to process the script. I made the script
syntax more friendly.
New example script...

==========
Check recipeListCount is 0

Do new
Check name is "New Recipe"

Keyin "PB&J" to name
Ch eck name is "PB&J"

Do save
Do close
Check recipeListCount is 1

Do goToListItem 1
Do openListItem
Check name is "PB&J"
==========
New Python code for test runner including skeleton of application model that
can be tested, but can't pass the tests...

==========
import string
class RecipeOrgModel:
recipeListCount = 0
name = "New Recipe"

def new(self):
pass

def save(self):
pass

def close(self):
pass

def goToListItem(self,itemNum):
pass

def openListItem(self):
pass
class Action:
failMessage = ""

def __init__(self, actionArgs):
pass

def tryAction(self, model):
return True
class NoAction (Action):
pass

class ActionDo (Action):
item = ""
args = ""

def __init__(self, actionArgs):
delimPos = string.find(actionArgs, " ")
self.args = ""
if delimPos==-1:
self.item = actionArgs
else:
self.item = actionArgs[0:delimPos]
self.args = string.strip(actionArgs[delimPos+1:])

def tryAction(self, model):
methodCall = "model." + self.item +"(" + self.args + ")"
exec(methodCall)
return True
class ActionKeyin (Action):
item = ""
value = ""

def __init__(self, actionArgs):
delimPos = string.find(actionArgs, " to ")
self.args = ""
if delimPos==-1:
self.item = actionArgs
else:
self.value = eval( actionArgs[0:delimPos] )
self.item = string.strip(actionArgs[delimPos+len(" to "):])

def tryAction(self, model):
setattr(model, self.item, self.value)
return True
class ActionCheck (Action):
item = ""
expectValue = ""

def __init__(self, actionArgs):
delimPos = string.find(actionArgs, " is ")
self.args = ""
if delimPos==-1:
self.item = actionArgs
else:
self.item = actionArgs[0:delimPos]
self.expectValue = eval( string.strip(actionArgs[delimPos+len(" is
"):]) )

def tryAction(self, model):
valueIs = getattr(model, self.item)
if self.expectValue == valueIs:
return True
else:
self.failMessage = ( "Expected " + str(self.expectValue) +
" but got " + str(valueIs) )
return False
class ActionUnknown (Action):
actionArgs = ""

def __init__(self, actionArgs):
self.actionArgs = actionArgs

def tryAction(self, model):
self.failMessage = "Test statement not understood: " + self.actionArgs
return False
def MakeTestAction(lineText):
delimPos = string.find(lineText, " ")
commandArgs = ""
if delimPos==-1:
commandType = lineText
else:
commandType = lineText[0:delimPos]
commandArgs = string.strip(lineText[delimPos+1:])

if commandType == "Do":
return ActionDo(commandArgs)
elif commandType == "Keyin":
return ActionKeyin(commandArgs)
elif commandType == "Check":
return ActionCheck(commandArgs)
elif commandType == "":
return NoAction(commandArgs)
else:
return ActionUnknown(commandType + " " + commandArgs)
class TestSession:
fileName=""
lines = None
model = RecipeOrgModel()

def __init__(self,fileName):
self.fileName = fileName

def run(self):
self.loadLines(self.fileName)

for line in self.lines:
print(line)
action = MakeTestAction(line)
actionOk = action.tryAction(self.model)
if not actionOk:
print(" !!! " + action.failMessage)
break

def loadLines(self, fileName):
file = open(fileName)
lines = file.readlines()
file.close

lines = map(string.strip,lines)
self.lines = lines
session = TestSession("test1.txt")
session.run()
==========
Ouptut of test runner using example script...

==========
Check recipeListCount is 0

Do new
Check name is "New Recipe"

Keyin "PB&J" to name
Check name is "PB&J"

Do save
Do close
Check recipeListCount is 1
!!! Expected 1 but got 0
==========
Jul 19 '05 #7

This discussion thread is closed

Replies have been disabled for this discussion.