Redirect the test result to csv file

Eggplant Functional
1

Hi All,

I’ve done implemented all of the functions that need to run test in one file, called A_lib, and create few more files, B_mod, C_mod, D_mod… for each particular module which reuse the function code from A_lib. Finally, I create a file called testcase which is using to run test as.

put ( [“B_mod” testcase1], [“C_mod” testcase2], [“D_mod” testcase3]) into mytestcases

repeat with each item of mytestcases
execute the test and export the result to cvs
end repeat.

Anybody please show me how to export the result to cvs file along with all the errors if there is any, and email this file to me after each run. I’d very appreciated.

Best Regards,
:roll:

2

I’m assuming that you mean “CSV” format (comma-separated values) rather than “CVS” (Concurrent Versions System [among others]). The LogFile.txt result files that Eggplant produces are tab-delimited – if you are able to work with that, then you don’t actually need to convert them. But if it has to be CSV, then you could do something like this:


repeat with each testcase of mytestcases 
	runWithNewResults testcase
	put file (the last item of ScriptResults(testcase))'s LogFile into theResults
	repeat with each line of theResults
		replace every occurrence of tab with "," in it
		put it & return after csvResults
	end repeat
	sendMail(from:"someone@yourcompany.com", to:"buffalo@yourcompany.com", subject:"Results for" && testcase, body:csvResults)
end repeat

ScriptResults is an Eggplant function that returns a list of property lists that describe all of the result files for the specified script, the last item of that list referring to the most recent result file. The LogFile property is the full path to that result file.

I have to warn you that due to some changes that Apple [quietly] made regarding their support of UNIX sendmail under Mac OS X 10.3, the sendmail function in Eggplant 1.5 is unfortunately prone to crashing the application. We have an all new implementation of the sendmail function in our upcoming Eggplant 2.0 release that improves on the current version in a number of areas, not the least of which is stability.

Let me know if you have any other questions.

3

Matt,

I’m so sorry that I did make my question clearly. What I want from the result file is the format below. In addition, I just want to email one complete result file only.

testcase1;failed;errors if any
testcase2;passed;
.
.
.
.

Thanks so much for your help.

Best Regards,

4

Okay, that’s even easier:

repeat with each testcase of mytestcases 
	runWithNewResults testcase
	put the result into outcome
	put testcase & ": " after resultsMailer
	put outcome's status after resultsMailer
	if outcome's status is "Failure" then
		put ", Errors: " & outcome's errors & ", Exceptions: " & \
				outcome's exceptions after resultsMailer
	end if
	put return after resultsMailer
end repeat
sendMail(from:"someone@yourcompany.com", to:"buffalo@yourcompany.com", \
		subject:"Results for" && testcase, body:resultsMailer)

Please note that you can’t get specific errors or exceptions for each file, only a count of them.

5

Matt,

could you please take a look my code below? It’s only email the word “resultmailers” in the body. If you know how, could you please modified it so that I could get email with the attachment file of resultmailer instead of text in the body.
Thanks so much.

put ([“PrintTestCases” testcase61],
[“PrintTestCases” testcase62],
[“PrintTestCases” testcase64],
[“PrintTestCases” testcase66],
[“PrintTestCases” testcase67],
[“PrintTestCases” testcase68],
[“PrintTestCases” testcase69]) into mytestcases

repeat with each testcase of mytestcases
runWithNewResults testcase
put the result into outcome
put testcase & ": " after resultsMailer
put outcome’s status after resultsMailer
if outcome’s status is “Failure” then
put ", Errors: " & outcome’s errors & ", Exceptions: " & \
outcome’s exceptions after resultsMailer
end if
put return after resultsMailer
end repeat

sendMail(from: “a.p@taboshi.bats.com”, to: “a.p@taboshi.bats.com”, subject: “Results for Kiosk Automation”, body:resultsMailer)

Best Regards

6

Sorry to be getting back to you a little late; I’m actually still a little unclear on exactly what’s going on. I can point out that each item that you’re putting into the mytestcases list is actually a function call that’s being evaluated before being added to the list. So you’re not getting list items of the form [“PrintTestCases” testcasen], you’re getting whatever the return value of testcasen is. You should put:

put myTestCases

before your repeat loop to see what values the list contains.

CORRECTION: Okay, so I’m wrong about the following: “Actually, I suspect that those function calls are returning nothing, and no matter how many times you put empty into an empty list, it’s still going to be empty (nothing times nothing is nothing, after all).” If the function calls return empty, you’ll still end up with a list of empty items and the repeat loop will try to iterate through that list, but it will throw an exception because this won’t work:

runWithNewResults empty -- will throw an exception

on the other hand, if those calls weren’t evaluated at all, this potentially won’t work either:

runWithNewResults ["PrintTestCases" testcasen] -- will throw an exception unless testcasen returns a script name

RunWithNewResults only works with scripts, so unless these function calls return script names, the script should be failing there.

We’re wondering if you’ve actually run the code exactly as you posted it, because try as we might, we can’t get it to run to completion – it always throws an exception somewhere. Our best guess is that maybe you tried running it with just one call in the part where you create the list, that one call returned nothing and gave you an empty list. So your repeat with each has nothing to repeat over, nothing is added to the resultsMailer variable, and when that variable is used in the sendmail command, it gets evaluated as a string, becoming just “resultsMailer”. Having an empty list is the only way that the posted code will skip over the repeat loop, and if the repeat loop is executed, then resultsMailer would at least have a bunch of carriage returns in it and wouldn’t appear as just it’s name.

If that’s not the case, we’ll need more information to diagnose the issue. Let us know if you have any more questions. BTW, I edited your post to obscure your e-mail address so it can’t be harvested by the spammers.

Regards,
Matt

7

Matt,

So sorry for took so long to reply because I just got back from vacation. Actually, I have about 6 files, each file has about 100 functions which are testcases. From testcase.script, I call all the function from 6 files above to run. Below is my revised script but it is not woring. Could you please take a look. What I want to do is after every testcase, open the file and write into this file either failed or passed. Btw, you were right, all of my function do not return any value. I’m so sorry for my dumb programing skill.

put ([“SendAFile” testcase727],
[“SendAFile” testcase728],
[“SendAFile” testcase732],
[“SendAFile” testcase735],
[“SendAFile” testcase736],
[“SendAFile” testcase738]) into mytestcases

put ([“SendAFile” testcase738]) into mytestcases
repeat with each testcase of mytestcases
runWithNewResults testcase
put “/Users/buffalo/Documents/kioskauto.suite/resultsMailer.csv” into testfile
put the result into outcome

if there is not a file testfile then
	create file testfile
end if

if outcome's status is "Failure" then
	open file testfile for appending
	put testcase & "; " & "Failed" after file testfile
else
	open file testfile for appending
	put testcase & "; " & "Passed" after file testfile
end if

put return after file testfile

end repeat

8

put ([“SendAFile” testcase738]) into mytestcases
repeat with each testcase of mytestcases

What do you get if you add the following line at this point in your script:

put testcase

runWithNewResults testcase

Unless testcase currently contains the name of a script, this line won’t work. RunWithNewResults only takes a scriptname as an argument.

put “/Users/buffalo/Documents/kioskauto.suite/resultsMailer.csv” into testfile
put the result into outcome

You need to switch the two preceding lines. Right now “the result” contains the result of the “put” statement, not the RunWithNewResults command. “The result” has a short shelf-life – it has to be called right after the command that you’re trying to get the result of.

if there is not a file testfile then
create file testfile
end if

This is fine, although it’s not really necessary; the file will be created when you write to it.

if outcome’s status is “Failure” then
open file testfile for appending
put testcase & "; " & “Failed” after file testfile
else
open file testfile for appending
put testcase & "; " & “Passed” after file testfile
end if

A couple of things here: You don’t need to open the file for appending – you can just write to it. And as this is written, you would be opening it for appending every time through the repeat loop, but never closing it. So, with a couple of other small changes, the above could be rewritten like so:

put testcase & "; " after file testfile
if outcome's status is "Failure" then
    put "Failed" after file testfile
else
    put "Passed" after file testfile
end if
put return after file testfile

put return after file testfile
end repeat

Those are the things I can see and suggest given the information you’ve provided. I hope this helps.

9

Matt,

I’ve revised the script based on your suggestion but it is not working. The resultmailer file was not created. Please help me.

repeat with each testcase of mytestcases
runWithNewResults testcase
put the result into outcome
put “/Users/buffalo/Documents/kioskauto.suite/resultsMailer.csv” into testfile

put ([“SendAFile” testcase727],
[“SendAFile” testcase728],
[“SendAFile” testcase732],
[“SendAFile” testcase735],
[“SendAFile” testcase736],
[“SendAFile” testcase738]) into mytestcases
put testcase & "; " after file testfile
if outcome’s status is “Failure” then
put “Failed” after file testfile
else
put “Passed” after file testfile
end if

put return after file testfile

end repeat

10

The code at the end of this message will work IF the mytestcases variable is populated with script names. But PLEASE run this piece of code and post the output:

put (["SendAFile" testcase727], \ 
		["SendAFile" testcase728], \ 
		["SendAFile" testcase732], \ 
		["SendAFile" testcase735], \ 
		["SendAFile" testcase736], \ 
		["SendAFile" testcase738]) into mytestcases
put mytestcases

If this still isn’t working, I need to know what the contents of “mytestcases” are. The only reason that this code would not be working is that the “testcases” that are getting run aren’t actually scripts.


put (["SendAFile" testcase727], \ 
		["SendAFile" testcase728], \ 
		["SendAFile" testcase732], \ 
		["SendAFile" testcase735], \ 
		["SendAFile" testcase736], \ 
		["SendAFile" testcase738]) into mytestcases 

put "/Users/buffalo/Documents/kioskauto.suite/resultsMailer.csv" into testfile 

repeat with each testcase of mytestcases 
	runWithNewResults testcase 
	put the result into outcome 
	
	put testcase & "; " after file testfile 
	if outcome's status is "Failure" then 
		put "Failed" after file testfile 
	else 
		put "Passed" after file testfile 
	end if 
	
	put return after file testfile 
	
end repeat

open testfile with "TextEdit"
11

Matt,

You are right, the testcase is function from sendafile script, and these function do not have any return value. Is there anyway that I could revised my script to work with function instead of script.

Thanks so much for your help.

PS: I’m so sorry for cut and paste the source code in previous post incorrected. Thanks for corrected.

Best Regards,

12

Perhaps this will help. I am making some guesses and assumptions here about how your scripts are organized.

If I understand correctly, you have a script called “SendAFile”, which contains a number of functions whose names all begin with “testcase”. You would like to call all of these functions, determine whether each one succeeded or failed, and generate a set of results for the entire batch.

To do all of this, you are trying to create a master script that will do the following:

  1. call a number of these testcase functions as though they were scripts
  2. collect all of the results from the different testcase functions
  3. generate an email message containing these results

I have tested the following master script:

put ("testcase727","testcase728","testcase732", \ 
		"testcase735", "testcase736", "testcase738") into mytestcases 

set the strictProperties to true -- so undefined testcases don't report success
repeat with each testcase of mytestcases 
	try
		get SendAFile.(testcase) -- call the testcase as a function
		put testcase & ";Passed" into reportLine
	catch error
		put testcase & ";Failed;" & error.reason into reportLine
	end try
	put reportLine & return after report
end repeat

put report -- display the results

The master script shown above works. If you also want to send an email with the results, add this line at the end:


-- email the results, too
sendmail (from: "a.p@taboshi.bats.com", to: "a.p@taboshi.bats.com", subject: "Results for Kiosk Automation", body:report)

The script above works for me here, when the following SendAFile script is present in the same suite:


function testcase727
	put "testcase727 is running"
end function

function testcase728
	throw "testcase728 error", "Bogus error message"
end function

function testcase735
	put "This is testcase735"
end function

Please try it like this, and let us know if you need additional assistance.

13

Getting the error while executing the All_testcases.script in a file:

FAILURE Screen_Error.tiff STUnknownMessage ERROR: No Such Function: ‘testcase1’

I have the scripts the scripts basic1,jobinfo1,jobinfo2 etc… in test.suite
and the below code in All_testcases.script

put{ [basic1 testcase1],
[jobinfo1 testcase2],
[jobinfo2 testcase3],
[ jobinfo3 testcase4],
[jobinfo4 testcase5],
[media1 testcase6],
[media2 testcase7],
[media3 testcase8],
[media4 testcase9],
[layout1 testcase10],
[layout2 testcase11],
[layout3 testcase12],
[layout4 testcase13],
[layout5 testcase14],
[layout6 testcase15],
[layout7 testcase16],
[layout8 testcase17],
[layout9 testcase18],
[layout10 testcase19],
[color1 testcase20],
[color2 testcase21],
[color3 testcase22],
[color4 testcase23],
[color5 testcase24],
[image1 testcase25],
[Finishing1 testcase26],
[Finishing2 testcase27],
[Finishing3 testcase28],
[Finishing4 testcase29],
[Finishing5 testcase30],
[Finishing6 testcase31],
[Finishing7 testcase32],
[Finishing8 testcase33],
[Finishing9 testcase34],
[Finishing10 testcase35],
[Finishing11 testcase36],
[Finishing12 testcase37],
[Finishing13 testcase38],
[Finishing14 testcase39],
[Finishing15 testcase40],
[Finishing16 testcase41],
[stamping1 testcase42],
[stamping2 testcase43],
[stamping3 testcase44] } into mytestcases

put “/Volumes/Eggplant_Related/test.suite/resultsMailer.csv” into testfile

repeat with each testcase of mytestcases
runWithNewResults testcase
put the result into outcome

put testcase & "; " after file testfile 
if outcome's status is "Failure" then 
	put "Failed" after file testfile 
else 
	put "Passed" after file testfile 
end if 

put return after file testfile

end repeat

open testfile with “TextEdit”

14

The problem here is that you can’t do “runWithNewResults” on a function (which is what all the things in your list are); it works only on scripts. In each iteration of your repeat loop “testcase” is a function call, which is actually getting evaluated before the RunWithNewResults call, and it’s the return value of that function call that Eggplant is attempting to run as though it was a script. If you want separate results for each item in your list, you’re going to have to break the functions out into separate scripts.

If you read the earlier posts in this topic, you’ll see that the original poster had the same problem and that we directed him to make sure that his list contained references to scripts and not functions.