I’ve been on vacation for a week and I’m delighted to see @chrisbryant respond so thoughtfully; there’s a lot of wisdom in your reply Chris! To an extent I’m mollified by the message in there, which is that it is unhelpful to see SenseTalk first and foremost as a programming language, and that the effective user will see it almost as an abstraction of the behaviour we want to exercise in the system under test. This helps, because a well-written SenseTalk script will explain what is being tested and how the test is conducted.
I’m groping towards restating my mental model of what we want and I’m under a little time pressure today but I wanted to respond while your replies were still fresh. The key point that I want to get across is that Github has millions of lines of code available for re-use that the budding writer can plug into their existing code base without needing to understand the implementation of the functions they’re using, they just need to understand how the API provided by the bundle is used and what pre-conditions and post-conditions apply when they use the bundle(s) they download. If Sensetalk had this feature, I’d argue that despite the complexity of the various SUTs that we test, common libraries and design patterns would spontaneously emerge that would leverage the work that enterprises have already invested in test software. We’re all missing out on a significant opportunity for code re-use because three or four prerequisites for it don’t exist yet.
Here’s my most high level generic model of what I want from a test, which I can talk around to flesh out the argument I’m making:
This is freely borrowed from the xUnit test frameworks with a bit of the Cucumber way of specificying a test in the middle, in “business user speak”. I’m not going to talk more about Cucumber because it is a distraction at this point, but it does provide a generalised way of specifying a user story in terms of test steps. More on Cucumber at Wikipedia and elsewhere.
Note that the “Begin Test Fixture” code maps well into SenseTalk’s support for loading data from Excel files, and setting up the file system of the SUT with state that the test needs to have available, or doing a miriad of other steps required to have a repeatable test. There are many ways of setting up state outside of reading variables from Excel, but that’s a classic example of SenseTalk makes a hard thing easy to do.
Now in our testing of a large medical case management system, we have refactored Begin Test Fixture code out into separate scripts that define functions that can be called to set the fixture up, and this should be a great place for common code to be shared between teams working on different flows through the system that all need to set up the same preconditions - for example, to make an appointment for a doctor to see a patient requires the same steps irrespective of whether the doctor is a General Practioner or a Dentist; the flows once into the test cases are wildly different but both flows need a new test patient to be registered. I’ll come back to this common code in a moment but bear in mind that “register a patient” and “create an appointment” are both candidates for being common libraries, and they exercise about 5% of the functionality of the system jointly. You can imagine that we might have 40 developers spread across a large number of pizza-sized teams that all want to work without copying and pasting code from one another - they want to use functions that abstract and encapsulate the test setup stage of the test out of the way so that it doesn’t block their progress on the unique steps of the test they are writing.
The next step is the “AS A …” where we plug in a user role as a variable; the user role makes the system perform some steps differently (a dentist usually wants to see dental charts, unlike a General Practioner). So variables are very important as parameters that can facilitate re-use.
I think we all understand how the GIVEN… WHEN… THEN… steps are coded in SenseTalk using standard control structures - we don’t have any issues with how SenseTalk supports these, and we particularly like the exception block structure for handling situations that can arise, but which are exceptional in terms of the normal flow of the test. We understand how to use logError and logSuccess to report the test step outcome using an assertion and we use those rigorously (we think).
Next we get into the End of test fixture code, which again is common (as the Begin Test Fixture code is) to many different scenarios. In our case, we may want to remove the registered patient from system here, after saving any test data artifacts that are significant and that come from running the test.
As we progressed with our code refactoring, we made more and more use of generic handlers or snippets for wrapping up processing steps into calls to common code functions; here’s a basic example that will probably be very familiar to you, although the specifics may vary between Keysight customers:
to ClickEvent MyObject,expOutCome, waitTime:mediumWait
if file (suiteinfo().imagesFolder&"/"&MyObject) exists or if file (suiteinfo().imagesFolder&"/"&MyObject&".png") exists then
Click {image:MyObject,waitfor:waitTime}
else
Click {text:MyObject,waitfor:waitTime, IgnoreNewlines:"Yes", IgnoreSpaces:"Yes", TrimWhitespace:"Yes"}
end if
RefreshScreen
if expOutCome is not empty then
if file (suiteinfo().imagesFolder&"/"&expOutCome) exists or if file (suiteinfo().imagesFolder&"/"&expOutCome&".png") exists then
waitFor waitTime, {image:expOutCome}
else
waitFor waitTime, {text:expOutCome,waitfor:waitTime, IgnoreNewlines:"Yes", IgnoreSpaces:"Yes", TrimWhitespace:"Yes"}
end if
end if
//CaptureScreen increment:yes
end ClickEvent
The challenging part of all of this is that we want to present common code (and its dependent images and other resource files) as components to be consumed as black box entities by teams, rather than through them copying and pasting from scripts that we provide. This means specifically that common code will need its own namespace to distinguish code from one provider from code from another (so that this particular ClickEvent function can be distinguised from another function with the same name from a different provider), and also import statements to identify which library is being used to provide the function, to disambiguate it. It also requires the ability I mentioned in a previous post of defining a method signature for a function separately from the implemention of the method (coding to interfaces); because without this, one can’t abstract the implementation of a function away from the interface or API that is used to make a call to it - you can’t achieve plug-and-play component based development.
That’s about as much as I can sensibly write in a single post response - hope you can see what I’m outlining and I very much look forward to your comments on it, folks 