Organizing tests into Categories

I’m full of interesting thought today! As my suite of tests gets larger I find that I want to somehow Flag 5 test in say a “browser” category… and another 5 in a “application” category… etc. And run each group only when necessary.

I could make different copies of my Master Script containing each of the scripts in the RUN. But there must be a more elegant solution.

I have a hunch these organizational attributes would be found in Eggplant Gold?


Ok, here is where you could leverage again the Eggplant OpenSuite() command. Try putting your tests in suites named by category, and use OpenSuite(/path/to/CategoryName.suite). You then can make your calls or use your category type resources.

You are getting the right ideas, and moving to the logical solution is moving to a more modular and designed solution. You may want to sit back on a white board with some software engineers and see how you would either factor out functionality, or design and assign responsibilities and functionality to SenseTalk objects within a suite. This can save a lot of time, and allow the data abstraction occur that allows simplified maintenance and ease of factoring out later.

One thing to keepin mind, try and keep your handlers simple, small and embracing some very modular functionality to occur. This lends itself often to a more simplified implementation that has less bloat and more functionality as well as small meaningful handler names that are not easily forgotten or confused with others like it with minor changes.

Question, Answer, Formulate, Test, Implement, Test, Refine, Test, start over :slight_smile:

The test building cycle is short and quick, the results are often astonishingly pleasing.

Sorry can’t speculate on future features of product or functionality, at least in public without NDAs in place :slight_smile:

Using a master script (as it sounds like you’re doing) that uses RunWithNewResults to run other scripts is a great way to go. You could adapt that approach to store the names of tests to be run in a file, which might make it a bit more flexible.

Another possibility to keep in mind is to use the Schedule tab to make a list of scripts to run as a batch. Each suite only has one schedule list, but there are a couple of tricks you can use to make this work for you.

The first trick is a way to effectively create multiple lists of scripts within a Schedule. To do this, in your case you might create two new scripts in your suite called “BrowserTests” and “ApplicationTests”. These scripts can be completely empty, as they will mainly serve as category names in the Schedule.

Drag the BrowserTests script into the Schedule, followed by all of the browser tests that you want to run. Select all of the tests and press the right arrow to indent them below the BrowserTests script. This way they’ll only run if the BrowserTests script succeeds (which it always will if it’s an empty script). But this allows you to disable the whole batch by clicking the checkbox next to the BrowserTests script.

Now add ApplicationTests and indent all of the application tests below it. At this point you can turn either category on or off with a single click. You can also turn individual tests on or off before running that batch.

The second trick you can use is to create a new suite and simply drag scripts into its Schedule. People aren’t always aware of this, but a Schedule can include scripts from other suites. So you may want to create some suites that don’t have any scripts of their own, but are only used to define and run a batch of other scripts.

I hope these ideas are helpful!

Just in case someone wants to see how Doug’s brilliant use of the Eggplant Scheduler would function, here is a suite set up for some basic demo.

Try en/disabling scripts and rerun the schedule. You can run from the GUI or from the /path/to/ CLI option. If you have one license, remember that you must quit the GUI first to run the CLI.

Setting Eggplant Scheduler dependancies and general usage are detailed starting on Page 55 of the Eggplant v 4.0 Reference Manual.