ATF Testing – Parameters, Scratchpad and Workflows

The Script Include below has a number of helper methods for ATF Testing – such as:

  1. getOrder() – get the parameter Set order number
  2. saveScratchpad() – save a variable ot a scratch pad which can be acceses from the next step
  3. getScratchpad() – get a variable from the scratch pad
  4. getParameter() – get a parameter value from the current Parameter set.
  5. getFinshedContext() – get the finished workflow context (wait for it to finish)
  6. verifyActivities() – verify the activities in the Context

According to the documentation, using parameters from within an ATF Server Script is not supported by SN – so interpret that appropriately.  This script include delves into the ATF Tables in a fairly serious way – so if SN change their implemention – this code will break.  However that is not unusual.

I experimented quite heavily with ATF for two weeks.  ATF definitely has its place but there are times when it is not appropriate – or difficult.

  1. If the Code is interfaces heavily with external applications and data integrity with outside databases is important – then ATF may not be appropriate.  The roll-back feature (which can’t be turned off) may cause loss of data integrity.  for example:  Testing CSD (which interfaces to SCCM) is problematic.  ATF fits best when the end-to-end Solution resides totally within SN.
  2. ATF is quite difficult to test success or failure of Workflows.  Hence I write this script include below.   If any SN staff read this, please consider this as an enhancement.  I believe the two methods below (getFinshedContext and verifyActivities) could be configured as a custom Workflow type – which would make it easier to use and would not required a a Scripting Step.
  3. Custom GUIs.  I was trying to test a custom Gui with custom Widgets that displayed in a list – and each Widget might display in the list in any order. I probably should have tried for longer – but I was uncertain how to “click” on the appropriate Widget.    It may have been possible – but it certainly was not easy.
  4. ATF fits best for routine testing of Request Items or Script Includes and REST APIs.
/*
Custom ANZ Workflow - for ATF Testing of Catalogue items
Doug Connell - Last Updated 2020-05-07

ATF HELP FUNCTIONS
==================
This script contains a number of helper functions for ATF Testing.

1) getOrder()           - get the parameter Set order number
2) saveScratchpad()     - save a variable ot a scratch pad which can be acceses from the next step
3) getScratchpad()      - get a variable from the scratch pad 
4) getParameter()       - get a parameter value from the current Parameter set.
5) getFinshedContext()  - get the finished workflow context (wait for it to finish)
6) verifyActivities()   - verify the activities in the Context


WORKFLOW CHECKING
=================
This Script Include can be used to check the success / failure of a Service Catalogue Item by looking at the workflow
and checking the actvities in the log (wf_history) to ensure that the Actvityies all finishes successfully.

Please see test: CSD - Deploy Item.  This script include was first developed for this test - but could be used for many others.

It is strongly recommended that ATF Parameterized Testing is used.

Paramaters Could include:

1) Item - Service Catalogue Item (sc_cat_item) to test 
2) User - User to imporsonate when opening the Catalogue item 
3) Manager - Manager to impersonate for approving this Catalogue item 
4) Workstation - Workstation used by the Catalogue Item
5) Workflow - Workflow for table sc_req_item to check to see if it is successful. 
6) Mandatory Actvity - Mandaotry activity to check to ensure that it finished 

*/

var ATFext = Class.create();

ATFANZext.prototype = {

// ###################################################################	
/*  Constructor for the class.

This class can be used to help parapeterized ATF testing - 
in particulat the testing of  Serice Catalogue Items with Workflows. 

Example
Class should be initialized with the sys_id of the ATF Test (sys_atf_test)
for example: ATFExt = new global.ATFANZext(test_sys_id);

Arguments
1) testId - sys_id of the ATF Test  (sys_atf_test)
2) debug - ture of false.  Set to true to turn on debugging. Use method log() to print the debugging.


*/
initialize: function(testId, debug) {
	this.testId = testId;
	this.debug = debug;
	this._debugMessage = "ATFANZext";
	this._logDebug("initialize: about to call: this._getParameters");
	this._parameterSet = this._getParameterSet(testId);
	this._scratchpad = this._loadScratchpad();
	// sleep for a short time so that the Test Results arw written in the correct order.
	// otherwise sometimes the start_time is the same for parameter set 1 and parameter set 2 and the function: ATFANZext.getTestResultOrder() does not work.
	// I may have to increase this time if the ordr is ever wrong.
	gs.sleep(100); 
},
	
// ###################################################################
/*
Function to print the debug to the System Log.
*/
log: function() {
	if (this.debug) {
		var p = "\n#################################################################\n";
		this._debugMessage = p + this._debugMessage + p;
		gs.log(this._debugMessage);
	}	
},

// ###################################################################
_logDebug: function(message) {
	if (this.debug) {
		this._debugMessage += "\n" + message;
	}
},

// ###################################################################
getGlideRecord: function(tableName, sysId) {
	var gr = new GlideRecord(tableName);
	gr.get(sysId);
	return gr;
},
	
// ###################################################################
// Get the parameter set order
getOrder: function() {
	var order =  this._parameterSet.order;
	this._logDebug("getOrder: order = " + order);
	return order;
},

// ###################################################################
// Save the object to the scratchpad
// The test results table has an unused field called test_description or test_result_json field which I can use to hold Json.
// I can then use this field to pass the scratchpad from one step to another.
// If this doesn't work - then we can use some other table - perhaps create an event
saveScratchpad: function(obj) {
	var testResult = this._testResult;
	var jsonString = JSON.stringify(obj);
	testResult.test_result_json = jsonString;
	testResult.update();
},
	
// ###################################################################
// Get the scratchpad.
getScratchpad: function() {
	return this._scratchpad;
},
	
// ###################################################################
// Get the scratchpad.
_loadScratchpad: function() {
	// --------------------------------------------
	var testResult = new GlideRecord("sys_atf_test_result");
	testResult.addQuery("test",this.testId);
	// Get the most recent test Results.
    testResult.orderByDesc("sys_created_on");
	testResult.query();
	this._logDebug( "_loadScatchpad: testResult.getRowCount() = " + testResult.getRowCount());
	testResult.next();
	var jsonString = testResult.test_result_json.toString();
	this._logDebug("_loadScatchpad: jsonString = " + jsonString);
	this._testResult = testResult;
	var obj = {};
	try {
		obj = JSON.parse(jsonString);
	}
	catch(error) {
		this._logDebug("_loadScatchpad: WARNING: test_description holds invalid json string");
	}
	this._scratchpad = obj;
	return obj;
},


	
// ###################################################################
// Gets the sys_id of the specified step 
// THIS FUNCTION IS NOT RECOMMENDED. It is easy to change the order of steps in a test.
// Therefore its better to hardcode the Step sys_id into your Server Test Script.
//  Theen if you add a Step or re-order your steps, the code wil still work.
getStepId: function (stepNumber) {
    var step  = new GlideRecord("sys_atf_step");
    step.addQuery("test", this.testId);
    step.addQuery("order", stepNumber);
    step.query();
    if ( step.next() ) {
        this._logDebug("getStepId: Step " + stepNumber + " Found, sys_id = " +  step.sys_id);
		this._logDebug("getStepId: Notes = " + step.notes);
		return step.sys_id;
    } else {
        this._logDebug("getStepId: ERROR: step " +stepNumber + " not Found"  );
    }
},
	
// ###################################################################	
/*
SUMMARY: Get the order from the most recent Parameter Run (sys_atf_parameter_run)
ARGUMENTS: testId - sys_id of the test (sys_atf_test)
RETURNS: returns the order (integer)
*/
_getParameterRunOrder: function(testId) {
	var parameterRun = new GlideRecord("sys_atf_parameter_run");
    parameterRun.addQuery("test",testId);
	// Get the most recent test Results.
    parameterRun.orderByDesc("order");
    parameterRun.orderByDesc("sys_created_on");
	//testResult.orderBy("created");
	parameterRun.query();
	this._logDebug( "_getParameterRunOrder: parameterRun.getRowCount() = " + parameterRun.getRowCount());
	parameterRun.next();
	this._logDebug( '_getParameterRunOrder: parameterRun.order = ' + parameterRun.order);
	return parameterRun.order;
},
	
// ###################################################################	
/*
SUMMARY: Gets order from the most recent Test Result (sys_atf_test_result)
ARGUMENTS: testId - sys_id of the test (sys_atf_test)
RETURNS: returns the order (integer)
*/
_getTestResultOrder: function(testId) {
	var testResult = new GlideRecord("sys_atf_test_result");
    testResult.addQuery("test",testId);
	// Get the most recent test Results.
    testResult.orderByDesc("start_time");
	//testResult.orderBy("created");
	testResult.query();
	this._logDebug( "_getTestResult: testResult.getRowCount() = " + testResult.getRowCount());
	testResult.next();
	var order = testResult.getDisplayValue("parameter_set_run") ;
	this._logDebug( '_getTestResult: order = ' + order);
	return order;
},
	
// ###################################################################	
/*
SUMMARY: Get the parameterSet GlideRecord (sys_atf_parameter_set)
ARGUMENTS: testId - sys_id of the test (sys_atf_test)
RETURNS: GlideRecord of table: sys_atf_parameter_set 
*/
_getParameterSet: function(testId) {
	var order = this._getTestResultOrder(testId);
	this._logDebug( "_getParameterSet: order = " + order);
	var parameterSet = new GlideRecord("sys_atf_parameter_set");
	parameterSet.addQuery("test",this.testId);
	parameterSet.addQuery("order",order);
	parameterSet.query();
	this._logDebug( "getParameterSet: testResult.getRowCount() = " + parameterSet.getRowCount());
	var paramObj = {};
	parameterSet.next();
	return parameterSet;
},


// ###################################################################
/*
SUMMARY: get the value of a ATF Parameter used in parametrized testing.
ARGUMENT: paramterName
RETURNS: String GlideRecord if its a reference field or String if it's another tpye of field.
*/
getParameterValue: function(parameterName) {
	this._logDebug("getParameterValue: parameterName = " + parameterName );
	// ---------------------------------------------------------
	// First find the parameter Value GlideRecord
	var parameterValue = new GlideRecord("sys_variable_value");
	parameterValue.addQuery("document", "sys_atf_parameter_set" );
	parameterValue.addQuery("document_key", this._parameterSet.sys_id);
	parameterValue.query();
	this._logDebug("getParameterValue: parameterValue.getRowCount() = " + parameterValue.getRowCount() );
	// ---------------------------------------------------------
	// Loop through the parameter Values and find one with the correct name
	var value = "";
	while (parameterValue.next()) {
		var label = parameterValue.variable.label;
		var type = parameterValue.variable.internal_type.name;
		var table = parameterValue.variable.reference.name;
		this._logDebug("getParameterValue: label = " + label + ", type = " + type + ", table = " + table);
		if ( label == parameterName ) {
			/*
			value = parameterValue.value;
			break;
			*/
			if ( type == "reference" ) {
				value =  this.getGlideRecord(table,parameterValue.value);
			}
			else {
				value = parameterValue.value;
			}
			break;
		}	
	}
	// ---------------------------------------------------------
	this._logDebug("getParameterValue: value = " + value );
	return value;
},

	

// ###################################################################	
/*
Gets the Workflow (wf_context) record based on the provided sys_id of the sc_req_item
There may be numeruous Workflows running for the same requested item, 
therefore the user must also provide the workflow name.

Arguments 
1) WorkFlowName = the name of the workflow for which the context should be fetched.
2) requestedItemId - GlideRecord of the sc_req_item that we are checking.

Returns
The GlideRecord of the wf_context found. Returns null if not found.

*/
_getContext: function(requestedItemId, workflowName) {
	//var workflowName = this.getParameter("Workflow");
	this._logDebug( "_getContext: requestedItemId = [" + requestedItemId + "]");
	this._logDebug( "_getContext: workflowName = [" + workflowName + "]");
	var context = new GlideRecord("wf_context");
	context.addQuery("id",requestedItemId);
	context.addQuery("name",workflowName);
	context.query();	
	this._logDebug("_getContext: count = " + context.getRowCount());
	context.next();
	this._logDebug("_getContext: context.name = " + context.name);
	return context;
},

	
// ###################################################################	
// Get the State of the Context - Not used
/*
_getContextState: function(wfContextId) {
	var wfContext = new GlideRecord('wf_context');
	wfContext.get(wfContextId);
	return wfContext.state;
},
*/

// ###################################################################	
/* 
Gets the Finished Workflow (wf_context) record based on the provided sys_id of the sc_req_item
Waits until the context has finished - up to maxWait time.

There may be numeruous Workflows running for the same requested item, 
therefore the user must also provide the workflow name.

Arguments 
1) WorkFlowName = the name of the workflow for which the context should be fetched.
2) requestedItemId - GlideRecord of the sc_req_item that we are checking.
4) timeout - timeout when waiting for the workflow to finish.

Returns
The GlideRecord of the wf_context found. Returns null if not found.
*/
getFinishedContext: function(requestedItemId, WorkflowName, timeout) {
	var waitedFor = 0;      // time in seconds that we have waited.
	var state = "";
	var waitSeconds = 30;
	var waitMs = waitSeconds * 1000;
	this._logDebug("getFinishedContext: requestedItemId = " + requestedItemId);
	this._logDebug("getFinishedContext: WorkflowName = " + WorkflowName);
	this._logDebug("getFinishedContext: timeout = " + timeout);
	// loop until the timeout
	while ( waitedFor < timeout ) {
		var context = this._getContext(requestedItemId, WorkflowName);
		state = context.state;
		// If the contect is finished then return the wf_context GlideRecord.
		if ( state == "finished" ) { 
			this._logDebug("getFinishedContext: context.sys_id = " + context.sys_id);
			this._logDebug("getFinishedContext: SUCCESS! context.name = " + context.name + " has Finished!");
			return context;
		}
		gs.sleep(waitMs);
		waitedFor = waitedFor + waitSeconds;
	}
	this._logDebug("getFinishedContext: ERROR! unable to find 'finished' wf_context after " + timeout + " seconds! exiting" );
	return context;
},
	
// ###################################################################	
/*
Verify the activities of the wf_context to ensure that they finished. 

Arguments 
1) mandatoryActivity = set to the name of a mandatory activity in the workflow that must be successful.
2) requestedItem - GlideRecord of the sc_req_item that we are checking.
3) workFlowName - Name of the workflow - the contect of this workflow is watched.
4) timeout - timeout when waiting for the workflow to finish.

Returns
This function returns true if the mandatory activity exists and has a state of "finished" - otherwise returns false.

*/
verifyActivities: function(requestedItemId, mandatoryActivity, workflowName, timeout) {	
	
	// var mandatoryActivity = this.getParameter("Mandatory Activity");
	this._logDebug("verifyActivities: mandatoryActivity = " + mandatoryActivity);
	this._logDebug("verifyActivities: WorkflowName = " + workflowName);
	var context = this.getFinishedContext(requestedItemId, workflowName, timeout);
    var contextId = context.sys_id; 
    var history = new GlideRecord("wf_history");
    history.addQuery("context",contextId);
    history.orderBy("activity_index");
    history.query();
	var result = false;
	
	var numberHistoryRecords = history.getRowCount();
	this._logDebug("verifyActivities: numberHistoryRecords = " + numberHistoryRecords);
     
    while ( history.next()) {
		var activityName = history.getDisplayValue("activity");
		var activityState = history.getValue("state");
		this._logDebug("verifyActivities: Actvity " + history.activity_index + " = " + activityName + ", state = " + activityState);
        if ( activityName == mandatoryActivity) {
			if (activityState === "finished" ) {
				result = true;
			}
		}
    }
	this._logDebug("verifyActivities: result = " + result);
	return result;
},


// ###################################################################	
type: 'ATFext'
};

Tagged in :

dconnell@hotmail.co.nz Avatar

Leave a Reply

Your email address will not be published. Required fields are marked *