robot.running package¶
Implements the core test execution logic.
The main public entry points of this package are of the following two classes:
TestSuiteBuilder
for creating executable test suites based on existing test case files and directories.TestSuite
for creating an executable test suite structure programmatically.
It is recommended to import both of these classes via the robot.api
package like in the examples below. Also TestCase
and Keyword
classes used internally by the
TestSuite
class are part of the public API.
In those rare cases where these classes are needed directly, they can be
imported from this package.
Examples¶
First, let’s assume we have the following test suite in file
activate_skynet.robot
:
*** Settings ***
Library OperatingSystem
*** Test Cases ***
Should Activate Skynet
[Tags] smoke
[Setup] Set Environment Variable SKYNET activated
Environment Variable Should Be Set SKYNET
We can easily parse and create an executable test suite based on the above file
using the TestSuiteBuilder
class as follows:
from robot.api import TestSuiteBuilder
suite = TestSuiteBuilder().build('path/to/activate_skynet.robot')
That was easy. Let’s next generate the same test suite from scratch
using the TestSuite
class:
from robot.api import TestSuite
suite = TestSuite('Activate Skynet')
suite.resource.imports.library('OperatingSystem')
test = suite.tests.create('Should Activate Skynet', tags=['smoke'])
test.keywords.create('Set Environment Variable', args=['SKYNET', 'activated'], type='setup')
test.keywords.create('Environment Variable Should Be Set', args=['SKYNET'])
Not that complicated either, especially considering the flexibility. Notice that the suite created based on the file could also be edited further using the same API.
Now that we have a test suite ready, let’s execute it
and verify that the returned
Result
object contains correct
information:
result = suite.run(critical='smoke', output='skynet.xml')
assert result.return_code == 0
assert result.suite.name == 'Activate Skynet'
test = result.suite.tests[0]
assert test.name == 'Should Activate Skynet'
assert test.passed and test.critical
stats = result.suite.statistics
assert stats.critical.total == 1 and stats.critical.failed == 0
Running the suite generates a normal output XML file, unless it is disabled
by using output=None
. Generating log, report, and xUnit files based on
the results is possible using the
ResultWriter
class:
from robot.api import ResultWriter
# Report and xUnit files can be generated based on the result object.
ResultWriter(result).write_results(report='skynet.html', log=None)
# Generating log files requires processing the earlier generated output XML.
ResultWriter('skynet.xml').write_results()
Subpackages¶
- robot.running.arguments package
- Submodules
- robot.running.arguments.argumentmapper module
- robot.running.arguments.argumentparser module
- robot.running.arguments.argumentresolver module
- robot.running.arguments.argumentspec module
- robot.running.arguments.argumentvalidator module
- robot.running.arguments.embedded module
- robot.running.arguments.javaargumentcoercer module
- robot.running.timeouts package
Submodules¶
robot.running.builder module¶
-
class
robot.running.builder.
TestSuiteBuilder
(include_suites=None, warn_on_skipped='DEPRECATED', extension=None, rpa=None)[source]¶ Bases:
object
Creates executable
TestSuite
objects.Suites are build based on existing test data on the file system.
See the overall documentation of the
robot.running
package for more information and examples.Parameters: - include_suites – List of suite names to include. If
None
or an empty list, all suites are included. When executing tests normally, these names are specified using the--suite
option. - warn_on_skipped – Deprecated.
- extension – Limit parsing test data to only these files. Files
are specified as an extension that is handled case-insensitively.
Same as
--extension
on the command line. - rpa – Explicit test execution mode.
True
for RPA andFalse
for test automation. By default mode is got from test data headers and possible conflicting headers cause an error.
- include_suites – List of suite names to include. If
robot.running.context module¶
robot.running.defaults module¶
robot.running.dynamicmethods module¶
-
class
robot.running.dynamicmethods.
GetKeywordNames
(lib)[source]¶ Bases:
robot.running.dynamicmethods._DynamicMethod
-
name
¶
-
-
class
robot.running.dynamicmethods.
RunKeyword
(lib)[source]¶ Bases:
robot.running.dynamicmethods._DynamicMethod
-
supports_kwargs
¶
-
name
¶
-
-
class
robot.running.dynamicmethods.
GetKeywordDocumentation
(lib)[source]¶ Bases:
robot.running.dynamicmethods._DynamicMethod
-
name
¶
-
-
class
robot.running.dynamicmethods.
GetKeywordArguments
(lib)[source]¶ Bases:
robot.running.dynamicmethods._DynamicMethod
-
name
¶
-
robot.running.handlers module¶
robot.running.handlerstore module¶
robot.running.librarykeywordrunner module¶
-
class
robot.running.librarykeywordrunner.
LibraryKeywordRunner
(handler, name=None)[source]¶ Bases:
object
-
library
¶
-
libname
¶
-
longname
¶
-
robot.running.libraryscopes module¶
robot.running.model module¶
Module implementing test execution related model objects.
When tests are executed normally, these objects are created based on the test
data on the file system by TestSuiteBuilder
, but external
tools can also create an executable test suite model structure directly.
Regardless the approach to create it, the model is executed by calling
run()
method of the root test suite. See the
robot.running
package level documentation for more information and
examples.
The most important classes defined in this module are TestSuite
,
TestCase
and Keyword
. When tests are executed, these objects
can be inspected and modified by pre-run modifiers and listeners.
The aforementioned objects are considered stable, but other objects in this
module may still be changed in the future major releases.
-
class
robot.running.model.
Keyword
(name='', doc='', args=(), assign=(), tags=(), timeout=None, type='kw')[source]¶ Bases:
robot.model.keyword.Keyword
Represents a single executable keyword.
These keywords never have child keywords or messages. The actual keyword that is executed depends on the context where this model is executed.
See the base class for documentation of attributes not documented here.
-
message_class
= None¶ Internal usage only.
-
run
(context)[source]¶ Execute the keyword.
Typically called internally by
TestSuite.run()
.
-
FOR_ITEM_TYPE
= 'foritem'¶
-
FOR_LOOP_TYPE
= 'for'¶
-
KEYWORD_TYPE
= 'kw'¶
-
SETUP_TYPE
= 'setup'¶
-
TEARDOWN_TYPE
= 'teardown'¶
-
args
¶
-
assign
¶
-
copy
(**attributes)¶ Return shallow copy of this object.
Parameters: attributes – Attributes to be set for the returned copy automatically. For example, test.copy(name='New name')
.See also
deepcopy()
. The difference between these two is the same as with the standardcopy.copy
andcopy.deepcopy
functions that these methods also use internally.New in Robot Framework 3.0.1.
-
deepcopy
(**attributes)¶ Return deep copy of this object.
Parameters: attributes – Attributes to be set for the returned copy automatically. For example, test.deepcopy(name='New name')
.See also
copy()
. The difference between these two is the same as with the standardcopy.copy
andcopy.deepcopy
functions that these methods also use internally.New in Robot Framework 3.0.1.
-
doc
¶
-
id
¶ Keyword id in format like
s1-t3-k1
.See
TestSuite.id
for more information.
-
keyword_class
= None¶
-
name
¶
-
parent
¶ Parent test suite, test case or keyword.
Keyword tags as a
Tags
object.
-
timeout
¶
-
type
¶
-
visit
(visitor)[source]¶ Visitor interface
entry-point.
-
-
class
robot.running.model.
ForLoop
(variables, values, flavor)[source]¶ Bases:
robot.running.model.Keyword
Represents a for loop in test data.
Contains keywords in the loop body as child
keywords
.-
flavor
¶
-
variables
¶
-
values
¶
-
FOR_ITEM_TYPE
= 'foritem'¶
-
FOR_LOOP_TYPE
= 'for'¶
-
KEYWORD_TYPE
= 'kw'¶
-
SETUP_TYPE
= 'setup'¶
-
TEARDOWN_TYPE
= 'teardown'¶
-
args
¶
-
assign
¶
-
copy
(**attributes)¶ Return shallow copy of this object.
Parameters: attributes – Attributes to be set for the returned copy automatically. For example, test.copy(name='New name')
.See also
deepcopy()
. The difference between these two is the same as with the standardcopy.copy
andcopy.deepcopy
functions that these methods also use internally.New in Robot Framework 3.0.1.
-
deepcopy
(**attributes)¶ Return deep copy of this object.
Parameters: attributes – Attributes to be set for the returned copy automatically. For example, test.deepcopy(name='New name')
.See also
copy()
. The difference between these two is the same as with the standardcopy.copy
andcopy.deepcopy
functions that these methods also use internally.New in Robot Framework 3.0.1.
-
doc
¶
-
id
¶ Keyword id in format like
s1-t3-k1
.See
TestSuite.id
for more information.
-
message_class
= None¶
-
name
¶
-
parent
¶ Parent test suite, test case or keyword.
-
run
(context)¶ Execute the keyword.
Typically called internally by
TestSuite.run()
.
Keyword tags as a
Tags
object.
-
timeout
¶
-
type
¶
-
visit
(visitor)¶ Visitor interface
entry-point.
-
-
class
robot.running.model.
TestCase
(name='', doc='', tags=None, timeout=None, template=None)[source]¶ Bases:
robot.model.testcase.TestCase
Represents a single executable test case.
See the base class for documentation of attributes not documented here.
-
template
¶ Name of the keyword that has been used as template when building the test.
None
if no is template used.
-
timeout
¶ Test timeout as a
Timeout
instance orNone
.This attribute is likely to change in the future.
-
copy
(**attributes)¶ Return shallow copy of this object.
Parameters: attributes – Attributes to be set for the returned copy automatically. For example, test.copy(name='New name')
.See also
deepcopy()
. The difference between these two is the same as with the standardcopy.copy
andcopy.deepcopy
functions that these methods also use internally.New in Robot Framework 3.0.1.
-
deepcopy
(**attributes)¶ Return deep copy of this object.
Parameters: attributes – Attributes to be set for the returned copy automatically. For example, test.deepcopy(name='New name')
.See also
copy()
. The difference between these two is the same as with the standardcopy.copy
andcopy.deepcopy
functions that these methods also use internally.New in Robot Framework 3.0.1.
-
doc
¶
-
id
¶ Test case id in format like
s1-t3
.See
TestSuite.id
for more information.
-
longname
¶ Test name prefixed with the long name of the parent suite.
-
name
¶
-
parent
¶
Test tags as a
Tags
object.
-
visit
(visitor)[source]¶ Visitor interface
entry-point.
-
-
class
robot.running.model.
TestSuite
(name='', doc='', metadata=None, source=None, rpa=False)[source]¶ Bases:
robot.model.testsuite.TestSuite
Represents a single executable test suite.
See the base class for documentation of attributes not documented here.
-
resource
¶ ResourceFile
instance containing imports, variables and keywords the suite owns. When data is parsed from the file system, this data comes from the same test case file that creates the suite.
-
configure
(randomize_suites=False, randomize_tests=False, randomize_seed=None, **options)[source]¶ A shortcut to configure a suite using one method call.
Can only be used with the root test suite.
Parameters: - randomize_xxx – Passed to
randomize()
. - options – Passed to
SuiteConfigurer
that will then set suite attributes, callfilter()
, etc. as needed.
Example:
suite.configure(included_tags=['smoke'], doc='Smoke test results.')
- randomize_xxx – Passed to
-
randomize
(suites=True, tests=True, seed=None)[source]¶ Randomizes the order of suites and/or tests, recursively.
Parameters: - suites – Boolean controlling should suites be randomized.
- tests – Boolean controlling should tests be randomized.
- seed – Random seed. Can be given if previous random order needs to be re-created. Seed value is always shown in logs and reports.
-
run
(settings=None, **options)[source]¶ Executes the suite based based the given
settings
oroptions
.Parameters: - settings –
RobotSettings
object to configure test execution. - options – Used to construct new
RobotSettings
object ifsettings
are not given.
Returns: Result
object with information about executed suites and tests.If
options
are used, their names are the same as long command line options except without hyphens. Some options are ignored (see below), but otherwise they have the same semantics as on the command line. Options that can be given on the command line multiple times can be passed as lists likevariable=['VAR1:value1', 'VAR2:value2']
. If such an option is used only once, it can be given also as a single string likevariable='VAR:value'
.Additionally listener option allows passing object directly instead of listener name, e.g.
run('tests.robot', listener=Listener())
.To capture stdout and/or stderr streams, pass open file objects in as special keyword arguments
stdout
andstderr
, respectively.Only options related to the actual test execution have an effect. For example, options related to selecting or modifying test cases or suites (e.g.
--include
,--name
,--prerunmodifier
) or creating logs and reports are silently ignored. The output XML generated as part of the execution can be configured, though. This includes disabling it withoutput=None
.Example:
stdout = StringIO() result = suite.run(variable='EXAMPLE:value', critical='regression', output='example.xml', exitonfailure=True, stdout=stdout) print result.return_code
To save memory, the returned
Result
object does not have any information about the executed keywords. If that information is needed, the created output XML file needs to be read using theExecutionResult
factory method.See the
package level
documentation for more examples, including how to construct executable test suites and how to create logs and reports based on the execution results.See the
robot.run
function for a higher-level API for executing tests in files or directories.- settings –
-
copy
(**attributes)¶ Return shallow copy of this object.
Parameters: attributes – Attributes to be set for the returned copy automatically. For example, test.copy(name='New name')
.See also
deepcopy()
. The difference between these two is the same as with the standardcopy.copy
andcopy.deepcopy
functions that these methods also use internally.New in Robot Framework 3.0.1.
-
deepcopy
(**attributes)¶ Return deep copy of this object.
Parameters: attributes – Attributes to be set for the returned copy automatically. For example, test.deepcopy(name='New name')
.See also
copy()
. The difference between these two is the same as with the standardcopy.copy
andcopy.deepcopy
functions that these methods also use internally.New in Robot Framework 3.0.1.
-
doc
¶
-
filter
(included_suites=None, included_tests=None, included_tags=None, excluded_tags=None)[source]¶ Select test cases and remove others from this suite.
Parameters have the same semantics as
--suite
,--test
,--include
, and--exclude
command line options. All of them can be given as a list of strings, or when selecting only one, as a single string.Child suites that contain no tests after filtering are automatically removed.
Example:
suite.filter(included_tests=['Test 1', '* Example'], included_tags='priority-1')
-
id
¶ An automatically generated unique id.
The root suite has id
s1
, its child suites have idss1-s1
,s1-s2
, …, their child suites get idss1-s1-s1
,s1-s1-s2
, …,s1-s2-s1
, …, and so on.The first test in a suite has an id like
s1-t1
, the second has an ids1-t2
, and so on. Similarly keywords in suites (setup/teardown) and in tests get ids likes1-k1
,s1-t1-k1
, ands1-s4-t2-k5
.
-
longname
¶ Suite name prefixed with the long name of the parent suite.
-
metadata
¶ Free test suite metadata as a dictionary.
-
name
¶ Test suite name. If not set, constructed from child suite names.
-
parent
¶
-
rpa
¶
Add and/or remove specified tags to the tests in this suite.
Parameters: - add – Tags to add as a list or, if adding only one, as a single string.
- remove – Tags to remove as a list or as a single string.
Can be given as patterns where
*
and?
work as wildcards. - persist – Add/remove specified tags also to new tests added to this suite in the future.
-
source
¶
-
suites
¶ Child suites as a
TestSuites
object.
-
test_count
¶ Number of the tests in this suite, recursively.
-
visit
(visitor)[source]¶ Visitor interface
entry-point.
-
robot.running.namespace module¶
-
class
robot.running.namespace.
Namespace
(variables, suite, resource)[source]¶ Bases:
object
-
libraries
¶
-
robot.running.outputcapture module¶
robot.running.randomizer module¶
-
class
robot.running.randomizer.
Randomizer
(randomize_suites=True, randomize_tests=True, seed=None)[source]¶ Bases:
robot.model.visitor.SuiteVisitor
-
start_suite
(suite)[source]¶ Called when suite starts. Default implementation does nothing.
Can return explicit
False
to stop visiting.
-
visit_test
(test)[source]¶ Implements traversing through the test and its keywords.
Can be overridden to allow modifying the passed in
test
without callingstart_test()
orend_test()
nor visiting keywords.
-
visit_keyword
(kw)[source]¶ Implements traversing through the keyword and its child keywords.
Can be overridden to allow modifying the passed in
kw
without callingstart_keyword()
orend_keyword()
nor visiting child keywords.
-
end_keyword
(keyword)¶ Called when keyword ends. Default implementation does nothing.
-
end_message
(msg)¶ Called when message ends. Default implementation does nothing.
-
end_suite
(suite)¶ Called when suite ends. Default implementation does nothing.
-
end_test
(test)¶ Called when test ends. Default implementation does nothing.
-
start_keyword
(keyword)¶ Called when keyword starts. Default implementation does nothing.
Can return explicit
False
to stop visiting.
-
start_message
(msg)¶ Called when message starts. Default implementation does nothing.
Can return explicit
False
to stop visiting.
-
start_test
(test)¶ Called when test starts. Default implementation does nothing.
Can return explicit
False
to stop visiting.
-
visit_message
(msg)¶ Implements visiting the message.
Can be overridden to allow modifying the passed in
msg
without callingstart_message()
orend_message()
.
-
visit_suite
(suite)¶ Implements traversing through the suite and its direct children.
Can be overridden to allow modifying the passed in
suite
without callingstart_suite()
orend_suite()
nor visiting child suites, tests or keywords (setup and teardown) at all.
-
robot.running.runkwregister module¶
robot.running.runner module¶
-
class
robot.running.runner.
Runner
(output, settings)[source]¶ Bases:
robot.model.visitor.SuiteVisitor
-
start_suite
(suite)[source]¶ Called when suite starts. Default implementation does nothing.
Can return explicit
False
to stop visiting.
-
visit_test
(test)[source]¶ Implements traversing through the test and its keywords.
Can be overridden to allow modifying the passed in
test
without callingstart_test()
orend_test()
nor visiting keywords.
-
end_keyword
(keyword)¶ Called when keyword ends. Default implementation does nothing.
-
end_message
(msg)¶ Called when message ends. Default implementation does nothing.
-
end_test
(test)¶ Called when test ends. Default implementation does nothing.
-
start_keyword
(keyword)¶ Called when keyword starts. Default implementation does nothing.
Can return explicit
False
to stop visiting.
-
start_message
(msg)¶ Called when message starts. Default implementation does nothing.
Can return explicit
False
to stop visiting.
-
start_test
(test)¶ Called when test starts. Default implementation does nothing.
Can return explicit
False
to stop visiting.
-
visit_keyword
(kw)¶ Implements traversing through the keyword and its child keywords.
Can be overridden to allow modifying the passed in
kw
without callingstart_keyword()
orend_keyword()
nor visiting child keywords.
-
visit_message
(msg)¶ Implements visiting the message.
Can be overridden to allow modifying the passed in
msg
without callingstart_message()
orend_message()
.
-
visit_suite
(suite)¶ Implements traversing through the suite and its direct children.
Can be overridden to allow modifying the passed in
suite
without callingstart_suite()
orend_suite()
nor visiting child suites, tests or keywords (setup and teardown) at all.
-
robot.running.signalhandler module¶
robot.running.status module¶
-
class
robot.running.status.
Exit
(failure_mode=False, error_mode=False, skip_teardown_mode=False)[source]¶ Bases:
object
-
teardown_allowed
¶
-
-
class
robot.running.status.
SuiteStatus
(parent=None, exit_on_failure_mode=False, exit_on_error_mode=False, skip_teardown_on_exit_mode=False)[source]¶ Bases:
robot.running.status._ExecutionStatus
-
critical_failure_occurred
()¶
-
error_occurred
()¶
-
failures
¶
-
message
¶
-
setup_executed
(failure=None)¶
-
status
¶
-
teardown_allowed
¶
-
teardown_executed
(failure=None)¶
-
-
class
robot.running.status.
TestStatus
(parent, critical)[source]¶ Bases:
robot.running.status._ExecutionStatus
-
critical_failure_occurred
()¶
-
error_occurred
()¶
-
failures
¶
-
message
¶
-
setup_executed
(failure=None)¶
-
status
¶
-
teardown_allowed
¶
-
teardown_executed
(failure=None)¶
-
-
class
robot.running.status.
TestMessage
(status)[source]¶ Bases:
robot.running.status._Message
-
setup_message
= 'Setup failed:\n%s'¶
-
teardown_message
= 'Teardown failed:\n%s'¶
-
also_teardown_message
= '%s\n\nAlso teardown failed:\n%s'¶
-
exit_on_fatal_message
= 'Test execution stopped due to a fatal error.'¶
-
exit_on_failure_message
= 'Critical failure occurred and exit-on-failure mode is in use.'¶
-
exit_on_error_message
= 'Error occurred and exit-on-error mode is in use.'¶
-
message
¶
-
robot.running.statusreporter module¶
robot.running.steprunner module¶
-
class
robot.running.steprunner.
ForInRangeRunner
(context, templated=False)[source]¶ Bases:
robot.running.steprunner.ForInRunner
-
run
(data, name=None)¶
-
-
class
robot.running.steprunner.
ForInZipRunner
(context, templated=False)[source]¶ Bases:
robot.running.steprunner.ForInRunner
-
run
(data, name=None)¶
-
-
class
robot.running.steprunner.
ForInEnumerateRunner
(context, templated=False)[source]¶ Bases:
robot.running.steprunner.ForInRunner
-
run
(data, name=None)¶
-
-
class
robot.running.steprunner.
InvalidForRunner
(context, flavor)[source]¶ Bases:
robot.running.steprunner.ForInRunner
Used to send an error from ForRunner() if it sees an unexpected error.
We can’t simply throw a DataError from ForRunner() because that happens outside the “with StatusReporter(…)” blocks.
-
run
(data, name=None)¶
-
robot.running.testlibraries module¶
robot.running.usererrorhandler module¶
-
class
robot.running.usererrorhandler.
UserErrorHandler
(error, name, libname=None)[source]¶ Bases:
object
Created if creating handlers fail – running raises DataError.
The idea is not to raise DataError at processing time and prevent all tests in affected test case file from executing. Instead UserErrorHandler is created and if it is ever run DataError is raised then.
Parameters: - error (robot.errors.DataError) – Occurred error.
- name (str) – Name of the affected keyword.
- libname (str) – Name of the affected library or resource.
-
longname
¶
-
doc
¶
-
shortdoc
¶
-
dry_run
(kw, context)¶
robot.running.userkeyword module¶
-
class
robot.running.userkeyword.
UserLibrary
(source, source_type='Resource file')[source]¶ Bases:
object
-
TEST_CASE_FILE_TYPE
= 'Test case file'¶
-
RESOURCE_FILE_TYPE
= 'Resource file'¶
-