robot.running package

Implements the core test execution logic.

The main public entry points of this package are of the following two classes:

  • TestSuiteBuilder for creating executable test suites based on existing test case files and directories.
  • TestSuite for creating an executable test suite structure programmatically.

It is recommended to import both of these classes via the robot.api package like in the examples below. Also TestCase and Keyword classes used internally by the TestSuite class are part of the public API. In those rare cases where these classes are needed directly, they can be imported from this package.

This package and especially all public code was rewritten in Robot Framework 2.8 to make it easier to generate and execute test suites programmatically. Rewriting of the test execution logic will continue in future releases, but changes to the public API ought to be relatively small.

Examples

First, let’s assume we have the following test suite in file activate_skynet.robot:

*** Settings ***
Library    OperatingSystem

*** Test Cases ***
Should Activate Skynet
    [Tags]    smoke
    [Setup]    Set Environment Variable    SKYNET    activated
    Environment Variable Should Be Set    SKYNET

We can easily parse and create an executable test suite based on the above file using the TestSuiteBuilder class as follows:

from robot.api import TestSuiteBuilder

suite = TestSuiteBuilder().build('path/to/activate_skynet.robot')

That was easy. Let’s next generate the same test suite from scratch using the TestSuite class:

from robot.api import TestSuite

suite = TestSuite('Activate Skynet')
suite.resource.imports.library('OperatingSystem')
test = suite.tests.create('Should Activate Skynet', tags=['smoke'])
test.keywords.create('Set Environment Variable', args=['SKYNET', 'activated'], type='setup')
test.keywords.create('Environment Variable Should Be Set', args=['SKYNET'])

Not that complicated either, especially considering the flexibility. Notice that the suite created based on the file could also be edited further using the same API.

Now that we have a test suite ready, let’s execute it and verify that the returned Result object contains correct information:

result = suite.run(critical='smoke', output='skynet.xml')

assert result.return_code == 0
assert result.suite.name == 'Activate Skynet'
test = result.suite.tests[0]
assert test.name == 'Should Activate Skynet'
assert test.passed and test.critical
stats = result.suite.statistics
assert stats.critical.total == 1 and stats.critical.failed == 0

Running the suite generates a normal output XML file, unless it is disabled by using output=None. Generating log, report, and xUnit files based on the results is possible using the ResultWriter class:

from robot.api import ResultWriter

# Report and xUnit files can be generated based on the result object.
ResultWriter(result).write_results(report='skynet.html', log=None)
# Generating log files requires processing the earlier generated output XML.
ResultWriter('skynet.xml').write_results()

Submodules

robot.running.builder module

class robot.running.builder.TestSuiteBuilder(include_suites=None, warn_on_skipped=False)[source]

Bases: object

Creates executable TestSuite objects.

Suites are build based on existing test data on the file system.

See the overall documentation of the robot.running package for more information and examples.

Parameters:
  • include_suites – List of suite names to include. If None or an empty list, all suites are included. When executing tests normally, these names are specified using the --suite option.
  • warn_on_skipped – Boolean to control should a warning be emitted if a file is skipped because it cannot be parsed or should it be ignored silently. When executing tests normally, this value is set with the --warnonskippedfiles option.
build(*paths)[source]
Parameters:paths – Paths to test data files or directories.
Returns:TestSuite instance.
class robot.running.builder.ResourceFileBuilder[source]

Bases: object

build(path_or_data, target=None)[source]
class robot.running.builder.StepBuilder[source]

Bases: object

build_steps(parent, data, template=None, kw_type='kw')[source]
build_step(parent, data, template=None, kw_type='kw')[source]

robot.running.context module

class robot.running.context.ExecutionContexts[source]

Bases: object

current
top
namespaces
start_suite(suite, namespace, output, dry_run=False)[source]
end_suite()[source]

robot.running.defaults module

class robot.running.defaults.TestDefaults(settings, parent=None)[source]

Bases: object

get_test_values(test)[source]
class robot.running.defaults.TestValues(test, defaults)[source]

Bases: object

robot.running.dynamicmethods module

robot.running.dynamicmethods.no_dynamic_method(*args)[source]
class robot.running.dynamicmethods.GetKeywordNames(lib)[source]

Bases: robot.running.dynamicmethods._DynamicMethod

name
class robot.running.dynamicmethods.RunKeyword(lib)[source]

Bases: robot.running.dynamicmethods._DynamicMethod

supports_kwargs
name
class robot.running.dynamicmethods.GetKeywordDocumentation(lib)[source]

Bases: robot.running.dynamicmethods._DynamicMethod

name
class robot.running.dynamicmethods.GetKeywordArguments(lib)[source]

Bases: robot.running.dynamicmethods._DynamicMethod

name

robot.running.handlers module

robot.running.handlers.Handler(library, name, method)[source]
robot.running.handlers.DynamicHandler(library, name, method, doc, argspec)[source]
robot.running.handlers.InitHandler(library, method, docgetter=None)[source]
class robot.running.handlers.EmbeddedArgumentsHandler(name_regexp, orig_handler)[source]

Bases: object

matches(name)[source]
create_runner(name)[source]

robot.running.handlerstore module

class robot.running.handlerstore.HandlerStore(source, source_type)[source]

Bases: object

TEST_LIBRARY_TYPE = 'Test library'
TEST_CASE_FILE_TYPE = 'Test case file'
RESOURCE_FILE_TYPE = 'Resource file'
add(handler, embedded=False)[source]
create_runner(name)[source]

robot.running.importer module

class robot.running.importer.Importer[source]

Bases: object

reset()[source]
close_global_library_listeners()[source]
import_library(name, args, alias, variables)[source]
import_resource(path)[source]
class robot.running.importer.ImportCache[source]

Bases: object

Keeps track on and optionally caches imported items.

Handles paths in keys case-insensitively on case-insensitive OSes. Unlike dicts, this storage accepts mutable values in keys.

add(key, item=None)[source]
values()[source]

robot.running.librarykeywordrunner module

class robot.running.librarykeywordrunner.LibraryKeywordRunner(handler, name=None)[source]

Bases: object

library
libname
longname
run(kw, context)[source]
dry_run(kw, context)[source]
class robot.running.librarykeywordrunner.EmbeddedArgumentsRunner(handler, name)[source]

Bases: robot.running.librarykeywordrunner.LibraryKeywordRunner

dry_run(kw, context)
libname
library
longname
run(kw, context)
class robot.running.librarykeywordrunner.RunKeywordRunner(handler, default_dry_run_keywords=False)[source]

Bases: robot.running.librarykeywordrunner.LibraryKeywordRunner

dry_run(kw, context)
libname
library
longname
run(kw, context)

robot.running.libraryscopes module

robot.running.libraryscopes.LibraryScope(libcode, library)[source]
class robot.running.libraryscopes.GlobalScope(library)[source]

Bases: object

is_global = True
start_suite()[source]
end_suite()[source]
start_test()[source]
end_test()[source]
class robot.running.libraryscopes.TestSuiteScope(library)[source]

Bases: robot.running.libraryscopes.GlobalScope

is_global
start_suite()[source]
end_suite()[source]
end_test()
start_test()
class robot.running.libraryscopes.TestCaseScope(library)[source]

Bases: robot.running.libraryscopes.TestSuiteScope

start_test()[source]
end_test()[source]
end_suite()
is_global
start_suite()

robot.running.model module

Module implementing test execution related model objects.

When tests are executed normally, these objects are created based on the test data on the file system by TestSuiteBuilder, but external tools can also create an executable test suite model structure directly. Regardless the approach to create it, the model is executed by calling run() method of the root test suite. See the robot.running package level documentation for more information and examples.

The most important classes defined in this module are TestSuite, TestCase and Keyword. When tests are executed, these objects can be inspected and modified by pre-run modifiers and listeners. The aforementioned objects are considered stable, but other objects in this module may still be changed in the future major releases.

class robot.running.model.Keyword(name='', doc='', args=(), assign=(), tags=(), timeout=None, type='kw')[source]

Bases: robot.model.keyword.Keyword

Represents a single executable keyword.

These keywords never have child keywords or messages. The actual keyword that is executed depends on the context where this model is executed.

See the base class for documentation of attributes not documented here.

message_class = None

Internal usage only.

run(context)[source]

Execute the keyword.

Typically called internally by TestSuite.run().

FOR_ITEM_TYPE = 'foritem'
FOR_LOOP_TYPE = 'for'
KEYWORD_TYPE = 'kw'
SETUP_TYPE = 'setup'
TEARDOWN_TYPE = 'teardown'
args
assign
children

Child keywords and messages in creation order.

doc
id

Keyword id in format like s1-t3-k1.

See TestSuite.id for more information.

keyword_class = None
keywords

Child keywords as a Keywords object.

messages

Messages as a Messages object.

name
parent

Parent test suite, test case or keyword.

tags

Keyword tags as a Tags object.

timeout
type
visit(visitor)[source]

Visitor interface entry-point.

class robot.running.model.ForLoop(variables, values, flavor)[source]

Bases: robot.running.model.Keyword

Represents a for loop in test data.

Contains keywords in the loop body as child keywords.

keyword_class

Internal usage only.

alias of Keyword

flavor
variables
values
FOR_ITEM_TYPE = 'foritem'
FOR_LOOP_TYPE = 'for'
KEYWORD_TYPE = 'kw'
SETUP_TYPE = 'setup'
TEARDOWN_TYPE = 'teardown'
args
assign
children

Child keywords and messages in creation order.

doc
id

Keyword id in format like s1-t3-k1.

See TestSuite.id for more information.

keywords

Child keywords as a Keywords object.

message_class = None
messages

Messages as a Messages object.

name
parent

Parent test suite, test case or keyword.

run(context)

Execute the keyword.

Typically called internally by TestSuite.run().

tags

Keyword tags as a Tags object.

timeout
type
visit(visitor)

Visitor interface entry-point.

class robot.running.model.TestCase(name='', doc='', tags=None, timeout=None, template=None)[source]

Bases: robot.model.testcase.TestCase

Represents a single executable test case.

See the base class for documentation of attributes not documented here.

keyword_class

Internal usage only.

alias of Keyword

template

Name of the keyword that has been used as template when building the test. None if no is template used.

timeout

Test timeout as a Timeout instance or None.

This attribute is likely to change in the future.

doc
id

Test case id in format like s1-t3.

See TestSuite.id for more information.

keywords

Keywords as a Keywords object.

Contains also possible setup and teardown keywords.

longname

Test name prefixed with the long name of the parent suite.

name
parent
tags

Test tags as a Tags object.

visit(visitor)[source]

Visitor interface entry-point.

class robot.running.model.TestSuite(name='', doc='', metadata=None, source=None)[source]

Bases: robot.model.testsuite.TestSuite

Represents a single executable test suite.

See the base class for documentation of attributes not documented here.

test_class

Internal usage only.

alias of TestCase

keyword_class

Internal usage only.

alias of Keyword

resource

ResourceFile instance containing imports, variables and keywords the suite owns. When data is parsed from the file system, this data comes from the same test case file that creates the suite.

imports

Deprecated. Use TestSuite.resource.imports instead.

variables

Deprecated. Use TestSuite.resource.variables instead.

user_keywords

Deprecated. Use TestSuite.resource.keywords instead.

configure(randomize_suites=False, randomize_tests=False, randomize_seed=None, **options)[source]

A shortcut to configure a suite using one method call.

Parameters:

Example:

suite.configure(included_tags=['smoke'],
                doc='Smoke test results.')
randomize(suites=True, tests=True, seed=None)[source]

Randomizes the order of suites and/or tests, recursively.

Parameters:
  • suites – Boolean controlling should suites be randomized.
  • tests – Boolean controlling should tests be randomized.
  • seed – Random seed. Can be given if previous random order needs to be re-created. Seed value is always shown in logs and reports.
run(settings=None, **options)[source]

Executes the suite based based the given settings or options.

Parameters:
  • settingsRobotSettings object to configure test execution.
  • options – Used to construct new RobotSettings object if settings are not given.
Returns:

Result object with information about executed suites and tests.

If options are used, their names are the same as long command line options except without hyphens, and they also have the same semantics. Options that can be given on the command line multiple times can be passed as lists like variable=['VAR1:value1', 'VAR2:value2']. If such an option is used only once, it can be given also as a single string like variable='VAR:value'.

Additionally listener option allows passing object directly instead of listener name, e.g. run('tests.robot', listener=Listener()).

To capture stdout and/or stderr streams, pass open file objects in as special keyword arguments stdout and stderr, respectively. Note that this works only in version 2.8.4 and newer.

Only options related to the actual test execution have an effect. For example, options related to selecting test cases or creating logs and reports are silently ignored. The output XML generated as part of the execution can be configured, though. This includes disabling it with output=None.

Example:

stdout = StringIO()
result = suite.run(variable='EXAMPLE:value',
                   critical='regression',
                   output='example.xml',
                   exitonfailure=True,
                   stdout=stdout)
print result.return_code

To save memory, the returned Result object does not have any information about the executed keywords. If that information is needed, the created output XML file needs to be read using the ExecutionResult factory method.

See the package level documentation for more examples, including how to construct executable test suites and how to create logs and reports based on the execution results.

See the robot.run function for a higher-level API for executing tests in files or directories.

doc
filter(included_suites=None, included_tests=None, included_tags=None, excluded_tags=None)[source]

Select test cases and remove others from this suite.

Parameters have the same semantics as --suite, --test, --include, and --exclude command line options. All of them can be given as a list of strings, or when selecting only one, as a single string.

Child suites that contain no tests after filtering are automatically removed.

Example:

suite.filter(included_tests=['Test 1', '* Example'],
             included_tags='priority-1')
id

An automatically generated unique id.

The root suite has id s1, its child suites have ids s1-s1, s1-s2, ..., their child suites get ids s1-s1-s1, s1-s1-s2, ..., s1-s2-s1, ..., and so on.

The first test in a suite has an id like s1-t1, the second has an id s1-t2, and so on. Similarly keywords in suites (setup/teardown) and in tests get ids like s1-k1, s1-t1-k1, and s1-s4-t2-k5.

keywords

Suite setup and teardown as a Keywords object.

longname

Suite name prefixed with the long name of the parent suite.

metadata

Free test suite metadata as a dictionary.

name

Test suite name. If not set, constructed from child suite names.

parent
remove_empty_suites()[source]

Removes all child suites not containing any tests, recursively.

set_tags(add=None, remove=None, persist=False)[source]

Add and/or remove specified tags to the tests in this suite.

Parameters:
  • add – Tags to add as a list or, if adding only one, as a single string.
  • remove – Tags to remove as a list or as a single string. Can be given as patterns where * and ? work as wildcards.
  • persist – Add/remove specified tags also to new tests added to this suite in the future.
source
suites

Child suites as a TestSuites object.

test_count

Number of the tests in this suite, recursively.

tests

Tests as a TestCases object.

visit(visitor)[source]

Visitor interface entry-point.

class robot.running.model.Variable(name, value, source=None)[source]

Bases: object

report_invalid_syntax(message, level='ERROR')[source]
class robot.running.model.Timeout(value, message=None)[source]

Bases: object

class robot.running.model.ResourceFile(doc='', source=None)[source]

Bases: object

imports
keywords
variables
class robot.running.model.UserKeyword(name, args=(), doc='', tags=(), return_=None, timeout=None)[source]

Bases: object

keywords
timeout

Keyword timeout as a Timeout instance or None.

tags

robot.running.namespace module

class robot.running.namespace.Namespace(variables, suite, resource)[source]

Bases: object

libraries
handle_imports()[source]
import_resource(name, overwrite=True)[source]
import_variables(name, args, overwrite=False)[source]
import_library(name, args=None, alias=None, notify=True)[source]
set_search_order(new_order)[source]
start_test()[source]
end_test()[source]
start_suite()[source]
end_suite()[source]
start_user_keyword()[source]
end_user_keyword()[source]
get_library_instance(libname)[source]
get_library_instances()[source]
reload_library(libname_or_instance)[source]
get_runner(name)[source]
class robot.running.namespace.KeywordStore(resource)[source]

Bases: object

get_library(name_or_instance)[source]
get_runner(name)[source]
class robot.running.namespace.KeywordRecommendationFinder(user_keywords, libraries, resources)[source]

Bases: object

recommend_similar_keywords(name)[source]

Return keyword names similar to name.

static format_recommendations(msg, recommendations)[source]

robot.running.outputcapture module

class robot.running.outputcapture.OutputCapturer(library_import=False)[source]

Bases: object

class robot.running.outputcapture.PythonCapturer(stdout=True)[source]

Bases: object

release()[source]
class robot.running.outputcapture.JavaCapturer(stdout=True)[source]

Bases: object

release()[source]

robot.running.randomizer module

class robot.running.randomizer.Randomizer(randomize_suites=True, randomize_tests=True, seed=None)[source]

Bases: robot.model.visitor.SuiteVisitor

start_suite(suite)[source]
visit_test(test)[source]
visit_keyword(kw)[source]
end_keyword(keyword)

Called when keyword ends. Default implementation does nothing.

end_message(msg)

Called when message ends. Default implementation does nothing.

end_suite(suite)

Called when suite ends. Default implementation does nothing.

end_test(test)

Called when test ends. Default implementation does nothing.

start_keyword(keyword)

Called when keyword starts. Default implementation does nothing.

Can return explicit False to stop visiting.

start_message(msg)

Called when message starts. Default implementation does nothing.

Can return explicit False to stop visiting.

start_test(test)

Called when test starts. Default implementation does nothing.

Can return explicit False to stop visiting.

visit_message(msg)

Implements visiting the message.

Can be overridden to allow modifying the passed in msg without calling start_message() or end_message().

visit_suite(suite)

Implements traversing through the suite and its direct children.

Can be overridden to allow modifying the passed in suite without calling start_suite() or end_suite() nor visiting child suites, tests or keywords (setup and teardown) at all.

robot.running.runkwregister module

robot.running.runner module

class robot.running.runner.Runner(output, settings)[source]

Bases: robot.model.visitor.SuiteVisitor

start_suite(suite)[source]
end_suite(suite)[source]
visit_test(test)[source]
end_keyword(keyword)

Called when keyword ends. Default implementation does nothing.

end_message(msg)

Called when message ends. Default implementation does nothing.

end_test(test)

Called when test ends. Default implementation does nothing.

start_keyword(keyword)

Called when keyword starts. Default implementation does nothing.

Can return explicit False to stop visiting.

start_message(msg)

Called when message starts. Default implementation does nothing.

Can return explicit False to stop visiting.

start_test(test)

Called when test starts. Default implementation does nothing.

Can return explicit False to stop visiting.

visit_keyword(kw)

Implements traversing through the keyword and its child keywords.

Can be overridden to allow modifying the passed in kw without calling start_keyword() or end_keyword() nor visiting child keywords.

visit_message(msg)

Implements visiting the message.

Can be overridden to allow modifying the passed in msg without calling start_message() or end_message().

visit_suite(suite)

Implements traversing through the suite and its direct children.

Can be overridden to allow modifying the passed in suite without calling start_suite() or end_suite() nor visiting child suites, tests or keywords (setup and teardown) at all.

class robot.running.runner.ModelCombiner(data, result, **priority)[source]

Bases: object

robot.running.signalhandler module

robot.running.status module

class robot.running.status.Failure[source]

Bases: object

class robot.running.status.Exit(failure_mode=False, error_mode=False, skip_teardown_mode=False)[source]

Bases: object

failure_occurred(failure=None, critical=False)[source]
error_occurred()[source]
teardown_allowed
class robot.running.status.SuiteStatus(parent=None, exit_on_failure_mode=False, exit_on_error_mode=False, skip_teardown_on_exit_mode=False)[source]

Bases: robot.running.status._ExecutionStatus

critical_failure_occurred()
error_occurred()
failures
message
setup_executed(failure=None)
status
teardown_allowed
teardown_executed(failure=None)
class robot.running.status.TestStatus(parent, critical)[source]

Bases: robot.running.status._ExecutionStatus

test_failed(failure)[source]
critical_failure_occurred()
error_occurred()
failures
message
setup_executed(failure=None)
status
teardown_allowed
teardown_executed(failure=None)
class robot.running.status.TestMessage(status)[source]

Bases: robot.running.status._Message

setup_message = 'Setup failed:\n%s'
teardown_message = 'Teardown failed:\n%s'
also_teardown_message = '%s\n\nAlso teardown failed:\n%s'
exit_on_fatal_message = 'Test execution stopped due to a fatal error.'
exit_on_failure_message = 'Critical failure occurred and exit-on-failure mode is in use.'
exit_on_error_message = 'Error occurred and exit-on-error mode is in use.'
message
class robot.running.status.SuiteMessage(status)[source]

Bases: robot.running.status._Message

setup_message = 'Suite setup failed:\n%s'
teardown_message = 'Suite teardown failed:\n%s'
also_teardown_message = '%s\n\nAlso suite teardown failed:\n%s'
message
class robot.running.status.ParentMessage(status)[source]

Bases: robot.running.status.SuiteMessage

setup_message = 'Parent suite setup failed:\n%s'
teardown_message = 'Parent suite teardown failed:\n%s'
also_teardown_message = '%s\n\nAlso parent suite teardown failed:\n%s'
message

robot.running.statusreporter module

class robot.running.statusreporter.StatusReporter(context, result, dry_run_lib_kw=False)[source]

Bases: object

robot.running.steprunner module

class robot.running.steprunner.StepRunner(context, templated=False)[source]

Bases: object

run_steps(steps)[source]
run_step(step, name=None)[source]
robot.running.steprunner.ForRunner(context, templated=False, flavor='IN')[source]
class robot.running.steprunner.ForInRunner(context, templated=False)[source]

Bases: object

run(data, name=None)[source]
class robot.running.steprunner.ForInRangeRunner(context, templated=False)[source]

Bases: robot.running.steprunner.ForInRunner

run(data, name=None)
class robot.running.steprunner.ForInZipRunner(context, templated=False)[source]

Bases: robot.running.steprunner.ForInRunner

run(data, name=None)
class robot.running.steprunner.ForInEnumerateRunner(context, templated=False)[source]

Bases: robot.running.steprunner.ForInRunner

run(data, name=None)
class robot.running.steprunner.InvalidForRunner(context, flavor)[source]

Bases: robot.running.steprunner.ForInRunner

Used to send an error from ForRunner() if it sees an unexpected error.

We can’t simply throw a DataError from ForRunner() because that happens outside the “with StatusReporter(...)” blocks.

run(data, name=None)

robot.running.testlibraries module

robot.running.testlibraries.TestLibrary(name, args=None, variables=None, create_handlers=True)[source]

robot.running.usererrorhandler module

class robot.running.usererrorhandler.UserErrorHandler(name, error, libname=None)[source]

Bases: object

Created if creating handlers fail – running raises DataError.

The idea is not to raise DataError at processing time and prevent all tests in affected test case file from executing. Instead UserErrorHandler is created and if it is ever run DataError is raised then.

longname
doc
shortdoc
create_runner(name)[source]
run(kw, context)[source]
dry_run(kw, context)

robot.running.userkeyword module

class robot.running.userkeyword.UserLibrary(source, source_type='Resource file')[source]

Bases: object

TEST_CASE_FILE_TYPE = 'Test case file'
RESOURCE_FILE_TYPE = 'Resource file'
class robot.running.userkeyword.UserKeywordHandler(keyword, libname)[source]

Bases: object

longname
shortdoc
create_runner(name)[source]
class robot.running.userkeyword.EmbeddedArgumentsHandler(keyword, libname, embedded)[source]

Bases: robot.running.userkeyword.UserKeywordHandler

longname
shortdoc
matches(name)[source]
create_runner(name)[source]

robot.running.userkeywordrunner module

class robot.running.userkeywordrunner.UserKeywordRunner(handler, name=None)[source]

Bases: object

longname
libname
arguments
run(kw, context)[source]
dry_run(kw, context)[source]
class robot.running.userkeywordrunner.EmbeddedArgumentsRunner(handler, name)[source]

Bases: robot.running.userkeywordrunner.UserKeywordRunner

arguments
dry_run(kw, context)
libname
longname
run(kw, context)