OpenStack is made-up of tens of projects and each of them with hundreds of possible configurations, making each cloud different. This leads to a situation where the mission of OpenStack, to ‘create an ubiquitous Open Source cloud computing platform’, is getting more complicated. Moving tasks and projects between different clouds providers (private or public) is a nightmare due to the differences between them.
DefCore sets base requirements by defining 1) capabilities, 2) code and 3) must-pass tests for all OpenStack products. This definition uses community resources and involvement to drive interoperability by creating the minimum standards for products labeled “OpenStack.”
RefStack is a source of tools for OpenStack interoperability testing, and what will be used to test our cloud and try to get the ‘OpenStack Compute Powered’ label.
Although the startet-compute:kit tag and the DefCore project are not directly related, we can combine them and work with both so a minimal OpenStack cloud can be deployed that is fully compatible with those products labeled ‘OpenStack’.
As usual we will create an specific user for this task, and get the required code from upstream to run the tests.
root@aio:~# useradd -m refstack root@aio:~# passwd refstack root@aio:~# usermod -a -G sudo refstack
refstack@aio:~# git clone git://git.openstack.org/stackforge/refstack-client
The refstack-client includes a script that will help as to prepare the environment for running all the tempest tests.
refstack@aio:~# cd refstack-client refstack@aio:~/refstack-client# ./setup_env refstack@aio:~/refstack-client# source .venv/bin/activate
Once the environment has been prepared, we need to create a tempest configuration file, tempest.conf, with our cloud environment data.
[auth] tempest_roles = _member_ [identity] auth_version = v2 admin_domain_name = Default admin_tenant_name = stack admin_password = minions admin_username = gru uri_v3 = http://aio:5000/v3 uri = http://aio:5000/v2.0
And we are ready to launch the tests. By default, refstack will executed the whole test suite, but since only the keystone identity service is available, we can limit the scope of the tests to just it.
(.venv)refstack@aio:~/refstack-client# ./refstack-client test -c ~/tempest.conf -vv -- tempest.api.identity ... Ran 161 tests in 184.264s OK 2015-08-20 10:36:06,643 refstack_client:272 INFO Tempest test complete. 2015-08-20 10:36:06,643 refstack_client:273 INFO Subunit results located in: .tempest/.testrepository/1 2015-08-20 10:36:06,664 refstack_client:276 INFO Number of passed tests: 161 2015-08-20 10:36:06,665 refstack_client:288 INFO JSON results saved in: .tempest/.testrepository/1.json
We have conducted a total of 161 tests, and all of them were completed successfully. We can also see how these results stack up against DefCore capabilities. We just need to upload the results to the refstack.net page and check it.
(.venv)stack@aio:~/refstack-client$ REFSTACK_URL=http://refstack.net/api ./refstack-client upload .tempest/.testrepository/1.json Test results will be uploaded to http://refstack.net/api. Ok? (yes/y): y Test results uploaded! URL: http://refstack.net/#/results/cf61b903-2589-43eb-9970-7129e17c595b
This cloud passes 1.8% (2/114) of the tests in the 2015.07 required capabilities for the OpenStack Powered Compute program. Excluding flagged tests, this cloud passes 1.8% (2/109) of the required tests. Compliance with 2015.07: NO
Of course we are not compliant with the OpenStack Powered Compute program (our target for our deployment), since we have just tested the keystone service, but on the plus side, we have passed all the identity service tests, so we are on the right path.
identity-v2-tokens-create [1/1] tempest.api.identity.v2.test_tokens.TokensTest.test_create_token identity-v3-tokens-create [1/1] tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
If we want to test the complete suite, you can do it easily, but at this moment it is pointless
(.venv)stack@aio:~/refstack-client$ ./refstack-client test -c ~/tempest.conf -vv --test-list https://raw.githubusercontent.com/openstack/defcore/master/2015.05/2015.05.required.txt