ansible_based_tests

#3 [WIP] - example how modulatirity testing framework could be integratedinside ansible
Opened by jscotka. Modified
jscotka/ansible_based_tests mtf  into  master

Merge

HI @stefw
I've cteated PoC how MTF can be integrated into ansible.
It schedule tests via special script run-them, what is able to handle various modules and koji builds for modules (actually trigerred by MBS change messages (status done))
If you would like, I can copy tests here, to have tests explicitly here and run then explicitly via avocado, not via magic run-them.sh script :-)

it is done in this run-them.sh script steps like: 

parse module build
set enviromental variables
avocado run /usr/share/moduleframework/examples/testing-module/*

I've created this PR as an feedback for section: Example: Modularity testing Framework
on page https://fedoraproject.org/wiki/Changes/InvokingTestsAnsible
For me it is working well and ansible could be used for invocation MTF tests very simple.

Actually there is just issue and it is how to interpret results, there could be something what is able to parse for example xunit output, or tap output, or whatever what avocado framework is able to produce.
Tests are scheduled by avocado as whole testsuite. I do not expect to run tests sepatatelly, to have there more ansible 'phases' with unique tests, but it is also possible to list tests and then run them separately, change command (you have to imagine that it is ansible 'for' loop :-)): 

avocado run /usr/share/moduleframework/examples/testing-module/*
->
for foo in /usr/share/moduleframework/examples/testing-module/*; do
    avocado run $foo
done

While fine for the moment, this would be something that we would need to document if we allow or not. COPR by nature can be very volatile so I would be not in favor of allowing it.
But that is something to worry about later, this is fine for now :)

testmodule-master-20170410111210 is hard-coded? Could there be a way to retrieve this?

@pingou yep, this package will be in fedora, it is in progress, so that this is just workaround to allow using newest version as soon as possible :-)

I actually does not have testmodule-master-20170410111210 hardcoded anywhere, it is just example. This is identification produced by MBS via fedora fedmsg. So I expect to pass this params from outside (CI, manual trigger) I'm not familiar with ansible, so I've hardcoded it there to show how it can work.
Edited by jscotka

I also don't exactly know how ansible will handle this testing:
Probably there is small confusing what you expect from ansible.

I expect that I'll install module on top of this ansible (rpm based, docker based, I'll prepare test env in my own), probably you expect that this is done by default?
So that I'm using some link to module build like testmodule-master-20170410111210 or could be used some local build of module, it is also possible

@pingou probably I've found this issue, it is caused by my misunderstanding how it should work.
probably I should have module downloaded from dist-git (modules namespace like packages are in /modules/example_distgit.yml
so that I should add something similar for modules like:
fedpkg clone modules/testmodule, in that case it will test what is in dist-git and prepare env for that.
Metadata
None
No Tags
Changes Summary 1
+12
file added
tests/mtf-testmodule/run_tests.yml
Powered by Pagure 1.0.0
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.