Project Update: Containers-based distributed tests runner

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Project Update: Containers-based distributed tests runner

Aravinda
**gluster-tester** is a framework to run existing "*.t" test files in
parallel using containers.

Install and usage instructions are available in the following
repository.

https://github.com/aravindavk/gluster-tester

## Completed:
- Create a base container image with all the dependencies installed.
- Create a tester container image with requested refspec(or latest
master) compiled and installed.
- SSH setup in containers required to test Geo-replication
- Take `--num-parallel` option and spawn the containers with ready
infra for running tests
- Split the tests based on the number of parallel jobs specified.
- Execute the tests in parallel in each container and watch for the
status.
- Archive only failed tests(Optionally enable logs for successful tests
using `--preserve-success-logs`)

## Pending:
- NFS related tests are not running since the required changes are
pending while creating the container image. (To know the failures run
gluster-tester with `--include-nfs-tests` option)
- Filter support while running the tests(To enable/disable tests on the
run time)
- Some Loop based tests are failing(I think due to shared `/dev/loop*`)
- A few tests are timing out(Due to this overall test duration is more)
- Once tests are started, showing real-time status is pending(Now
status is checked in `<logdir>/regression-<job>.log` for example
`/var/log/gluster-tester/regression-3.log`
- If the base image is not built before running tests, it gives an
error. Need to re-trigger the base container image step if not built.
(Issue: https://github.com/aravindavk/gluster-tester/issues/11)
- Creating an archive of core files
- Creating a single archive from all jobs/containers
- Testing `--ignore-from` feature to ignore the tests
- Improvements to the status output
- Cleanup(Stop test containers, and delete)

I opened an issue to collect the details of failed tests. I will
continue to update that issue as and when I capture failed tests in my
setup.
https://github.com/aravindavk/gluster-tester/issues/9

Feel free to suggest any feature improvements. Contributions are
welcome.
https://github.com/aravindavk/gluster-tester/issues

--
Regards
Aravinda
http://aravindavk.in


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Project Update: Containers-based distributed tests runner

Amar Tumballi Suryanarayan
This is a nice way to validate the patch for us.

Question I have is did we measure the time benefit of running them in parallel with containers?

Would be great to see the result in getting this tested in a cloud env, with 5 parallel threads and 10 parallel threads. 

-Amar


On Fri, Jun 14, 2019 at 7:44 PM Aravinda <[hidden email]> wrote:
**gluster-tester** is a framework to run existing "*.t" test files in
parallel using containers.

Install and usage instructions are available in the following
repository.

https://github.com/aravindavk/gluster-tester

## Completed:
- Create a base container image with all the dependencies installed.
- Create a tester container image with requested refspec(or latest
master) compiled and installed.
- SSH setup in containers required to test Geo-replication
- Take `--num-parallel` option and spawn the containers with ready
infra for running tests
- Split the tests based on the number of parallel jobs specified.
- Execute the tests in parallel in each container and watch for the
status.
- Archive only failed tests(Optionally enable logs for successful tests
using `--preserve-success-logs`)

## Pending:
- NFS related tests are not running since the required changes are
pending while creating the container image. (To know the failures run
gluster-tester with `--include-nfs-tests` option)
- Filter support while running the tests(To enable/disable tests on the
run time)
- Some Loop based tests are failing(I think due to shared `/dev/loop*`)
- A few tests are timing out(Due to this overall test duration is more)
- Once tests are started, showing real-time status is pending(Now
status is checked in `<logdir>/regression-<job>.log` for example
`/var/log/gluster-tester/regression-3.log`
- If the base image is not built before running tests, it gives an
error. Need to re-trigger the base container image step if not built.
(Issue: https://github.com/aravindavk/gluster-tester/issues/11)
- Creating an archive of core files
- Creating a single archive from all jobs/containers
- Testing `--ignore-from` feature to ignore the tests
- Improvements to the status output
- Cleanup(Stop test containers, and delete)

I opened an issue to collect the details of failed tests. I will
continue to update that issue as and when I capture failed tests in my
setup.
https://github.com/aravindavk/gluster-tester/issues/9

Feel free to suggest any feature improvements. Contributions are
welcome.
https://github.com/aravindavk/gluster-tester/issues

--
Regards
Aravinda
http://aravindavk.in


_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel




--
Amar Tumballi (amarts)

_______________________________________________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel