Tests - py.test 2.0
About me: PrashantKumar
Mentor: AlexanderSchremmer
Repository: https://bitbucket.org/pkumar/pytest_moin
Project Diary: pkumar_diary
Project Plan and Work Flow: pytest2_projectplan
[Introduction]
Welcome to the Testing world of MoinMoin. This year in GSoC i.e. 'GSoC 2011', an idea was proposed about reimplementation of MoinMoin tests for pytest-2. Actually, it is not just reimplementation but a total refinement of tests including a better framework and testing mechanism. Following are the details of the project and the proposed plan of action.
Abstract: Re implement our tests for py.test 2.0. Improve test coverage. Do a sanity check on the tests. Rewrite tests.
[Benefits to the MoinMoin community]
1. Many new features and enhancements have been introduced to pytest-2.0. Implementation of MoinMoin tests for pytest-2.0 with less complicated tests and better testing framework would certainly result into an improved testing mechanism and error handling in MoinMoin.
2. Improving the test coverage will surely help in improvement of code quality of MoinMoin.
3. A detailed documentation on the work to help the contributers for further improvements on the work.
[Project Details]
This project is not just about porting MoinMoin tests over to pytest-2 but it has far greater goals. It is more about complete refinement of MoinMoin tests using pytest-2. The idea here is to provide a better testing framework for MoinMoin. It is also important that the new tests should be less complex and more efficient. Current version of MoinMoin i.e. moin 2.0 uses py.test 1.3.4. Implementing the test of MoinMoin for pytest-2 would require rewriting many tests due to numerous enhancements and new features introduced in pytest-2 which includes:
- New invocations through python interpreter.
- New configuration through ini-files.
- Removal of reading option values from conftest.py files or env variables.
- Access of module globals from xfail/skipif expressions.
One other feature of the framework should be that it should be sustainable and support further improvements. It means that if the tests need to be implemented for pytest 2.4 or need to be modified after a year or something, only slight change in the fixers should be enough and an automated process would be taking care of implementation of respective fixers to the tests. Since, it would be quite troublesome to implement all the tests again and again with each new modification or new release of MoinMoin dependencies, this feature is quite necessary from development perspective.
[Milestones]
The project is planned to be split across following Phases:
Before Midterm Evaluation [May 23 - July 15]
1. [May 23 - May 30] Porting current tests of MoinMoin over to pytest 2.0
2. [May 31 - June 7] Working on different test issues of MoinMoin which includes:
- leaking problem of the current tests
- File Leak issue with fs2.
3. [June 8 - June 15] Check for any new issue due to porting of tests over to pytest-2 and continue the work on fixing issues.
4. [June 16 - June 22] Work on deciding and finalizing new testing framework. This includes following steps:
- Going through different projects and their testing framework.
- Working on the possible occurence of different testing issues in different frameworks and the ease of handling those issues.
Discuss about different possible frameworks for MoinMoin tests with the mentor.
Accounting different test frameworks and selecting the most optimum one as per the need and structure of MoinMoin.
5. [June 23 - June 29] Start the implementation of new framework for MoinMoin tests. Implementation of tests for new framework includes:
- Decrease the complexity of test codes and its infrastructure.
- Rewrite few tests which fulfils the same purpose, but in a more effecient way.
- Making the test codes more automated and their behavior more pythonic.
6. [June 30 - July 7] Implement following modules for the new test framework:
- apps
- _tests
- config
- util
- converter
- auth
- datastruct
7. [July 8 - July 15] Finalizing the above modules which includes:
- Ensure if all the tests run properly for all the specified versions of python.
- It should not have any current existing issues like 'leaking of test' or the other.
- Check for any other possible issues and fix it.
[Deliverables]
1. Improved testing framework of MoinMoin implemented for pytest 2.0 for following modules:
- apps
- _tests
- config
- util
- converter
- auth
- datastruct
2. Tests with their major issues fixed. 3. A detailed tutorial till the current progress.
Before final Evaluation [July 16 - August 15]
8. [July 16 - July 23] Implementing remaining test modules for the new test framework i.e. :
- storage
- security
- items
- themes
9. [July 24 - July 31] Finalizing the above modules.
10. [August 1 - August 7] Working on improvement of test coverage which includes:
- Adding new tests for better testing mechanism.
- Adding new scripts to make the tests more automated and sustainable.
- A complete and detailed checking of all the tests and its corresponding results.
Ensure the proper working of MoinMoin and fix the testing issues(if any).
11. [August 8 - August 15] Last week of work would include the following:
- Final editing and completion of tutorials.
- Finalizing all the tests and wrap up.
[Deliverables]
1. Better and improved tests of MoinMoin implemented for py.test 2.0 having following features:
- Better testing framework.
- Less complex test infrastructure.
- More efficient testing mechanism.
2. A detailed tutorial on the project work to help other contributers.