psy e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
..
phpunit e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
test_files e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggAnnotationTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggBatchTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCommentTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreAccessCollectionsTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreAccessSQLTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreAnnotationAPITest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreAttributeLoaderTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreConfigTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreDatabaseQueueTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreFilestoreTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreGetEntitiesBaseTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreGetEntitiesFromAnnotationsTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreGetEntitiesFromAttributesTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreGetEntitiesFromMetadataTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreGetEntitiesFromPrivateSettingsTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreGetEntitiesFromRelationshipTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreGetEntitiesTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreHelpersTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreMetadataAPITest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreMetadataCacheTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreMetastringsTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreOutputAutoPTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCorePluginsAPITest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreRegressionBugsTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreRiverAPITest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreSkeletonTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggCoreUnitTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggEntityPreloaderIntegrationTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggEntityTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggGroupTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggObjectTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggRelationshipTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggSiteTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggTravisInstallTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
ElggUserTest.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
README.md e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago
suite.php e48014c2e0 moved from https://github.com/epsylon/Elgg-Lorea-Hydra 6 years ago

README.md

Vision

We want to make manual testing unnecessary for core developers, plugin authors, and site administrators by promoting and enabling fast, automated testing at every level of the Elgg stack.

We look forward to a world where the core developers do not need to do any manual testing to verify the correctness of code contributed to Elgg. Similarly, we envision a world where site administrators can upgrade and install new plugins with confidence that everything works well together.

Running Tests

Elgg Core Test Suite

Currently our tests are split in two pieces:

  • PHPUnit tests are located in /tests/phpunit -- these are strictly unit tests for now. No integration or end to end tests.
  • SimpleTest tests are located in /tests/ -- These are supported but we strongly encourage writing new tests in PHPUnit and plan to migrate these to PHPUnit at our earliest convenience.

Since we have a phpunit.xml configuration at the root of Elgg, testing should be as easy as:

git clone http://github.com/Elgg/Elgg
cd Elgg
phpunit

Plugin tests

Ideally plugins are configured in such a way that they can be unit-tested much like Elgg core. Plugin developers are free to implement their own methods for unit testing, but we encourage everyone to make it as easy as Elgg core:

git clone http://github.com/developer/elgg-plugin plugin
cd plugin
phpunit

End-to-end tests

Since Elgg plugins have so much power to override, filter, and modify Elgg's and other plugins' behavior, it's important to be able to run end-to-end tests on a staging server with your final configuration before deploying to production.

TODO: Make it easy to run all Elgg integration and acceptance tests from the admin area given the current plugin configuration. (without worrying about database corruption!)

Motivation

Briefly, the wins we expect from testing are:

  • Increased confidence in the system.
  • More freedom to refactor.
  • Built-in, up-to-date documentation.

We love community contributions, but in order to maintain stability we cannot accept outside contributions without first verifying their correctness. By promoting automated testing, the core developers can avoid the hassle of manual verification before accepting patches. It also means that external developers don't have to spend time earning trust with the core team. If a patch comes in and has tests to verify it, then we can be confident it works without worrying about the reputation of the contributor.

Note that these benefits can also extend to the plugins repository. Site owners are encouraged to "test plugins thoroughly" before deploying them on a production site. As of March 2013, this translates to manually verifying all the features that the plugin promises to offer. But Elgg provides a huge number of features, and it's not reasonable to test for all of them on every browser you want to support on every device you want to support. But what if plugin developers could write tests for their plugins and site owners could just run the tests for all installed plugins to verify the functionality is maintained? Then they wouldn't be limited to just picking plugins from "trusted" developers or "stable" releases. They could see that, indeed, nothing broke when they upgraded that critical plugin from 1.3 to 2.5, and push the upgrade to production with confidence.

The reason this isn't happening today is because Elgg itself is not easily testable at this level yet. We want to change that.

Strategy

we have several guiding principles that we think will be helpful in bringing our vision into reality.

In short, we are advocating:

  • Continuous integration -- if Travis isn't happy, we're not happy
  • Dependency injection -- For creating highly testable, modular code
  • BDD -- Tests should verify features and provide documentation, not rehash the Class API

Continuous Integration

We run all of our tests on Travis CI so that we can get real time feedback on the correctness of incoming pull requests and development as it progresses. If Travis isn't happy, we don't commit to the repo. This empowers us to merge pull requests in at a rapid pace, so long as they pass the tests. It also allows us to reject pull requests without detailed investigation if they do not pass the tests. We can get past the "does it work or not" question and talk about the things that humans need to talk about: API design, usefulness to the project, whether it belongs in core or a plugin, etc. We want as many features as possible provided by Elgg core to be verified automatically by tests running on Travis.

Dependency Injection

In order to maximize testability, all dependencies need to be explicit. Global functions, Singletons, and service locators are death for testability because it's impossible to tell what dependencies are hiding under the covers, and it's even harder to mock out those dependencies. Mocking is critical because you want your unit tests to test only one class at a time. Test failures in a TestCase should not result due to brokenness in a dependency; test failures should only indicate brokenness in the class under test. This makes everything much easier to debug. As of March 2013, most of Elgg still assumes and uses global state, and that has made Elgg and Elgg plugins historically very difficult to test. Fortunately we are moving in the opposite direction now, and a lot of work in Elgg 1.9 has gone into refactoring core components to be more dependency injectable. We are already reaping the benefits from that effort.

Behavior-Driven Development

For us this means we name tests for features rather than methods. When you test for features, you are encouraged to write fewer, smaller, logical tests. When a test fails, we can know exactly what feature is compromised. Furthermore, when naming your tests for features, the list of tests provides documentation on what features the system supports. Documentation is something that is typically very troublesome to keep up to date, but when documentation and verification are one and the same, it becomes very easy to keep the documentation up to date.

Consider these two test methods:

  • testRegister()
  • testCanRegisterFilesAsActionHandlers()

From just looking at the names, testRegister tells you that the class under test probably has a method named register. If this test passes, it presumably verifies that it is behaving correctly, but doesn't tell you what correct behavior entails, or what the original author of the test was intending to verify. If that method has multiple correct uses that you need to test for, this terse naming convention also encourages you to write a very long test which tests for all conditions and features of said method. Test failure could be caused by any one of those uses being compromised, and it will take more time to figure out where the true problem lies.

On the other hand, testCanRegisterFilesAsActionHandlers tells you that there are these things called "actions" that need to be "handled" and that files can be registered as valid handlers for actions. This introduces newcomers to project terminology and communicates clearly the intent of the test to those already familiar with the terminology.

For a good example of what we're looking for, check out /tests/phpunit/Elgg/ViewServiceTest.php