SEANetMap: Test processes
Table of Contents
1 Document status
This is Draft 2015-05-05-01 of John Tigue's documentation of test processes for the SEANetMap project.
The other documents in this set can be found in the index.
When this document reaches a sufficiently developed state where it is ready for multiple people to edit it, its content will be moved into the SEANetMap's GitHub hosted wiki, entitled Test processes, accessable at the following URL:
https://github.com/codeforseattle/seanetmap/wiki/Test-processes
Once this document's content resides in the above mentioned SEANetMap wiki pages, it can be expected to evolve and track the processes beyond version 1.0.x.
It is hoped that the document will be in the wiki by Thursday, 2015-05-07.
(The only reason this document is being developed in this fashion is because the author (John Tigue) can write much faster in his "native" editing environment. Editing via GitHib flavored markdown (GFM, the text markup that GitHub wiki pages are authored in) is slower, although exporting to GFM will be desirable once there are multiple people editing this document. Tigue's editing environment can export to GFM.)
Feedback is most welcome; email such to john@tigue.com.
1.1 Version history
The version are named with the pattern DATE-SERIAL, where DATE is of the from YYYY-MM-DD and SERIAL is a two digit code, reset to 0l each day. For example, 2015-05-05-01 is the first release of May 5th, 2015.
- 2015-05-05-01
- First public release
2 Introduction
This document describes the test processes for the SEANetMap project. These processes are relevant for both development and deployment context, which is why the topic was broken out to its own document.
3 Test system status
- Currently, there is not testing process at all. This means no isolated
tests. That means that following the above instructions will actually hit M-Lab's servers. Normally, that would give me a sense of vertigo: "a dev hitting production servers without even any test? Oh, my." In this situation though, this is not such a big deal as M-Lab will have yet another NDT test in their data store. And since SEANetMap does not have its own database yet, no junk is getting in "there." Obviously, this is not sustainable but it at least is not doing any real harm at this time. This needs to change before SEANetMap actually has a database that it is writing to.
Very quickly though the project needs to reach a level of process maturity wherein a developer or test context can get a completely local (no hitting remote servers while running) environment up and running, with minimal effort.
As it is there is a SQLite database installed and initialize. Altough it is not clear if much is actually going on in there.
4 Test tooling plan
Test as a topic gets its own section as it is applicable in both dev-local-box and CI/CD context.
Goals:
- No dev work persist noise to any production server.
- Fast isolated unit tests that developers will actually use continually during development.
- Test suites which can be used in CI/CD context (read: Travis and such)
First round:
- Mocha for test suites of Node and in-browsers.
- Karma to drive the Mocha tests into the browsers.
- In-browser server mocking via Sinon.JS
- In Node, mocing via Nock.
Second round:
- Database mocking
- For pure isolated unit test, need to mock data store in JavaScript.
- Not as critical as round one. Developers will hit the box-local SQLite database.
- Full suite of isolated unit test, based on Mocha. Deployed in Travis.
5 Dev box local testing
SEANetMap is a public open source project. As such we want to have a friendly, welcoming culture. In order to make that actual work in the context of producing quality software, there need to be sandboxes for developers and deployers to work safely in.
For developers that means have a completely isolated development environment. This is not the case yet, as referred to in the previous section. Isolated unit test are desirable (and a goal) but simply not hitting external servers on the web during development would be very desirable.
One on the main places where SEANetMap hits an external server is when NDT tests are run. Also, the tests take some number of seconds. It would be very good to have a mock of an M-Lab server. Isolated tests like this are also very fast (sub-second) and as such developers actaully run them, and development proceeds at a quicker pace.
Such tooling would also be used in a CI/CD context.