SEANetMap: Developer guide
Table of Contents
1 Document status
This is Draft 2015-05-04-01 of John Tigue's developer guide for the SEANetMap project. This document is specific to the deployment process for version 1.0.x of SEANetMap.
The other documents in this set can be found in the index.
When this document reaches a sufficiently developed state where it is ready for multiple people to edit it, its content will be moved into the SEANetMap's GitHub hosted wiki, entitle Developer Guide, accessable at the following URL:
https://github.com/codeforseattle/seanetmap/wiki/Developer-guide
Once this document's content resides in the above mentioned SEANetMap wiki pages, it can be expected to evolve and track the processes beyond version 1.0.x.
It is hoped that the document will be in the wiki by Thursday, 2015-05-07.
(The only reason this document is being developed in this fashion is because the author (John Tigue) can write much faster in his "native" editing environment. Editing via GitHib flavored markdown (GFM, the text markup that GitHub wiki pages are authored in) is slower, although exporting to GFM will be desirable once there are multiple people editing this document. Tigue's editing environment can export to GFM.)
Feedback is most welcome; email such to john@tigue.com.
1.1 Version history
The version are named with the pattern DATE-SERIAL, where DATE is of the from YYYY-MM-DD and SERIAL is a two digit code, reset to 0l each day. For example, 2015-05-04-01 is the first release of May 4th, 2015.
- 2015-05-05-01
- First public release
2 Introduction
SEANetMap is a pretty vanilla Node app that can be setup painlessly via the regular `git clone` followed by `npm install`.
If you want to actually run real queries against M-Lab's data in Google's cloud, then some extra setup work is required (see 5). Nontheless, it is a goal of this developer on-ramping document to provide developers an initial process which does not require extra work beyond the above mentioned steps. This is accomplished by mocking M-Lab with test doubles. These test doubles also come into play while running isolated unit tests via Mocha et alia.
The above also implies that you can simply do a `git clone` followed by a `npm install` and start hacking away at will without fear of disturbing anything in produciton servers.
3 Status
Currenty parts of this document are ahead of reality. When that is the case that fact will be clearly denoted in the text.
The following is an overview of the actual state of the (intended) process as described in this document.
4 The main README
As is the style with such GitHub hosted open source projects, there is a minimalist, terse set of instruction for "Getting started developing" on the main README.md page. Those instructions are for a standard git-clone-and-init process for a Node.js project. They are as follows: From the command line
- `git clone` the repo: https://github.com/codeforseattle/seanetmap.git
- `cd` into the repo just copied
- `npm install` to install all of the necessary dependencies
- `./setup-db.js` to create a sqlite database in `db/`
- `node server` to start the Node.js server
- in a browser visit http://localhost:8000
5 M-Lab interfacing
[Neither bq2geojson nor telescope have yet been worked with for SEANetMap]
There are two pieces of technology from M-Lab that are interesting for the purposes of SEANetMap acting as a client to M-Lab: bq2geojson and Telescope. The former is Node code so it would seem to be a natural fit for SEANetMap, while the latter is Python code but is more mature code, lower level, and broader in scope.
It would seem that both might be used. But if you find yourself needing some M-Lab client query machinery that neither provides, please try to contribute to bq2geojson and not any nasty perverted Python :-).
5.1 bq2geojson
https://github.com/m-lab/bq2geojson
"A tool to (relatively) easily convert M-Lab BigQuery data into hexbinned GeoJSON files and files necessary to make functional maps.
"This is small Node.js application which aims to make it relatively easy for just about anyone to fetch M-Lab data from Google BigQuery and convert it into a format (GeoJSON) that is easily usable by mapping software, Leaflet in this particular case."
So, good that it is Node based, and it is very specific about what it does: make GeoJSON from M-Lab queries. And it is already tested with Leaflet, which the project uses.
The small catch is that: "You will also need to have the Google Cloud SDK installed."
5.2 Telescope
Specifically, that means using the Python library Telescope written by M-Lab. As per its README:
Telescope requires the user to have a Google account with at least one project in Google Developer Console.
https://github.com/m-lab-tools/telescope Telescope is a Python library by M-Lab for querying their Google BigQuery datastore.
- This could be a way SEANetMap gets data out of M-Lab and stored locally.
There are tools for Node to work with Python code, so the bridge could be made: