Skip to content
Snippets Groups Projects
Commit 737baa82 authored by Alessandro Cerioni's avatar Alessandro Cerioni
Browse files

Converged tree, naming ...

parent c6929ae9
No related branches found
No related tags found
No related merge requests found
Pipeline #2600 passed
Showing
with 91 additions and 29 deletions
docs/assets/mail-service.png

29.9 KiB

# Legacy AUTH
aliases & ES
......@@ -81,3 +81,8 @@ docker-compose --project-name admin-gui up
## Authentication
It uses the same authentication methods as the portail data web app, so if you have an account you will be able to log in. However only a user belongging to the admin group will be able to create, modify and delete entities. The groups are managed by the api gateway, for more information refer to the [Authentication and Authorization](../../miscellaneous/authentication&authorization.md) section.
## TO DEVELOP
The management of images w/ respect to the Organizations and Credits services on the one hand, the Media Library on the other.
......@@ -53,3 +53,12 @@ So we used...Elasticsearch. It is super fast and even have geographical research
## Credits
## Licence
### TODO
bannière accés restreint
drafts
MVT+WMS
listing of data accesses ?
button for license ?
Quid de la couche AUTHZ ?
......@@ -2,13 +2,13 @@
## Features
This service allows end-users to keep track of the improvements that are applied to this project throughout its development. improvements are classified by three categories:
This service allows end-users to keep track of the improvements that are applied to this project throughout its development. Such improvements are classified by three categories:
1. bugfixes
2. minor improvements
3. major improvements
Information it typically entered by developers and eventually presented to end-users via a dedicated page of data.grandlyon.com Web Portal.
Information is typically entered by developers and presented to end-users via a dedicated page of data.grandlyon.com Web Portal.
A record in the underlying database includes the following information:
......@@ -20,7 +20,7 @@ A record in the underlying database includes the following information:
* `bugFixes`: a list of bugfixes;
* `minorImprovements`: a list of minor improvements;
* `majorImprovements`: a list of major improvements;
* `language`: the language it is typed in, either French of English. In fact, each changelog event is entered twice, once in French, once in English, in order for the log to be bilingual.
* `language`: the language in which information is typed in, either French of English. In fact, each changelog event is entered twice, once in French, once in English, in order for the resulting log to be bilingual.
Bugfixes, minor and major improvements are simply modeled as strings of characters.
......@@ -30,7 +30,7 @@ This service has no dependency on other components.
## Endpoints
This service exposes one single endpoint, `/changelog`, supporting all the CRUD methods in a RESTful fashion.
This service exposes one single endpoint, `/changelog`, supporting all the CRUD methods in a RESTful fashion. An healthcheck endpoint is exposed as well, `/health`, returning a `200` HTTP code when all indicators are healthy, `503` in the opposite case. The service is considered to be healthy if the underlying database is up.
## Implementation
The service is implemented using the [NestJS](https://nestjs.com/) framework. We refer the reader to the [NestJS-based micro-services](../../miscellaneous/nestjs-micro-services.md) page for further details concerning the latter framework and the features it provides.
......
......@@ -2,7 +2,7 @@
## Features
This service allows one to manage a catalog of the Open Source initiatives enabling and powering this development project.
This service allows the editorial team to manage a catalog of the Open Source initiatives enabling and powering this development project.
Each record of the catalog includes the following information:
......@@ -22,10 +22,14 @@ This service has no dependency on other components.
The service provides two endpoints,
1. `/credits`
2. `/links`
1. `/credits`;
2. `/links`,
allowing one to perform CRUD operations on credits and links in a RESTful manner.
allowing one to perform CRUD operations on credits and links in a RESTful manner. Moreover, an healtcheck endpoint is provided,
3. `/health`,
returning a `200` HTTP code if the API is healthy, `503` otherwise. For the service to be healthy, the underlying database must be up.
## Implementation
......
# CSV Catalog Downloader
## Features
This service allows end-users to download a CSV export of the metadata catalog. Such export is generated by submitting a query to a dedicated endpoint of one of the HTTP APIs exposed by GeoNetwork. As the latter query is typically resolved by GeoNetwork in a few minutes time, this service was initially conceived as a **cache layer**, preventing end-users from waiting for such a considerable amount of time. Beyond acting as a cache layer, this service also performs the following transformations:
* the column separator used by GeoNetwork, `###`, is replaced by a more conventional comma;
* the exported CSV file is encoded either in UTF-8 or WINDOWS-1252 depending on the `User-Agent` header of the incoming requests.
data.grandlyon.com's metadata catalog being refreshed at most once per day, overnight, this service refreshes its cache with the same frequency.
## Dependencies
This service only depends on GeoNetwork.
## Endpoints
The service exposes only one endpoint, `/`, returning a CSV file as attachment.
## Implementation
The service is implemented in the Python language, using [Flask](https://palletsprojects.com/p/flask/) as Web framework, [Gunicorn](https://gunicorn.org/) as WSGI HTTP Server and the [Advanced Python Scheduler](https://apscheduler.readthedocs.io/en/stable/) to schedule daily updates of the cache.
# Mail service
## What it does
## Features
This service allow to deliver mails to any email address from the address specified in the configuration. It also provides two particular endpoints, one for the user to give a feedback and another one for general contact purpose. Those endpoints will both send an email to the admin address specified in the configuration.
## Dependencies
## Endpoints
`/contact`: formulaire de contact
`/feedback`:
`/send`
`/health` This service will return a `200` http status code when all indicators are healthy. Otherwise it will return a `503` http status code.
For this service we declared an health indicator that verify that the connection to the SMTP Server is available.
## Implementation
## How it works
![mail-service](../../assets/mail-service.png)
......@@ -16,21 +34,10 @@ However the service does not send this JSON directly to the distant SMTP server.
NestJS provides a [swagger module](https://docs.nestjs.com/recipes/swagger) that can be easily integrated. Using specific annotations alongside your endpoints declaration, this module will automatically generates a swagger documentation, reachable at `/api-doc`.
## Service health
NestJS provides a [health module](https://github.com/nestjs/terminus) based on Terminus, that gives you the opportunity to declare predefined or custom health indicators. It exposes the health status of the service at `/health`.
This service will return a `200` http status code when all indicators are healthy. Otherwise it will return a `503` http status code.
For this service we declared an health indicator that verify that the connection to the RabbitMQ is available.
## Stats
We are using a Node module called [swagger-stats](http://swaggerstats.io/).
It traces API calls, monitors API performance and usage statistics. It exposes the metrics in different formats, such as Prometheus format, so you may use Prometheus and Graphana for API monitoring and alerting.
Those metrics are available at `/swagger-stats/metrics`.
## Templates
For more information about this module, visit the [official swagger-stat page](http://swaggerstats.io/docs.html).
## AUTHZ
## Docker
It is possible to run this service using Docker containers, using the `docker-compose.yml` and `Dockerfile` files.
For more information, refer to the project [service-email][add a link]
\ No newline at end of file
email-writer
......@@ -40,7 +40,7 @@ The data.grandlyon.com's Web Portal includes the following main features:
Concerned entity | ID - Description | Users
--- | --- | ---
Datasets | F1 - Presenting metadata (title, description, terms, ...) <br> F2 - Showing tabular views of data (if applicable) <br> F3 - Showing map views of data (if applicable) <br> F4 - Showing linked resources (APIs, files) <br> F5 - Performing full-text searches against data and metadata <br> F6 - Performing faceted searches against metadata <br> F7 - Composing custom queries against the Core APIs | end-users
Datasets | F0 - Download the entire dataset catalog as a CSV file <br> F1 - Presenting metadata (title, description, terms, ...) <br> F2 - Showing tabular views of data (if applicable) <br> F3 - Showing map views of data (if applicable) <br> F4 - Showing linked resources (APIs, files) <br> F5 - Performing full-text searches against data and metadata <br> F6 - Performing faceted searches against metadata <br> F7 - Composing custom queries against the Core APIs | end-users
Users | F8 - Creating/updating/deleting accounts <br> F9 - Performing logins and logouts <br> F10 - Gaining access to restricted-access resources <br> | end-users
Editorial Content | F11 - Editing static pages (approach, terms of use, ...) <br> F12 - Displaying static pages (approach, terms of use, ...) <br> F13 - Editing articles <br> F14 - Displaying articles <br> F15 - Editing the catalog of partners (data producers, ...) <br> F16 - Displaying the catalog of partners (data producers, ...) <br> F17 - Editing the catalog of data reuses (applications, websites, articles, ...) <br> F18 - Displaying the catalog of data reuses (applications, websites, articles, ...) <br> F19 - Editing the catalog of the Open Source initiatives powering this project <br> F20 - Displaying the catalog of the Open Source initiatives powering this project | back-office <br> end-users <br> back-office <br> end-users <br> back-office <br> end-users <br> back-office <br> end-users <br> back-office <br> end-users
Support | F21 - Allowing users to contact the user support | end-users
......@@ -79,7 +79,7 @@ The following table lists, in alphabetical order, the components of data.grandly
ID | Name | Type | Website | Usage
--- | --- | --- | --- | ---
C1 | Legacy AUTH | specific | | AUTHN/AUTHZ service
C1 | Legacy AUTH | custom | | AUTHN/AUTHZ service
C2 | PostgreSQL + PostGIS | off-the-shelf | [Link](https://postgis.net/) | Data Source
C3 | GeoNetwork | off-the-shelf | [Link](https://www.geonetwork-opensource.org/) | Metadata Catalog
C4 | MapServer | off-the-shelf | [Link](https://mapserver.org/) | WMS and MVT Server
......@@ -112,7 +112,7 @@ M1 | Legacy AUTH | ... | ... | Facilitating the interaction between the Web Port
ID | Name | Repo | Doc | Usage | Deps
--- | --- | --- | --- | --- | ---
P1 | Map Proxy | ... | ... | Proxying requests that the Web Portal performs against the cartographic Web Services (WMS, MVT) hosted by data.grandlyon.com's Core macro-component| C4
P1 | Map Proxy | ... | ... | Proxying requests that the Web Portal performs against the cartographic Web Services (WMS, MVT) hosted by data.grandlyon.com's Core macro-component| A1, C4
......@@ -142,19 +142,31 @@ ID | Name | Repo | Doc | Usage | Deps
S1 | Authentication | ... | [Link](../components/services/authentication.md) | Allowing users to create accounts and to perform logins and logouts | A3, S5
S2 | Changelog | [Link](https://gitlab.alpha.grandlyon.com/refonte-data/service-changelog) | [Link](../components/services/changelog.md) | Keeping track of the new features available to users throughout the developments|
S3 | Credits | ... | [Link](../components/services/credits.md) | Catalog of the Open Source initiatives powering the data.grandlyon.com project | |
S4 | CSV Catalog Downloader | ... | ... | Allowing users to download cached CSV exports of the metadata catalog | C3 |
S5 | Mail | ... | [Link](../components/services/mail.md) | Sending e-mails to both end- and back-office users | A6
S4 | CSV Catalog Downloader | ... | [Link](../components/services/csv-catalog-downloader.md) | Allowing users to download cached CSV exports of the metadata catalog | C3 |
S5 | Mailer | ... | [Link](../components/services/mail.md) | Sending e-mails to both end- and back-office users | A6
S6 | Media Library | ... | ... | Managing the images shown across the various pages of the Web Portal (organizations, posts, ...) |
S7 | Organizations | ... | ... | A catalog of the partners of the data.grandlyon.com, typically Data Providers | S6
S8 | Resources Helper | ... | ... | Assisting users in the composition of custom queries against the Core APIs |
S9 | Reuses | ... | ... | A catalog of the known reuses of the data shared through data.grandlyon.com | S6
S10 | Social Media Share Helper
### Specific applications
### Custom applications
ID | Name | Repo | Doc | Usage | Deps
--- | --- | --- | --- | --- | ---
D1 | Web App | ... | ... | Web Application representing the actual Web Portal as seen by its end-users | S[1,9]
D2 | Admin GUI | ... | ... | Back-office oriented web application allowing users to administrate some of the editorial content as well as the configuration of some functionality | S[1,3], S[6,9]
### Miscellaneous
ID | Name | Repo | Doc | Usage | Deps
--- | --- | --- | --- | --- | ---
M1 | Maintenance Page | ... | ... |
<!-- <span id="test">mon test</span>
<a href='#test'>link</a> -->
# TODO
* Elasticsearch -> off-the-shelf app + custom cache & AUTHZ layer
* Scripts related to the Kong setup
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment