diff --git a/docs/assets/mail-service.png b/docs/assets/mail-service.png deleted file mode 100644 index 8729bfa95ca0ba053de7fc93efe1c81dae7db2db..0000000000000000000000000000000000000000 Binary files a/docs/assets/mail-service.png and /dev/null differ diff --git a/docs/components/core/legacy-auth.md b/docs/components/core/legacy-auth.md index b641de002aa9a463d873b2aa541cac0db0536897..63139e140029deeab06a05aa19e759e684fa324a 100644 --- a/docs/components/core/legacy-auth.md +++ b/docs/components/core/legacy-auth.md @@ -1 +1,3 @@ # Legacy AUTH + +aliases & ES diff --git a/docs/components/specific-apps/admin-gui.md b/docs/components/custom-apps/admin-gui.md similarity index 95% rename from docs/components/specific-apps/admin-gui.md rename to docs/components/custom-apps/admin-gui.md index 7cd08929a95107e96497befe6460f93c5b295a69..4422a95b1a6c2e2de9544ebeb1fc69c791e0eb30 100644 --- a/docs/components/specific-apps/admin-gui.md +++ b/docs/components/custom-apps/admin-gui.md @@ -81,3 +81,8 @@ docker-compose --project-name admin-gui up ## Authentication It uses the same authentication methods as the portail data web app, so if you have an account you will be able to log in. However only a user belongging to the admin group will be able to create, modify and delete entities. The groups are managed by the api gateway, for more information refer to the [Authentication and Authorization](../../miscellaneous/authentication&authorization.md) section. + + +## TO DEVELOP + +The management of images w/ respect to the Organizations and Credits services on the one hand, the Media Library on the other. diff --git a/docs/components/specific-apps/web-app.md b/docs/components/custom-apps/web-app.md similarity index 97% rename from docs/components/specific-apps/web-app.md rename to docs/components/custom-apps/web-app.md index 66c760037c05b7b80d19ec231391ee98295037fd..a226ecb94445196e2ddb12e40d2265f4eeb301ac 100644 --- a/docs/components/specific-apps/web-app.md +++ b/docs/components/custom-apps/web-app.md @@ -53,3 +53,12 @@ So we used...Elasticsearch. It is super fast and even have geographical research ## Credits ## Licence + + +### TODO + +bannière accés restreint +drafts +MVT+WMS +listing of data accesses ? +button for license ? diff --git a/docs/components/middlewares/legacy-auth-middleware.md b/docs/components/middlewares/legacy-auth.md similarity index 100% rename from docs/components/middlewares/legacy-auth-middleware.md rename to docs/components/middlewares/legacy-auth.md diff --git a/docs/components/miscellaneous/maintenance-page.md b/docs/components/miscellaneous/maintenance-page.md new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/docs/components/off-the-shelf-apps/elasticsearch.md b/docs/components/off-the-shelf-apps/elasticsearch.md index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..7af3f92af6d975b0efb100c76693ef71355e082d 100644 --- a/docs/components/off-the-shelf-apps/elasticsearch.md +++ b/docs/components/off-the-shelf-apps/elasticsearch.md @@ -0,0 +1 @@ +Quid de la couche AUTHZ ? diff --git a/docs/components/proxies/web-mapping-services.md b/docs/components/proxies/web-mapping-services.md new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/docs/components/services/changelog.md b/docs/components/services/changelog.md index 1d58de4ed61c60a6cdd4efe153c0b87176b70652..d81dd7b1a8af0035f0fd128cdc0962ad8df73ea7 100644 --- a/docs/components/services/changelog.md +++ b/docs/components/services/changelog.md @@ -2,13 +2,13 @@ ## Features -This service allows end-users to keep track of the improvements that are applied to this project throughout its development. improvements are classified by three categories: +This service allows end-users to keep track of the improvements that are applied to this project throughout its development. Such improvements are classified by three categories: 1. bugfixes 2. minor improvements 3. major improvements -Information it typically entered by developers and eventually presented to end-users via a dedicated page of data.grandlyon.com Web Portal. +Information is typically entered by developers and presented to end-users via a dedicated page of data.grandlyon.com Web Portal. A record in the underlying database includes the following information: @@ -20,7 +20,7 @@ A record in the underlying database includes the following information: * `bugFixes`: a list of bugfixes; * `minorImprovements`: a list of minor improvements; * `majorImprovements`: a list of major improvements; -* `language`: the language it is typed in, either French of English. In fact, each changelog event is entered twice, once in French, once in English, in order for the log to be bilingual. +* `language`: the language in which information is typed in, either French of English. In fact, each changelog event is entered twice, once in French, once in English, in order for the resulting log to be bilingual. Bugfixes, minor and major improvements are simply modeled as strings of characters. @@ -30,7 +30,7 @@ This service has no dependency on other components. ## Endpoints -This service exposes one single endpoint, `/changelog`, supporting all the CRUD methods in a RESTful fashion. +This service exposes one single endpoint, `/changelog`, supporting all the CRUD methods in a RESTful fashion. An healthcheck endpoint is exposed as well, `/health`, returning a `200` HTTP code when all indicators are healthy, `503` in the opposite case. The service is considered to be healthy if the underlying database is up. ## Implementation The service is implemented using the [NestJS](https://nestjs.com/) framework. We refer the reader to the [NestJS-based micro-services](../../miscellaneous/nestjs-micro-services.md) page for further details concerning the latter framework and the features it provides. diff --git a/docs/components/services/credits.md b/docs/components/services/credits.md index 1fbced97c44cedb2736a2e4e47ef394db8bf13bb..7a3da9f4070448a347569a938a67de2869cf440a 100644 --- a/docs/components/services/credits.md +++ b/docs/components/services/credits.md @@ -2,7 +2,7 @@ ## Features -This service allows one to manage a catalog of the Open Source initiatives enabling and powering this development project. +This service allows the editorial team to manage a catalog of the Open Source initiatives enabling and powering this development project. Each record of the catalog includes the following information: @@ -22,10 +22,14 @@ This service has no dependency on other components. The service provides two endpoints, -1. `/credits` -2. `/links` +1. `/credits`; +2. `/links`, -allowing one to perform CRUD operations on credits and links in a RESTful manner. +allowing one to perform CRUD operations on credits and links in a RESTful manner. Moreover, an healtcheck endpoint is provided, + +3. `/health`, + +returning a `200` HTTP code if the API is healthy, `503` otherwise. For the service to be healthy, the underlying database must be up. ## Implementation diff --git a/docs/components/services/csv-catalog-downloader.md b/docs/components/services/csv-catalog-downloader.md index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..060155dc709b51fe8e7a1cbde558730504214388 100644 --- a/docs/components/services/csv-catalog-downloader.md +++ b/docs/components/services/csv-catalog-downloader.md @@ -0,0 +1,22 @@ +# CSV Catalog Downloader + +## Features + +This service allows end-users to download a CSV export of the metadata catalog. Such export is generated by submitting a query to a dedicated endpoint of one of the HTTP APIs exposed by GeoNetwork. As the latter query is typically resolved by GeoNetwork in a few minutes time, this service was initially conceived as a **cache layer**, preventing end-users from waiting for such a considerable amount of time. Beyond acting as a cache layer, this service also performs the following transformations: + +* the column separator used by GeoNetwork, `###`, is replaced by a more conventional comma; +* the exported CSV file is encoded either in UTF-8 or WINDOWS-1252 depending on the `User-Agent` header of the incoming requests. + +data.grandlyon.com's metadata catalog being refreshed at most once per day, overnight, this service refreshes its cache with the same frequency. + +## Dependencies + +This service only depends on GeoNetwork. + +## Endpoints + +The service exposes only one endpoint, `/`, returning a CSV file as attachment. + +## Implementation + +The service is implemented in the Python language, using [Flask](https://palletsprojects.com/p/flask/) as Web framework, [Gunicorn](https://gunicorn.org/) as WSGI HTTP Server and the [Advanced Python Scheduler](https://apscheduler.readthedocs.io/en/stable/) to schedule daily updates of the cache. diff --git a/docs/components/services/mail.md b/docs/components/services/mailer.md similarity index 57% rename from docs/components/services/mail.md rename to docs/components/services/mailer.md index f74485404ba624e4e2ac4a76cc578759b0ac67e0..b4ec516fe43ec8f8e574a7908c47e0a413a9e291 100644 --- a/docs/components/services/mail.md +++ b/docs/components/services/mailer.md @@ -1,9 +1,27 @@ # Mail service -## What it does +## Features This service allow to deliver mails to any email address from the address specified in the configuration. It also provides two particular endpoints, one for the user to give a feedback and another one for general contact purpose. Those endpoints will both send an email to the admin address specified in the configuration. + +## Dependencies + +## Endpoints + +`/contact`: formulaire de contact + +`/feedback`: + +`/send` + + +`/health` This service will return a `200` http status code when all indicators are healthy. Otherwise it will return a `503` http status code. +For this service we declared an health indicator that verify that the connection to the SMTP Server is available. + +## Implementation + + ## How it works  @@ -16,21 +34,10 @@ However the service does not send this JSON directly to the distant SMTP server. NestJS provides a [swagger module](https://docs.nestjs.com/recipes/swagger) that can be easily integrated. Using specific annotations alongside your endpoints declaration, this module will automatically generates a swagger documentation, reachable at `/api-doc`. -## Service health - -NestJS provides a [health module](https://github.com/nestjs/terminus) based on Terminus, that gives you the opportunity to declare predefined or custom health indicators. It exposes the health status of the service at `/health`. -This service will return a `200` http status code when all indicators are healthy. Otherwise it will return a `503` http status code. - -For this service we declared an health indicator that verify that the connection to the RabbitMQ is available. -## Stats -We are using a Node module called [swagger-stats](http://swaggerstats.io/). -It traces API calls, monitors API performance and usage statistics. It exposes the metrics in different formats, such as Prometheus format, so you may use Prometheus and Graphana for API monitoring and alerting. -Those metrics are available at `/swagger-stats/metrics`. +## Templates -For more information about this module, visit the [official swagger-stat page](http://swaggerstats.io/docs.html). +## AUTHZ -## Docker -It is possible to run this service using Docker containers, using the `docker-compose.yml` and `Dockerfile` files. -For more information, refer to the project [service-email][add a link] \ No newline at end of file +email-writer diff --git a/docs/components/services/resources.md b/docs/components/services/resources-helper.md similarity index 100% rename from docs/components/services/resources.md rename to docs/components/services/resources-helper.md diff --git a/docs/components/services/reuses.md b/docs/components/services/reuses.md new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/docs/components/services/social-media-share-helper.md b/docs/components/services/social-media-share-helper.md new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/docs/overview/index.md b/docs/overview/index.md index f672c19c14a9ae648bcf306f0d911439cd8803b0..855d87c0233efd92e1ea912902d7e11fce55bca2 100644 --- a/docs/overview/index.md +++ b/docs/overview/index.md @@ -40,7 +40,7 @@ The data.grandlyon.com's Web Portal includes the following main features: Concerned entity | ID - Description | Users --- | --- | --- -Datasets | F1 - Presenting metadata (title, description, terms, ...) <br> F2 - Showing tabular views of data (if applicable) <br> F3 - Showing map views of data (if applicable) <br> F4 - Showing linked resources (APIs, files) <br> F5 - Performing full-text searches against data and metadata <br> F6 - Performing faceted searches against metadata <br> F7 - Composing custom queries against the Core APIs | end-users +Datasets | F0 - Download the entire dataset catalog as a CSV file <br> F1 - Presenting metadata (title, description, terms, ...) <br> F2 - Showing tabular views of data (if applicable) <br> F3 - Showing map views of data (if applicable) <br> F4 - Showing linked resources (APIs, files) <br> F5 - Performing full-text searches against data and metadata <br> F6 - Performing faceted searches against metadata <br> F7 - Composing custom queries against the Core APIs | end-users Users | F8 - Creating/updating/deleting accounts <br> F9 - Performing logins and logouts <br> F10 - Gaining access to restricted-access resources <br> | end-users Editorial Content | F11 - Editing static pages (approach, terms of use, ...) <br> F12 - Displaying static pages (approach, terms of use, ...) <br> F13 - Editing articles <br> F14 - Displaying articles <br> F15 - Editing the catalog of partners (data producers, ...) <br> F16 - Displaying the catalog of partners (data producers, ...) <br> F17 - Editing the catalog of data reuses (applications, websites, articles, ...) <br> F18 - Displaying the catalog of data reuses (applications, websites, articles, ...) <br> F19 - Editing the catalog of the Open Source initiatives powering this project <br> F20 - Displaying the catalog of the Open Source initiatives powering this project | back-office <br> end-users <br> back-office <br> end-users <br> back-office <br> end-users <br> back-office <br> end-users <br> back-office <br> end-users Support | F21 - Allowing users to contact the user support | end-users @@ -79,7 +79,7 @@ The following table lists, in alphabetical order, the components of data.grandly ID | Name | Type | Website | Usage --- | --- | --- | --- | --- -C1 | Legacy AUTH | specific | | AUTHN/AUTHZ service +C1 | Legacy AUTH | custom | | AUTHN/AUTHZ service C2 | PostgreSQL + PostGIS | off-the-shelf | [Link](https://postgis.net/) | Data Source C3 | GeoNetwork | off-the-shelf | [Link](https://www.geonetwork-opensource.org/) | Metadata Catalog C4 | MapServer | off-the-shelf | [Link](https://mapserver.org/) | WMS and MVT Server @@ -112,7 +112,7 @@ M1 | Legacy AUTH | ... | ... | Facilitating the interaction between the Web Port ID | Name | Repo | Doc | Usage | Deps --- | --- | --- | --- | --- | --- -P1 | Map Proxy | ... | ... | Proxying requests that the Web Portal performs against the cartographic Web Services (WMS, MVT) hosted by data.grandlyon.com's Core macro-component| C4 +P1 | Map Proxy | ... | ... | Proxying requests that the Web Portal performs against the cartographic Web Services (WMS, MVT) hosted by data.grandlyon.com's Core macro-component| A1, C4 @@ -142,19 +142,31 @@ ID | Name | Repo | Doc | Usage | Deps S1 | Authentication | ... | [Link](../components/services/authentication.md) | Allowing users to create accounts and to perform logins and logouts | A3, S5 S2 | Changelog | [Link](https://gitlab.alpha.grandlyon.com/refonte-data/service-changelog) | [Link](../components/services/changelog.md) | Keeping track of the new features available to users throughout the developments| S3 | Credits | ... | [Link](../components/services/credits.md) | Catalog of the Open Source initiatives powering the data.grandlyon.com project | | -S4 | CSV Catalog Downloader | ... | ... | Allowing users to download cached CSV exports of the metadata catalog | C3 | -S5 | Mail | ... | [Link](../components/services/mail.md) | Sending e-mails to both end- and back-office users | A6 +S4 | CSV Catalog Downloader | ... | [Link](../components/services/csv-catalog-downloader.md) | Allowing users to download cached CSV exports of the metadata catalog | C3 | +S5 | Mailer | ... | [Link](../components/services/mail.md) | Sending e-mails to both end- and back-office users | A6 S6 | Media Library | ... | ... | Managing the images shown across the various pages of the Web Portal (organizations, posts, ...) | S7 | Organizations | ... | ... | A catalog of the partners of the data.grandlyon.com, typically Data Providers | S6 S8 | Resources Helper | ... | ... | Assisting users in the composition of custom queries against the Core APIs | S9 | Reuses | ... | ... | A catalog of the known reuses of the data shared through data.grandlyon.com | S6 +S10 | Social Media Share Helper -### Specific applications +### Custom applications ID | Name | Repo | Doc | Usage | Deps --- | --- | --- | --- | --- | --- D1 | Web App | ... | ... | Web Application representing the actual Web Portal as seen by its end-users | S[1,9] D2 | Admin GUI | ... | ... | Back-office oriented web application allowing users to administrate some of the editorial content as well as the configuration of some functionality | S[1,3], S[6,9] +### Miscellaneous + +ID | Name | Repo | Doc | Usage | Deps +--- | --- | --- | --- | --- | --- +M1 | Maintenance Page | ... | ... | <!-- <span id="test">mon test</span> <a href='#test'>link</a> --> + + +# TODO + +* Elasticsearch -> off-the-shelf app + custom cache & AUTHZ layer +* Scripts related to the Kong setup