index.md

# Overview

data.grandlyon.com is a web platform owned by the Métropole de Lyon, aiming at facilitating and promoting the sharing of (open) data provided by public and private actors of the Lyon metropolitan area.

data.grandlyon.com consists of two macro-components, a Web Portal and a Core:

```plantuml
@startuml

package "data.grandlyon.com" {
  () "API" - [Core]
  [Web Portal] .> [API] : uses
}
@enduml
```

In this respect, the Web Portal is just one among the (hopefully) many Core's clients, in the same way as 3rd-party client applications:

```plantuml
@startuml

package "3rd-party Client Application" {
  [Some Component]
}

package "data.grandlyon.com" {
  () "API" - [Core]
  [Web Portal] .> [API] : uses
}

[Some Component] .> [API] : uses
@enduml
```

**This development project concerns the Web Portal component only.**

## Main features

The data.grandlyon.com's Web Portal includes the following main features:

Concerned entity | ID - Description | Users
--- | --- | ---
Datasets | F0 - Download the entire dataset catalog as a CSV file <br> F1 - Presenting metadata (title, description, terms, ...) <br> F2 - Showing tabular views of data (if applicable) <br> F3 - Showing map views of data (if applicable) <br> F4 - Showing linked resources (APIs, files) <br> F5 - Performing full-text searches against data and metadata <br> F6 - Performing faceted searches against metadata <br> F7 - Composing custom queries against the Core APIs | end-users |  
Users  | F8 - Creating/updating/deleting accounts <br> F9 - Performing logins and logouts <br> F10 - Gaining access to restricted-access resources <br> | end-users
Editorial Content | F11 - Editing static pages (approach, terms of use, ...) <br> F12 - Displaying static pages (approach, terms of use, ...) <br> F13 - Editing articles <br> F14 - Displaying articles <br> F15 - Editing the catalog of partners (data producers, ...) <br> F16 - Displaying the catalog of partners (data producers, ...) <br> F17 - Editing the catalog of data reuses (applications, websites, articles, ...) <br> F18 - Displaying the catalog of data reuses (applications, websites, articles, ...) <br> F19 - Editing the catalog of the Open Source initiatives powering this project <br> F20 - Displaying the catalog of the Open Source initiatives powering this project <br> F21 - Sharing content on social media | back-office <br> end-users <br> back-office <br> end-users <br> back-office <br> end-users <br> back-office <br> end-users <br> back-office <br> end-users <br> any
Support | F22 - Allowing users to contact the user support | end-users

<!-- * it showcases the datasets hosted by the Core component, presenting:
    * metadata (title, description, terms, ...)
    * tabular view (if applicable)
    * map view (if applicable)
    * linked resources (APIs, files)
* it allows users to
    * perform full-text searches within data and metadata
    * create accounts, establish authenticated sessions
    * gain access to restricted-access datasets
    * compose custom queries against the Core APIs
    * contact the user support, provide feedback
* it shows editorial contents:
    * static pages
    * posts
    * catalogs of
        * partners (data producers, ...)
        * reuses of data (applications, websites, articles, ...)
        * Open Source software the project relies on -->

The data.grandlyon.com's Web Portal also features an independent GUI allowing the back-office users to administer the editorial content.

## Architecture

[...an overview...]

![microservice-architecture](../assets/microservice-architecture.png)

## Components

### Core components

The following table lists, in alphabetical order, the components of data.grandlyon.com's Core macro-component that are used by the Web Portal:

ID | Name  | Usage | Type | Website
--- | --- | --- | --- | ---
C1 | Legacy AUTH | AUTHN/AUTHZ service | custom |
C2 | PostgreSQL + PostGIS | Data Source | off-the-shelf |  [Link](https://postgis.net/)
C3 | GeoNetwork | Metadata Catalog | off-the-shelf |  [Link](https://www.geonetwork-opensource.org/)
C4 | MapServer | WMS and MVT Server | off-the-shelf | [Link](https://mapserver.org/)

The latter components being provided by the Core macro-component of the data.grandlyon.com platform, their development is beyond the scope of this project.

<!-- Notes:
* The component C1 is deemed as "legacy" because... -->

### Off-the-shelf applications

ID | Name | Usage | Website
--- | --- | --- | ---
A1 | Elasticsearch | Search Engine| [Link](https://www.elastic.co/fr/products/elasticsearch)
A2 | Ghost | Headless CMS | [Link](https://ghost.org/)
A3 | Kong | API Gateway | [Link](https://konghq.com/kong/)
A4 | Matomo | Web Analytics | [Link](https://matomo.org/)
A5 | MinIO | Object Store | [Link](https://min.io/)
A6 | Postfix | SMTP Server| [Link](http://www.postfix.org/)



### Middlewares

ID | Name | Usage | Lang  | Repo | Doc | Deps
--- | --- | --- | --- | --- | --- | ---
M1 | Legacy AUTH | Facilitating the interaction between the Web Portal Authentication Service (S1) and the AUTHN/AUTHZ Service (C1) hosted by data.grandlyon.com's Core macro-component | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/middlewares/legacy-auth) | [Link](../components/middlewares/legacy-auth.md) |  C1, S5

### Proxies

ID | Name | Usage | Lang | Repo | Doc | Deps
--- | --- | --- | --- | --- | --- | ---
P1 | Web Mapping Services| Proxying requests that the Web Portal performs against the cartographic Web Services (WMS, MVT) hosted by data.grandlyon.com's Core macro-component | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/proxies/query) | [Link](../components/proxies/query.md) | A1, C4



### Indexers

The following custom indexers are used to index documents in Elasticsearch (A1) in order to make them full-text searchable.

ID | Name | Usage | Lang | Repo | Doc  | Deps
--- | --- | --- | --- | --- | --- | ---
I1 | Editorial Content | Indexing the posts and pages edited via Ghost | Python | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/indexers/editorial-content) | [Link](../components/indexers/editorial-content.md) |  A2
I2 | Metadata and Data | Indexing the metadata and data hosted, respectively, by data.grandlyon.com's catalog (C3) and database (C4) | Python | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/indexers/metadata-and-data) | [Link](../components/indexers/metadata-and-data.md) |  C3, C4

### Tools

ID | Name | Usage | Python | Repo | Doc | Deps
--- | --- | --- | --- | --- | --- | ---
T1 | MinIO master-slave replication tool | Replicating/mirroring the assets stored in one "master" instance of MinIO to some other "slave" instances | Python | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/tools/minio-master-slave-mirror) | [Link](../components/tools/minio-master-slave-mirror.md) | A5



### Services

In alphabetical order:

ID | Name | Usage | Lang | Repo | Doc | Deps
--- | --- | --- | --- | --- | --- | ---
S1 | Authentication | Allowing users to create accounts and to perform logins and logouts | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/authentication)  | [Link](../components/services/authentication.md) |  A3, S5
S2 | Changelog | Keeping track of the new features made available to users | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/changelog) | [Link](../components/services/changelog.md) |
S3 | Credits | Catalog of the Open Source initiatives powering the data.grandlyon.com project | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/credits) | [Link](../components/services/credits.md) |   |
S4 | CSV Catalog Downloader | Allowing users to download cached CSV exports of the metadata catalog | Python | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/csv-catalog-downloader) | [Link](../components/services/csv-catalog-downloader.md) |  C3 |
S5 | Mailer | Sending e-mails to both front- and back-office users | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/mailer) | [Link](../components/services/mailer.md) |  A6
S6 | Media Library | Managing the images shown across the various pages of the Web Portal (organizations, posts, ...) | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/media-library) | [Link](../components/services/media-library.md) | |
S7 | Organizations | A catalog of the partners of the data.grandlyon.com, typically Data Providers | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/organizations) | [Link](../components/services/organizations.md) | S6
S8 | Resources Helper | Assisting users in the composition of custom queries against the Core APIs | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/resources-helper) | [Link](../components/services/resources-helper.md) |  |
S9 | Reuses | A catalog of the known reuses of the data shared through data.grandlyon.com | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/reuses) | [Link](../components/services/reuses.md) | S6
S10 | Social Media Share Helper | Sharing editorial content on social media | TypeScript | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/services/social-media-share-helper) | [Link](../components/services/social-media-share-helper.md) | A1 |

### Custom applications
ID | Name | Usage | Lang | Repo | Doc  | Deps
--- | --- | --- | --- | --- | --- | ---
D1 | Web App | Web Application representing the actual Web Portal as seen by its end-users | TypeScript, HTML, CSS | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/custom-apps/web-app) | [Link](../components/custom-apps/web-app/overview.md) |  S[1,9]
D2 | Admin GUI | Back-office oriented web application allowing users to administrate some of the editorial content as well as the configuration of some functionality | TypeScript, HTML, CSS | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/custom-apps/web-app) | [Link](../components/custom-apps/admin-gui.md) | S[1,3], S[6,9]

### Miscellaneous

ID | Name | Usage | Lang | Repo | Doc | Deps
--- | --- | --- | --- | --- | --- | ---
M1 | Maintenance Page | A static HTML page to be used whenever the official site has to go offline because of some maintenance tasks | HTML, CSS | [Link](https://forge.grandlyon.com/web-et-numerique/web-et-numerique-internet/data.grandlyon.com/web-portal/components/miscellaneous/maintenance-page) | [Link](../components/miscellaneous/maintenance-page.md) |

<!-- <span id="test">mon test</span>
<a href='#test'>link</a> -->


# TODO

* Elasticsearch -> off-the-shelf app + custom cache & AUTHZ layer
* Scripts related to the Kong setup