Overview
data.grandlyon.com is a web platform owned by the Métropole de Lyon, aiming at facilitating and promoting the sharing of (open) data provided by public and private actors of the Lyon metropolitan area.
data.grandlyon.com consists of two macro-components, a Web Portal and a Core:
In this respect, the Web Portal is just one among the (hopefully) many Core's clients, in the same way as 3rd-party client applications:
This development project concerns the Web Portal component only.
Main features
The data.grandlyon.com's Web Portal includes the following main features:
| Concerned entity | ID - Description | Users |
|---|---|---|
| Datasets | F0 - Download the entire dataset catalog as a CSV file F1 - Presenting metadata (title, description, terms, ...) F2 - Showing tabular views of data (if applicable) F3 - Showing map views of data (if applicable) F4 - Showing linked resources (APIs, files) F5 - Performing full-text searches against data and metadata F6 - Performing faceted searches against metadata F7 - Composing custom queries against the Core APIs |
end-users |
| Users | F8 - Creating/updating/deleting accounts F9 - Performing logins and logouts F10 - Gaining access to restricted-access resources |
end-users |
| Editorial Content | F11 - Editing static pages (approach, terms of use, ...) F12 - Displaying static pages (approach, terms of use, ...) F13 - Editing articles F14 - Displaying articles F15 - Editing the catalog of partners (data producers, ...) F16 - Displaying the catalog of partners (data producers, ...) F17 - Editing the catalog of data reuses (applications, websites, articles, ...) F18 - Displaying the catalog of data reuses (applications, websites, articles, ...) F19 - Editing the catalog of the Open Source initiatives powering this project F20 - Displaying the catalog of the Open Source initiatives powering this project |
back-office end-users back-office end-users back-office end-users back-office end-users back-office end-users |
| Support | F21 - Allowing users to contact the user support | end-users |
The data.grandlyon.com's Web Portal also features an independent GUI allowing the back-office users to administer the editorial content.
Architecture
...an overview...
Components
Core components
The following table lists, in alphabetical order, the components of data.grandlyon.com's Core macro-component that are used by the Web Portal:
| ID | Name | Type | Website | Usage |
|---|---|---|---|---|
| C1 | Legacy AUTH | custom | AUTHN/AUTHZ service | |
| C2 | PostgreSQL + PostGIS | off-the-shelf | Link | Data Source |
| C3 | GeoNetwork | off-the-shelf | Link | Metadata Catalog |
| C4 | MapServer | off-the-shelf | Link | WMS and MVT Server |
The latter components being provided by the Core macro-component of the data.grandlyon.com platform, their development is beyond the scope of this project.
Off-the-shelf applications
| ID | Name | Website | Usage |
|---|---|---|---|
| A1 | Elasticsearch | Link | Search Engine |
| A2 | Ghost | Link | Headless CMS |
| A3 | Kong | Link | API Gateway |
| A4 | Matomo | Link | Web Analytics |
| A5 | MinIO | Link | Object Store |
| A6 | Postfix | Link | SMTP Server |
Middlewares
| ID | Name | Lang | Repo | Doc | Usage | Deps |
|---|---|---|---|---|---|---|
| M1 | Legacy AUTH | TS | Link | Link | Facilitating the interaction between the Web Portal Authentication Service (S1) and the AUTHN/AUTHZ Service (C1) hosted by data.grandlyon.com's Core macro-component | C1, S5 |
Proxies
| ID | Name | Repo | Doc | Usage | Deps |
|---|---|---|---|---|---|
| P1 | Web Mapping Services | Link | Link | Proxying requests that the Web Portal performs against the cartographic Web Services (WMS, MVT) hosted by data.grandlyon.com's Core macro-component | A1, C4 |
Indexers
The following custom indexers are used to index documents in Elasticsearch (A1) in order to make them full-text searchable.
| ID | Name | Repo | Doc | Usage | Deps |
|---|---|---|---|---|---|
| I1 | Editorial Content | Link | Link | Indexing the posts and pages edited via Ghost | A2 |
| I2 | Metadata and Data | Link | Link | Indexing the metadata and data hosted, respectively, by data.grandlyon.com's catalog (C3) and database (C4) | C3, C4 |
Tools
| ID | Name | Repo | Doc | Usage | Deps |
|---|---|---|---|---|---|
| T1 | MinIO master-slave replication tool | Link | Link | Replicating/mirroring the assets stored in one "master" instance of MinIO to some other "slave" instances | A5 |
Services
In alphabetical order:
| ID | Name | Repo | Doc | Usage | Deps |
|---|---|---|---|---|---|
| S1 | Authentication | Link | Link | Allowing users to create accounts and to perform logins and logouts | A3, S5 |
| S2 | Changelog | Link | Link | Keeping track of the new features available to users throughout the developments | |
| S3 | Credits | Link | Link | Catalog of the Open Source initiatives powering the data.grandlyon.com project | |
| S4 | CSV Catalog Downloader | Link | Link | Allowing users to download cached CSV exports of the metadata catalog | C3 |
| S5 | Mailer | Link | Link | Sending e-mails to both end- and back-office users | A6 |
| S6 | Media Library | Link | Link | Managing the images shown across the various pages of the Web Portal (organizations, posts, ...) | |
| S7 | Organizations | Link | Link | A catalog of the partners of the data.grandlyon.com, typically Data Providers | S6 |
| S8 | Resources Helper | Link | Link | Assisting users in the composition of custom queries against the Core APIs | |
| S9 | Reuses | Link | Link | A catalog of the known reuses of the data shared through data.grandlyon.com | S6 |
| S10 | Social Media Share Helper | Link | Link | ... | ... |
Custom applications
| ID | Name | Repo | Doc | Usage | Deps |
|---|---|---|---|---|---|
| D1 | Web App | Link | Link | Web Application representing the actual Web Portal as seen by its end-users | S[1,9] |
| D2 | Admin GUI | Link | Link | Back-office oriented web application allowing users to administrate some of the editorial content as well as the configuration of some functionality | S[1,3], S[6,9] |
Miscellaneous
| ID | Name | Repo | Doc | Usage | Deps |
|---|---|---|---|---|---|
| M1 | Maintenance Page | Link | Link | A static HTML page to be used whenever the official site has to go offline because of some maintenance tasks |
# TODO
- Elasticsearch -> off-the-shelf app + custom cache & AUTHZ layer
- Scripts related to the Kong setup