diff --git a/docs/components/custom-apps/admin-gui.md b/docs/components/custom-apps/admin-gui.md index f16d54deba265a76195425f9c560f71c81e6ce03..a41306a7940d7b224e61c28c0956670c24d547ca 100644 --- a/docs/components/custom-apps/admin-gui.md +++ b/docs/components/custom-apps/admin-gui.md @@ -14,7 +14,7 @@ The interface has been developed with the Angular framework, so you need to have Once you have Node.js installed you need to install the Angular CLI. Open a command line and run: -``` +```bash npm install -g @angular/cli ``` @@ -24,17 +24,17 @@ Open the `/src/assets/config/config.json` file and update the configuration in o For more explainations on how the configuration system works refer to the dedicated [section](../../miscellaneous/multi-environments-docker-build-for-angular.md). - - ### Start the web application Using the Angular CLI: -``` + +```bash ng serve ``` or the npm script (which use the Angular CLI) -``` + +```bash npm run start ``` @@ -45,36 +45,41 @@ We defined in the package.json two scripts that can build the application. One t Those two scripts contain an option `--max_old_space_size=<nb of max RAM octets the node process can take>`. Make sure the number does not exceed your RAM capacity (you can simply remove the option, it will use the default value: 512mb on 32-bit systems and 1gb on 64-bit systems). By incrising the node process memory limit we decrease the build time. For development environment (not optimized) -``` + +```bash npm run build:dev ``` For production environment (optimized) -``` + +```bash npm run build:prod ``` ### Build and deploy with Docker The related files are: + - `docker-compose.yml` which indicates what Dockerfile should be used, what image is going to be build, volumes, exposed port... - `Dockerfile` which describe the steps to build the docker image. Some environment variables need to be set before building or running the image. Use the appropriate command to set those variables: `SET` on Windows, `export` on Linux. -``` +```bash export TAG=<version> export APP_PORT=<port the application should be running on> export CONFIG_FILE_PATH=<path to your config file> ``` Then to build the image run (you can change prod to dev if you don't want an optimized build): -``` + +```bash docker-compose build --build-arg conf=prod admin-gui ``` Once the image is built on your machine, run the following command to start a container: -``` + +```bash docker-compose --project-name admin-gui up ``` @@ -82,7 +87,6 @@ docker-compose --project-name admin-gui up It uses the same authentication methods as the portail data web app, so if you have an account you will be able to log in. However only a user belonging to the admin group will be able to create, modify and delete entities. The groups are managed by the api gateway, for more information refer to the [Authentication and Authorization](../../miscellaneous/authentication&authorization.md) section. - ## TO DEVELOP The management of images w/ respect to the Organizations and Credits services on the one hand, the Media Library on the other. diff --git a/docs/components/custom-apps/web-app/changelog.md b/docs/components/custom-apps/web-app/changelog.md index de91a65f7ba58221377d3ebd85af3c4c0c1fd197..2751e94512cab3a2013a5b1743ea610f54bf1fba 100644 --- a/docs/components/custom-apps/web-app/changelog.md +++ b/docs/components/custom-apps/web-app/changelog.md @@ -2,8 +2,6 @@  -In order to keep our users updated with the latest features developped and bugs that have been fixed, we implemented a changelog feature. +The main role of the changelog page consists in keeping users posted about the latest developments and bugfixes. Content is retrieved from a dedicated [back-end service](../../services/changelog.md) and displayed in descending order of publication date. Depending on the current language of the Web Application, either the French or the English version is shown. -The component retrieves from the dedicated [service](../../services/changelog.md) the changelog of each released version from the most recent to the oldest. - -Each changelog being available both in French and English, the component only displays the ones matching the current language of the application. +As a side role, the changelog page allows users to contact the back-office team, via the [dedicated form](./contact.md). diff --git a/docs/components/custom-apps/web-app/contact.md b/docs/components/custom-apps/web-app/contact.md index ad6cef8403544185a56f492b86f6975df6b227e7..13a454234197d16f7cb56900c76d6da4a3fcd21e 100644 --- a/docs/components/custom-apps/web-app/contact.md +++ b/docs/components/custom-apps/web-app/contact.md @@ -2,12 +2,12 @@  -The contact page allow a user to contact the support team through the [email](../../services/mailer.md) service. +The contact page allow a user to send mails to the support team (cf. [email service](../../services/mailer.md)). -When a user is not logged in, he/she will need to fill some personal information (firstname, lastname, email) in order for the support team to be able to reply. In the case he/she is logged in the information will already be completed. +User information (firstname, lastname, email) is automatically filled whenever the user is logged in. Otherwise, the task is left to the user. -The user can select a subject from a predefined list but can also enter un custom one. +The message subject can be either chosen from a dropdown list or typed in (custom subject). -The component has been developped to read a `subject` query parameter from the current url. If one exists the component sets the `subject` value of the form with the value of the query parameter. This is the way we implemented the `Contact us` feature on the info page of a dataset detail. Indeed, when one clicks the button, he/she is redirected to the contact page where the `subject` is the name of the dataset that was beeing consulted. +The component can read the `subject` query parameter from the current url. If such a query parameter is not trivial, then the component uses its value to sets accordingly the `subject` value of the form. The `Contact Us` feature available on the information tab of each dataset page relies on this fonctionnality. Indeed, when the `Contact Us` button is clicked, the user is redirected to the contact page with a prefilled `subject`, equal to the name of the visited dataset. -In order to submit the contact form, the user has to accept the general term of use and the processing of the information by checking the dedicated box at the bottom of the form. +In order to activate the submit button, the user has to accept some terms, by checking the dedicated box at the bottom of the form. diff --git a/docs/components/custom-apps/web-app/credits.md b/docs/components/custom-apps/web-app/credits.md index 7ecaf4d5695cda4cc1dce39768ed7a24994cc3d1..f1417aa10b918b119db0fdc314ffdc7f8586d48e 100644 --- a/docs/components/custom-apps/web-app/credits.md +++ b/docs/components/custom-apps/web-app/credits.md @@ -2,8 +2,6 @@  -This page lists the open source tools used by data.grandlyon.com. +This page showcases the Open Source initiatives powering the data.grandlyon.com Web platform. -The list is fetched from the dedicated [credits](../../services/credits.md) service. - -For each credit the page displays the logo, the name and a short description. Each card (credit) is clickable and will open a new tab at the address of the website of the credit. +Content is fetched from the dedicated [credits service](../../services/credits.md). The logo, name and short description of each record are displayed. Each "card" is clickable and the click will open a new navigation tab, pointing at the address of the official website of the concerned record. diff --git a/docs/components/custom-apps/web-app/dataset/data.md b/docs/components/custom-apps/web-app/dataset/data.md index 46234b1a18ab8de3ac78b153a8a14b880bd2a891..09f4ee52f627b751909218c33e893aff41a6659f 100644 --- a/docs/components/custom-apps/web-app/dataset/data.md +++ b/docs/components/custom-apps/web-app/dataset/data.md @@ -1,74 +1,72 @@ # Data -This is the main page to observe the data of a dataset. This page has two components: a data table and a map. The map will be displayed only for geographical datasets (it makes sense). +The Data tab allows users to explore the records associated with a given dataset. The tab includes two main components: + +1. a data table; +2. a map, displayed only for geographical datasets (it makes sense, doesn't it?).  -## Fullscreen and Toggle Map/data +The user can toggle the visibility of each of the two components by clicking on the dedicated buttons. The entire tab can also go fullscreen. + +## Data table + +Each entry of a given dataset is rendered as a line in a table. Data value having simple types (string, integer, float, ...) are fully displayed. Complex data values, such as JSON objects, are replaced by a placeholder signaling the presence of a complex property. Records are retrieved from the search engine in a paginated way (30 items per page), in order not to overload the browser. + +**QUESTION: how is scrolling achieved? Do we use the dedicated Elasticsearch feature?** -To increase the flexibility and the comfort of the user to display the data, it's possible to put in fullscreen the components. -Furtheremore there is the ability to hide and show each component as we wish, depending if we want more to focus on the data values itself or to have a geographical representation. -## The data table -Each data entry of the dataset is rendered as a line, where columns are the properties names. All the values are displayed, with an exception with what we call *complex properties*. It's arrays or objects that might be difficult to display properly in a table (such as opening and closing hours). In this case, these values will be rendered correctly in the right window if we click on the data entry. -All the data are not retrieved in first place. We get them 30 entries by 30 entries after scrolling, to create a more smooth experience for the user. +### Search within the data entries -### Research in the data +A dedicated text input allows the user to perform a full-text search within the entire dataset. The number of matching records is shown and visually illustrated in the guise of a progress bar. -The research input allows the user to make a full-text research on all the data for all the properties. At any time it is possible to have the number of data matching with the current research. +**QUESTION: how are search terms handled? Do we use a + concatenating the various terms?** -### Properties display toggle +### Column visibility -In case of datasets with a lot of properties, the user might want to display only few properties to compare or sort it's values. Thus it is possible to select or unselect the active properties in the data table. +The visibility of each column can be toggled by clicking on a dedicated button and (de)selecting the various checkboxes. -### Data table interactions +### Interactions with the data table -* Each column is enabled for sorting alphabetically. +* Each column can be sorted either alphabetically or numerically, according to the data type detected by the search engine. -* If the map component is displayed (geographical datasets), after a click in an entry in the data table, the map will be centered to the clicked element and a window on the right is opened with all the detailed data (including complex properties) +* Upon clicking on a given entry on the table, the entry gets fully displayed (including complex properties) on a dedicated viewer. If the map component is visible, then the map gets centered on the clicked entry. -* All the entries that have been clicked (both from the data table but also from the map) are highligthed with a different background color +* Already visited entries are highlighted, on the table, with a different background color. -## The map -### Mapbox GL JS +## Map -On the client side, the library [Mapbox GL JS](https://github.com/mapbox/mapbox-gl-js) is used. This is a very interesting tool improving quickly. -Just for information the creator of Leaflet, Vladimir Agafonkin, is now working for Mapbox. +### Under the hood -### How is the map data fetched and displayed +The map component is powered by the [Mapbox GL JS](https://github.com/mapbox/mapbox-gl-js) library. -The main difficulty to display correctly the data was the heterogeneity of the datasets, both in size and in types. For example we can have a dataset with millions of polygons to display, or a very small one but with personnalized icons instead of a geometric figure. -To manage this very broad range of display we came with the following strategy. ->Note: this is for unrestricted datasets. For restricted datasets there will be one specific strategy that we will explain later. +### Layers -At the begigning, we wanted to display just one MVT layer. It's vector, fast and integration is awesome with Mapbox. One difficulty, as we mentionned, is that one data can be represented by one circle, one polygon, or one icon, and different colors. It is because the producer of the data can apply a style that will make sense for them. And for this reason we needed to keep using the WMS service to represent with fidelity? the data. -So first, **to display and see the data, the WMS service is used**. Using cache make it quite fast and we respect the style of the data producers. +Three overlaid layers are used: -Then the question of the interaction. **To let the user interact with the map, we load the MVT layer**. The trick is to paint it transparent by default, and apply a color on hover or click events. The user has the feeling to interact directly with the features on the map (displayed by WMS), but in reality interacts with the MVT layer above it. +1. a base layer, either vector or raster depending of the choice of the user. The raster layer is based on the [orthoimages](https://data.grandlyon.com/en/jeux-de-donnees/orthophotographie-2018-metropole-lyon-format-tiff/donnees) provided by the Métropole de Lyon with the addition of street labels coming from the vector base layer. -#### Restricted datasets +2. A raster layer, displaying the entire (geographic) dataset by issuing requests against a Web Map Service (WMS). -For these datasets here is the rules: +3. A transparent vector layer, fed by a Mapbox Vector Tiles (MVT) service, allowing users to interact with the raster features displayed by the WMS. Individual features get highlighted on hover and click events. -* display all the data on the map -* display only a sample into the table -* on the map, only the sample data can be interacted with +#### Restricted-access datasets -So we can not anymore use the MVT service for the interaction layer, because it uses all the data (and we don't want that). In this case we use instead the sample data retrieved by Elasticsearch (the ones displayed in the table), to construct a geojson and use it to create the layer. -At the end we can see all the data displayed on the map, but the interaction and the one visible inside the table are only a sample. +Some of the datasets hosted by data.grandlyon.com are only accessible to authorized users. For such datasets, two different behaviors were implemented: -### Vector & satellite base layers +1. for **authorized users**: the very same behavior as for unrestriced datasets, in a seamless way; -There are two base layers available: vector and satellite. +2. for **unauthorized users**: the raster layer, powered by the WMS, displays the full dataset; user interaction is limited to a small subset of geographical features and achieved by constructing a GeoJSON out of the sample returned by the search engine. -* the vector tiles are retrieved by the Métropolole of Lyon using the Open Map Tiles specification. It only covers the area of Métropole de Lyon but the details are more important than the Etalab data -* the satellite layer is using the pictures of the *Orthophotographie 2018 de la Métropole de Lyon* dataset. +This feature relies on a custom user-aware component, proxying requests to the Web Mapping Services hosted by the data.grandlyon.com Core. #### 3D display -This is an emulated 3D, using 2D information. The vector base layer contains the height of the buidings. With Mapbox GL JS it is possible to create a layer by setting some style to the polygons with the *fill-extrusion* property. +This is an emulated 3D, using 2D information. The vector base layer contains the height of the buidings (<- FALSE !). With Mapbox GL JS it is possible to create a layer by setting some style to the polygons with the *fill-extrusion* property. + +**QUESTION: which dataset is used?** ### Research by address & Geolocalisation @@ -76,7 +74,7 @@ It is possible to search an address in the map. We use an internal geocoder API It's also possible to center the map to the user position. It uses the native [Navigator API](https://developer.mozilla.org/en-US/docs/Web/API/Navigator/geolocation). -### *Copy the map* feature +### *Share* feature This allow with a button click to copy the link of the map with the current context. This context contains: the zoom, position, bearing, pitch and base layer. @@ -131,3 +129,6 @@ Finally you can reference it in your style file as the following: We have to do the conversion one by one. If multiple `.ttf` have to be converted, then maybe a `.sh` script is better. In our case we needed only `Titillium Web` to add in our map glyphs. If another font is needed and you want to use them in a single style file, then you need to combine them. For this you can use [glyph-pbf-composite](https://github.com/mapbox/glyph-pbf-composite) + + +**TO ADD: "bannière rouge"** diff --git a/docs/components/custom-apps/web-app/drafts.md b/docs/components/custom-apps/web-app/drafts.md index 39c24030628975e97dd3e1c7eac17e574878685e..47a1db68fffbad8e317017291a27669303420e74 100644 --- a/docs/components/custom-apps/web-app/drafts.md +++ b/docs/components/custom-apps/web-app/drafts.md @@ -1,5 +1,7 @@ # Drafts -This page is dedicated to the users belonging to the `ghost-editors` group. +This page presents a card per post in draft status. Such draft posts can be retrieved from the search engine by applying a filter (**which one**?). -The purpose of the page is to list non published articles. Each article is presented with its image cover, title and the last modification date and are clickable. Clicking on an article redirects to the article's page and so we are able to preview it as it will be displayed once published. Thereby we can anticipate any display bug before the publication of the article. +Each card includes the article cover image, title and the last modification date. The click on a card allows the user to navigate to a page where the concerned post is displayed as if it were actually published (cf. **TODO: link to the documentation of the article page**). + +This page is only accessible to the members of a dedicated user group. configured through the [API Gateway](../../off-the-shelf-apps/api-gateway.md) and its ACL plugin. We refer the reader to [this page](../../../miscellaneous/nestjs-micro-services.md) for further information about how user authorizations are implemented. diff --git a/docs/components/custom-apps/web-app/feedback.md b/docs/components/custom-apps/web-app/feedback.md index 53625a05516ea8a6e7e5d0ac6467628632cce88a..70c0ff5675ade7d9913e7a056654e6ea68da4067 100644 --- a/docs/components/custom-apps/web-app/feedback.md +++ b/docs/components/custom-apps/web-app/feedback.md @@ -2,6 +2,12 @@  -The goal of this feature is to provide users an easy way to give their opinion to the development team but also to submit any suggestions of improvment, problems or bugs they might have encountered. +The goal of this feature is to let end users provide feedback about their experience with the Web Application : bug reports, suggestions for improvement, etc. -The feedback form is composed of a textarea for the feedback, a 'rating' field where the value can be `1 star`, `2 stars`, `3 stars` or `No opinion` and an optional email field if the user wants a reply from the team. If an email is provided the user has to accept the general term of use and the process of the information in order to sumbit its feedback. +The feedback form is composed by : + +* a free text area; +* a set of buttons allowing the user to rate the website (`1 star`, `2 stars`, `3 stars`, `No opinion`); +* an optional email field, which the user can fill in if a reply from the support team is desired. + +In case the email address is provided, the user has to accept some terms for the submit button to be enabled. diff --git a/docs/components/custom-apps/web-app/homepage.md b/docs/components/custom-apps/web-app/homepage.md index 4eed13ffdbd2a18ad0ee3096f32fa1b37b982f9e..d3463a2b86d1d7e92df1ab3ee6238e55af9d2b3b 100644 --- a/docs/components/custom-apps/web-app/homepage.md +++ b/docs/components/custom-apps/web-app/homepage.md @@ -2,13 +2,11 @@  -This is the landing page of data.grandlyon.com. +This is the landing page of the data.grandlyon.com Web portal. The main goal of this page is to encourage the user to discover the dataset catalog. Two entrypoints are provided: -The main goal of the page is to encourage the user to discover the catalog of dataset. To do so, it gives two entrypoints: +* The search bar, allowing the user to find datasets that match the typed search terms. For more information about the search bar, please refer to [this](./header.md#Search-bar) section of the documentation. +* The explore button, which redirects the user to the search results page, with no filter activated: all the datasets are returned. -* The search bar, it allows the user to find dataset that match, with more or less accuracy, with the search string he/she typed. For more information about the search bar please refer to [this](./header.md#Search-bar) section of the documentation. -* The explore button, it also redirects the user to the search page but with no particular criteria and let him or her freely discover the available dataset. +The homepage also highlights the three latest news, retrieved from the search engine. In case any post is tagged as "featured" - a fonctionnality provided by the [Ghost CMS](../../off-the-shelf-apps/cms.md) - such post occupies the leftmost position. *QUESTION*: *what if multiple posts are featured* ? -The homepage also highlights the lastest news. Indeed the page retrieves from elasticsearch, where the content of our [CMS Ghost](../../off-the-shelf-apps/cms.md) as been indexed, the last three articles. However based on one of Ghost's functionality, that allow to *feature* an article, the first of the three news will always be the featured article when one exists. - -An `All news` button allows one to access the search page with the `News` tab directly selected. +Finally, the `All news` button allows one to land on the `News` tab of search results page. diff --git a/docs/components/off-the-shelf-apps/cms.md b/docs/components/off-the-shelf-apps/cms.md index 66ecf7c69ceca70f18a3d8037eac8171bca05b1d..2fd8b0085c69f01242dcd4cdf6acaf01bd252a20 100644 --- a/docs/components/off-the-shelf-apps/cms.md +++ b/docs/components/off-the-shelf-apps/cms.md @@ -9,11 +9,12 @@ For all the editorial content on the portal one CMS is used to offer the possibi The choice went for the open source CMS [Ghost](https://ghost.org/), replacing Wordpress as our first choice at the beginning of the project. Whereas it's not perfect and there is still room for improvement, this is a great tool to use to create content in a beautiful user interface. The API allows us to get all the information needed for the web application by a REST API. +## Hooks for indexing it inside Elasticsearch - ## Hooks for indexing it inside Elasticsearch The CMS offers a hooks functionnality. Put it simply, you can trigger any API call after some events. In our case, after saving a page or an article, the Elasticsearch (ES) indexer service will be called to re-index the article or the page which was just modified. In few seconds the modification will be visible inside the webapp without any manual manipulation. ## Integration into the webapp + After the content is indexed inside ES, very little work is needed to integrate it into the web application. The content actually retrieved as HTML, and the code is pretty clean, so very few CSS adjustment is needed to display a nice article. diff --git a/docs/components/proxies/web-mapping-services.md b/docs/components/proxies/web-mapping-services.md index 0e169a68b6ca037bf160349af27b08596a7652e7..7562108dcc7370ad3a50b7d0acd4c841dbda4cd8 100644 --- a/docs/components/proxies/web-mapping-services.md +++ b/docs/components/proxies/web-mapping-services.md @@ -1,9 +1,28 @@ -# Reuses +# Web mapping services ## Features +This service is the unique entrypoint of the map in the portal toward the geographical services of the datasets hosted on our platform. It proxifies two services: + +* `WMS` +* `MVT` + +Depending on the dataset's license, those services might be in open access or restricted access. Our goal is to display to a particular user exactly what it's access are allowing him to see. It wasn't possible to add the authentication pieces as the map services are expecting it (basic auth) directly from the web application this is why we created this proxy. + +In order for the map to work properly and to always be able to display the full data on the portal, it has been decided that WMS would always be opened even if the user doesn't have access to the dataset. + ## Dependencies +This proxy has two dependencies: + +* `Elasticsearch` +* `MapServer` + ## Endpoints +It provides two endpoints: + +* `/wms`: +* `/mvt`: + ## Implementation diff --git a/docs/miscellaneous/nestjs-micro-services.md b/docs/miscellaneous/nestjs-micro-services.md index 5506b7e27d9226b9ddfdc4cc59dd7f3ef5c73d64..7d04d8d04d252f7661be3e337858ed852b037451 100644 --- a/docs/miscellaneous/nestjs-micro-services.md +++ b/docs/miscellaneous/nestjs-micro-services.md @@ -16,4 +16,4 @@ A service will return a `200` http status code when all indicators are healthy. ## Logging -## Guards or how to implement an AUTHZ layer +## Guards or: how to implement an AUTHZ layer