Newer
Older
What is DACC ? A cozy blackbox where apps can send data in order to be anonymised and aggregate to provide usage stats.
## Infrastructure workflow
We have two environments, a dev one and a production. Both of them look like the following schema :
On the development env, the 'cozy env' refers to our 'Alpha' VM.
There is a service named `aggregatorUsageEvent`, running on a daily basis, sending pre-processed data to cozy. This service is responsible for parsing traced events store in database and send them to cozy.
The service is configured to run every day randomly before **07:00AM**.
This section explains how the application is tracking usage events. Here are the main step:
- All events are stored during the use of the application in the doctype **com.grandlyon.ecolyo.usageevents**
- A service is responsible of the aggregation of these events to create ANONYMIZED indicators. These indicators are then sent to a remote docType.
| Title | Tech name | target | context | startDate | result | trigger | Cozy Dacc Name |
| ------------------------------------- | --------------- | ------ | ---------- | --------- | ------ | -------------------------------------------------------------------------------- | ---------------------------- |
| Complétion d'un nouveau profile | ProfileSetEvent | --- | --- | --- | --- | Lors de l'affichage de la page de félicitation profileTypeFinished | navigation-action-daily |
| Consultation du nouveau bilan mensuel | ReportFromEvent | --- | 'analysis' | --- | 1 | Lorsque l'utilisateur consulte sa page analyse quand la notification est activée | summary-subscription-monthly |
:::warning
All indicators are only sent if a user validate first version of CGU, the one containing dacc validation
:::
All monthly indicators call can be found after this comment on the code
| Indicator | Trigger | value |
| ------------------------------ | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| consumption-variation-monthly | Tous les mois si l'utilisateur a été connecté au moins une fois dans le mois | La variation de consommation par type de fluide. Les paramètres différents sont le type de profil ainsi que le nombre de mois entre la première connexion de l'utilisateur et le moment où est remonté l'indicateur (seniority) |
| summary-subscription-monthly | Tous les mois si un utilisateur a souscrit a la newsletter ou c'est désabonné | 1 si l’utilisateur est inscrit sinon 0 |
| fluid-data-granularity-monthly | Tous les mois sans condition | 1 si l'utilisateur possède de la donnée a la demi heure sur le mois. Sinon 0 |
| profile-count-monthly | Tous les mois si le profil a été complété au moins une fois | Le nombre de fois ou le profile a été complété |
The daily process get all events of previous day and process them in the following indicators
| Indicator | Trigger | value |
| ---------------------------- | -------------- | ------------------------------------------------------------------------------------------------------- |
| konnector-connected-daily | Tous les jours | La valeur est 1. Les valeurs sont catégorisées par type de fluide (electricity / electricity:water ...) |
| summary-subscription-monthly | Tous les jours | 1 si l'utilisateur est revenu sur Ecolyo via le lien présent dans la newsletter |
All indicator are post to a remote doctype, which will allow the application to read a defined external API. This doctype is created by Cozy itself.
value: number | null
group1?: object
group2?: object
group3?: object
- *startDate*: used to defined the start date of the indicator
- *value*: contains a value relevant to the defined indicator (example: number of page viewed on a day).
- *groups*: contains all attributes defining a indicator. Please refer to [indicators definition](https://stats.cozycloud.cc/question#eyJkYXRhc2V0X3F1ZXJ5Ijp7ImRhdGFiYXNlIjo0LCJxdWVyeSI6eyJzb3VyY2UtdGFibGUiOjh9LCJ0eXBlIjoicXVlcnkifSwiZGlzcGxheSI6InRhYmxlIiwidmlzdWFsaXphdGlvbl9zZXR0aW5ncyI6e319)
| ENUM | value |
| ----------------------------- | ------------------------------ |
| CONSUMPTION_VARIATION_MONTHLY | consumption-variation-monthly |
| FLUID_DATA_GRANULARITY | fluid-data-granularity-monthly |
| KONNECTOR_CONNECTED_PER_DAY | konnector-connected-daily |
| PROFILE_COUNT_MONTHLY | profile-count-monthly |
| SUMMARY_SUBSCRIPTION_MONTHLY | summary-subscription-monthly |
There is two remote doctypes pointing to two different mode. `dacc-dev` is used on alpha.
```json
"dacc": {
"type": "cc.cozycloud.dacc",
"verbs": ["ALL"]
},
"dacc-dev": {
"type": "cc.cozycloud.dacc.dev",
"verbs": ["ALL"]
},
```
First you need to modify the service in order to be able to see data in local :
- Run a local server that will listen and log events.
- Edit dacc URL (line 67) to send requests to your local server at `http://localhost:8081`
```ts title="aggregatorUsageEvents.ts"
await client
.getStackClient()
.fetchJSON(
'POST',
environments.isProduction()
? '/remote/cc.cozycloud.daccv_2'
: 'http://localhost:8081',
{
data: JSON.stringify(indicator),
}
);
- For **monthly indicators**, remove date protection (ligne 1100)
```js
if (
profile
// &&
// DateTime.local()
// .setZone('utc', {
// keepLocalTime: true,
// })
// .startOf('day').day === profile.monthlyAnalysisDate.day
) {
```
- Generate usage events by navigating through the app
You will be able to see previous events and freshly generated events in the db `com-grandlyon-ecolyo-usageevent`
:warning: because the cron run every day before 7am, it will take events from the day before. So if you want to test specific usage, you will have to edit the `eventDate` of events you want to send.
- Then you are finally ready to test ! Just run the following :
yarn build-dev:browser && yarn run cozy-konnector-dev -m ./manifest.webapp ./build/services/aggregatorUsageEvents/ecolyo.js
*To save time, when running multiple times and if aggregatorUsageEvents didn't change, you don't have to build.*
yarn run cozy-konnector-dev -m ./manifest.webapp ./build/services/aggregatorUsageEvents/ecolyo.js
- You should see sent indicators in the logs of your local server
"data": {"createdBy":"ecolyo","measureName":"connection-count-monthly","startDate":"2021-12-09","value":3}
"data": {"createdBy":"ecolyo","measureName":"connection-count-monthly","startDate":"2021-12-09","value":3}
You have to reach cozy and discuss with them about new indicators. They will implement it for `dacc-dev` and `dacc` and you will be able to test it on alpha env. Remember you will need **at least 5 data** of each type to see it in metabase.
All the data aggregated in the dacc can be visualized with a *Metabase* website hosted by [cozy](https://stats.cozycloud.cc/).
## Errors
More here in [DJU](./profile.md#dju).
If there is an authentication error, the api will returns an error in html format to cozy, so you'll get from cozy an error 502 as following, because cozy expects a json response :
```json
{
"status": "502",
"title": "Bad Gateway",
"detail": "the content-type for the response is not authorized",
"source": {}
}
```