Skip to content
Snippets Groups Projects
Commit ec495833 authored by Nathan Rodet's avatar Nathan Rodet
Browse files

Added documentation

parent 17bb65c3
No related branches found
No related tags found
No related merge requests found
# Labo Rust
# Scaleway Instances Template - Quickstart
Infrastructure as Code project to deploy lab infrastructure and configure instances. Working with Atrium, Gitlab CI, Terraform, Cloud-init, bash scripts and Scaleway.
## How does it work ?
### How does it work ?
The template will provision several resources (including your instances) to the Scaleway cloud provider using Terraform and Gitlab CI.
After provisioning, configuration of the instances and deployment of the services (Atrium and TTYD) will be done using Cloud-init and bash scripts.
The FORGE (Gitlab) manage the Infrastructure with Terraform at any state, plan, creation, modification and destruction with the CI. The FORGE store and will provide the configuration information
such as credentials to Terraform only during the running state to improve security. Then, informations are destroyed with the Gitlab Agent container.
> All your information must be stored in Gitlab and passed as variable to Terraform.
On run state, Terraform will create, modify and destroy infrastructure resources in Scaleway to match the configuration described in the configuration files.
### How the deployment is managed ?
**Resources deployed :**
The whole project is managed by Gitlab using the Gitlab CI.
Launching the pipeline will start the deployment of your infrastructure in 3 steps.
| resource-type | plan | inbound port |
|----------------------------------|--------|--------------|
| scaleway_instance_ip | | |
| scaleway_domain_record | | |
| scaleway_domain_record | | |
| scaleway_instance_security_group | | 22, 443 |
| scaleway_instance_server | DEV1-L | |
**Plan :**
Terraform will perform a "fake deployment" and initialize everything he needs before deploying.
After performing, you will be able to see the plan of the future creation/modification/destruction in the log of the pipeline.
If the modification could lead to break changes and errors, you must not launch the next step of the pipeline.
**Creation :**
Terraform will be applying the exact changes you reviewed in the Plan step.
**Destruction :**
Terraform will destroy the resources he managed to create (or manually imported one).
After the resources provisionned with Terraform, Cloud-init will configure the instances by running bash scripts, creating files... And 3 services will be running for each instance.
## How can I use it ?
**Running services :**
### Step 0 : Prerequisite
- Atrium (Reverse proxy, TLS encryption and HTTPS to the others hosted services)
- Code-server
- Webtop
**On The Forge (Gitlab)**, you must have or ask for the right to fork the repository.
After deployment, each resource can be accessed though HTTPS depending on their count number :
- https://desktop-0.rust-0.daag.alpha.grandlyon.com/
- https://code-0.rust-0.daag.alpha.grandlyon.com/
- https://terminal-0.rust-0.daag.alpha.grandlyon.com/
- https://rust-0.daag.alpha.grandlyon.com/
**Scaleway** needs you to have credentials to use their services. You must go to [Scaleway API key page](https://console.scaleway.com/iam/api-keys)
and create one for your project. Please make sure to add a description starting with `Terraform - <your description>`.
**The Certificate Authority (CA)** Let's Encrypt needs an email. He will use it to send a notification before the certificate expires to proceed with
the renewal.
- https://desktop-1.rust-1.daag.alpha.grandlyon.com/
- https://code-1.rust-1.daag.alpha.grandlyon.com/
- https://terminal-1.rust-1.daag.alpha.grandlyon.com/
- https://rust-1.daag.alpha.grandlyon.com/
### Credentials
**Desktop username : abc**
**Atrium username : admin**
**Password : <Gitlab Stored>**
## How to add new resources
You can add Terraform resources directly to the main.tf file.
Template are available to the link below.
**Terraform Registry for Scaleway provider : https://registry.terraform.io/providers/scaleway/scaleway/latest/docs**
## How to setup configuration file for services
### cloud-init : file creation
In this example, you will find how to create a file to a defined path with cloud-init.
Content inside `%` are meant to be replaced with sed command to be able to use environment variables or user-data.
**Example:**
```yaml
write_files:
- content: |
hostname: %atrium_hostname%.daag.alpha.grandlyon.com
debug_mode: false
letsencrypt_email: %atrium_letsencrypt_email%
tls_mode: Auto
apps: # optional : applications served by atrium
- id: 1
name: Code %atrium_count_index%
icon: web_asset
color: 4292030255
is_proxy: true
host: code-%atrium_count_index%
target: localhost:8080
- id: 2
name: Desktop %atrium_count_index%
icon: web_asset
color: 4292030255
is_proxy: true
host: desktop-%atrium_count_index%
target: localhost:8081
path: /root/atrium.yaml
```
**Bash commands example to replace `%` content :**
```bash
sed -i "s/%atrium_hostname%/$(scw-userdata atrium_hostname)/g" /root/atrium.yaml
sed -i "s/%atrium_letsencrypt_email%/$(scw-userdata atrium_letsencrypt_email)/g" /root/atrium.yaml
sed -i "s/%atrium_count_index%/$(scw-userdata atrium_count_index)/g" /root/atrium.yaml
```
## Downloading file
The example show how to curl a configuration file from repository to a defined path with cloud-init.
### Step 1 : Getting your own repository
Fork the project from [this repository](https://forge.grandlyon.com/systemes-dinformation/alpha/pocs/scaleway-instances-template)
### Step 2 : Setting up the variables with credentials and others information
Go to the `Settings` menu, `CI/CD` and look for the `Variables` submenu.
Once it's done, you will need to create the variable bellow :
- **ENVIRONMENT** : Environment of the deployment, contained in domains, subdomains and resources name convention.
- **TF_VAR_INSTANCES_COUNT** : Number of instances to create.
- **TF_VAR_LETSENCRYPT_EMAIL** : Lets encrypt mail for renewal.
- **TF_VAR_PROJECT_DOMAIN** : External domain from Scaleway. (Example : <xxx>.alpha.grandlyon.com)
- **TF_VAR_SCW_ACCESS_KEY** : Scaleway Access Key.
- **TF_VAR_SCW_PROJECT_ID** : Project ID from your Scaleway project.
- **TF_VAR_SCW_SECRET_KEY** : Scaleway Secret Key.
- **TF_VAR_USER_PASSWORD** : User password for accessing services and instances.
### Step 3 : Every is set up, deployment is coming !
Go the `CI/CD` menu and `Pipelines`.
From now, you will need to click on `Run pipeline` with your selected branch.
Since you added variables before, you don't need to put any in this step, click `Run Pipeline` again.
**Checking logs :** You can click on every step to check logs in case of errors or not.
**Check plan :** I recommend to always check the plan when you're trying things to do not perform destruction
on production infrastructure.
Once plan is done and ok, click the `Apply button` to deploy the infrastructure. Please be sure to click the `Destroy
button` to destroy the infrastructure when you don't need it anymore.
### Step 4 : Accessing your services
After deployment, each resource can be accessed though HTTPS, URL depends of the variable you defined before :
- **TTYD :** https://terminal-<INSTANCE_COUNT>.<ENVIRONMENT>-<INSTANCE_COUNT>.<PROJECT_DOMAIN/
- **Atrium :** https://<ENVIRONMENT>-<INSTANCE_COUNT>.<PROJECT_DOMAIN>/
**Example :**
```yaml
runcmd:
#
# cloud-init execute cmd before HOME and USER are set.
#
- "export HOME=/root"
- "export USER=root"
#
# Install Atrium
#
- "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/config-files/atrium.yaml -o root/atrium.yaml"
- "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/config-files/atrium.sh | bash"
```
## Use Gitlab variables in instances
From my instances, I should be able to access the variables as user-data using the command `scw-userdata <my-variable>` after declaring them in Terraform.
**Some documentation : https://blog.scaleway.com/introducing-scaleway-cloud-init-support/**
**Example : Declare in Terraform**
```HCL
user_data = {
atrium_count_index = count.index
atrium_hostname = scaleway_domain_record.subdomain_record[count.index].name
atrium_letsencrypt_email = var.LETSENCRYPT_EMAIL
user_password = var.USER_PASSWORD
cloud-init = file("../instance-scripts/cloud-init.yml") // this is not a variable, but the declaration of cloud-init file.
}
```
** Example : Retrieve in instance
```bash
scw-userdata atrium_count_index
```
## Setup Terraform Locally
First, you must setup 2 local files for your variables :
### variables-local.tf
Create a file **variables-local.tf** containing the following code :
```hcl
variable "FORGE_PROJECT_ID" {
type = string
description = "Forge Project ID"
default = "your project id"
sensitive = true
}
variable "FORGE_USERNAME" {
type = string
description = "Forge Username"
default = "your username"
sensitive = true
}
variable "FORGE_ACCESS_TOKEN" {
type = string
description = "Forge Access Token"
default = "your access token"
sensitive = true
}
```
### variables-local.tfvars
Now, you can create a file for your variables information called **variables-local.tfvars** containing the following code :
```hcl
### SCW variables
SCW_PROJECT_ID = ""
SCW_ACCESS_KEY = ""
SCW_SECRET_KEY = ""
INSTANCES_COUNT = ""
ENVIRONMENT = ""
```
### Terraform init - Gitlab remote tfstate
You can grab a init command from your gitlab project on menu Infrastructure > Terraform.
Select your environment and click the actions button, then you will only need to provide a gitlab project token.
Command should look like :
```bash
export GITLAB_ACCESS_TOKEN=<YOUR-ACCESS-TOKEN>
terraform init \
-backend-config="address=https://forge.grandlyon.com/api/v4/projects/<project-id>/terraform/state/<tfstate-name>" \
-backend-config="lock_address=https://forge.grandlyon.com/api/v4/projects/<project-id>/terraform/state/<tfstate-name>/lock" \
-backend-config="unlock_address=https://forge.grandlyon.com/api/v4/projects/<project-id>/terraform/state/<tfstate-name>/lock" \
-backend-config="username=xxxxxxx" \
-backend-config="password=$GITLAB_ACCESS_TOKEN" \
-backend-config="lock_method=POST" \
-backend-config="unlock_method=DELETE" \
-backend-config="retry_wait_min=5"
terraform init -var-file=variables-local.tfvars
```
### Terraform plan - With variables file
```bash
terraform plan -var-file=variables-local.tfvars -out=tfplan
```
### Terraform apply - With plan
```bash
terraform apply tfplan
```
### Terraform destroy - With variables file
```bash
terraform destroy -var-file=variables-local.tfvars
```
\ No newline at end of file
- https://terminal-1.rust-1.daag.alpha.grandlyon.com/
- https://rust-1.daag.alpha.grandlyon.com/
\ No newline at end of file
......@@ -27,7 +27,7 @@ variable "PROJECT_DOMAIN" {
variable "LETSENCRYPT_EMAIL" {
type = string
sensitive = true
description = "User password for accessing services"
description = "Lets encrypt mail for renewal"
}
variable "USER_PASSWORD" {
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment