README.md

Alpha Project Manager

An Infrastructure as Code manager to deploy lab infrastructure and configure instances. Working with Atrium, Gitlab CI, Terraform, Cloud-init and Scaleway.

How does it work ?

The FORGE (Gitlab) manage the Infrastructure with Terraform at any state, plan, creation, modification and destruction with the CI. The FORGE store and will provide the configuration information such as credentials to Terraform only during the running state to improve security. Then, informations are destroyed with the Gitlab Agent container.

On run state, Terraform will create, modify and destroy infrastructure resources in Scaleway to match the configuration described in the configuration files.

Resources deployed :

resource-type	plan	inbound port
scaleway_instance_ip
scaleway_domain_record
scaleway_domain_record
scaleway_instance_security_group		22, 443
scaleway_instance_server	DEV1-L

After the resources provisionned with Terraform, Cloud-init will configure the instances by running bash scripts, creating files... And 3 services will be running for each instance.

Running services :

• Atrium (Reverse proxy, TLS encryption and HTTPS to the others hosted services)
• Code-server
• Webtop

After deployment, each resource can be accessed though HTTPS depending on their count number :

• https://desktop-0.rust-0.daag.alpha.grandlyon.com/

• https://code-0.rust-0.daag.alpha.grandlyon.com/

• https://rust-0.daag.alpha.grandlyon.com/

• https://desktop-1.rust-1.daag.alpha.grandlyon.com/

• https://code-1.rust-1.daag.alpha.grandlyon.com/

• https://rust-1.daag.alpha.grandlyon.com/

User :

• Username: abc
• Password:

How to add new resources

You can add Terraform resources directly to the main.tf file. Template are available to the link below.

Terraform Registry for Scaleway provider : https://registry.terraform.io/providers/scaleway/scaleway/latest/docs

How to setup configuration file for services

cloud-init : file creation

In this example, you will find how to create a file to a defined path with cloud-init. Content inside % are meant to be replaced with sed command to be able to use environment variables or user-data.

example:

write_files:
  - content: |
      hostname: %atrium_hostname%.daag.alpha.grandlyon.com
      debug_mode: false
      letsencrypt_email: %atrium_letsencrypt_email%
      tls_mode: Auto
      apps: # optional : applications served by atrium
        - id: 1
          name: Code %atrium_count_index%
          icon: web_asset
          color: 4292030255
          is_proxy: true
          host: code-%atrium_count_index%
          target: localhost:8080
        - id: 2
          name: Desktop %atrium_count_index%
          icon: web_asset
          color: 4292030255
          is_proxy: true
          host: desktop-%atrium_count_index%
          target: localhost:8081
    path: /root/atrium.yaml

Bash commands example to replace % content :

sed -i "s/%atrium_hostname%/$(scw-userdata atrium_hostname)/g" /root/atrium.yaml
sed -i "s/%atrium_letsencrypt_email%/$(scw-userdata atrium_letsencrypt_email)/g" /root/atrium.yaml
sed -i "s/%atrium_count_index%/$(scw-userdata atrium_count_index)/g" /root/atrium.yaml

Downloading file

The example show how to curl a configuration file from repository to a defined path with cloud-init.

example :

runcmd:
#
# cloud-init execute cmd before HOME and USER are set.
#
  - "export HOME=/root"
  - "export USER=root"
#
# Install Atrium
#
  - "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/config-files/atrium.yaml -o root/atrium.yaml"
  - "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/config-files/atrium.sh | bash"

Use Gitlab variables in instances

Example

From my instances, I should be able to access the variables as user-data using the command scw-userdata <my-variable> after declaring them in Terraform. Some documentation : https://blog.scaleway.com/introducing-scaleway-cloud-init-support/

user_data = {
    atrium_count_index = count.index
    atrium_hostname = scaleway_domain_record.subdomain_record[count.index].name
    atrium_letsencrypt_email = var.LETSENCRYPT_EMAIL
    user_password = var.USER_PASSWORD
    cloud-init = file("../instance-scripts/cloud-init.yml") // this is not a variable, but the declaration of cloud-init file.
}

Setup Terraform Locally

First, you must setup 2 local files for your variables :

variables-local.tf

Create a file variables-local.tf containing the following code :

variable "FORGE_PROJECT_ID" {
  type        = string
  description = "Forge Project ID"
  default     = "your project id"
  sensitive   = true
}

variable "FORGE_USERNAME" {
  type        = string
  description = "Forge Username"
  default     = "your username"
  sensitive   = true
}

variable "FORGE_ACCESS_TOKEN" {
  type        = string
  description = "Forge Access Token"
  default     = "your access token"
  sensitive   = true
}

variables-local.tfvars

Now, you can create a file for your variables information called variables-local.tfvars containing the following code :

### SCW variables

SCW_PROJECT_ID         = ""
SCW_ACCESS_KEY         = ""
SCW_SECRET_KEY         = ""
INSTANCES_COUNT        = "2"
ENVIRONMENT            = ""

Terraform init - Gitlab remote tfstate

You can grab a init command from your gitlab project on menu Infrastructure > Terraform. Select your environment and click the actions button, then you will only need to provide a gitlab project token. Command should look like :

export GITLAB_ACCESS_TOKEN=<YOUR-ACCESS-TOKEN>
terraform init \
    -backend-config="address=https://forge.grandlyon.com/api/v4/projects/<project-id>/terraform/state/<tfstate-name>" \
    -backend-config="lock_address=https://forge.grandlyon.com/api/v4/projects/<project-id>/terraform/state/<tfstate-name>/lock" \
    -backend-config="unlock_address=https://forge.grandlyon.com/api/v4/projects/<project-id>/terraform/state/<tfstate-name>/lock" \
    -backend-config="username=xxxxxxx" \
    -backend-config="password=$GITLAB_ACCESS_TOKEN" \
    -backend-config="lock_method=POST" \
    -backend-config="unlock_method=DELETE" \
    -backend-config="retry_wait_min=5"

terraform init -var-file=variables-local.tfvars

Terraform plan - With variables file

terraform plan -var-file=variables-local.tfvars -out=tfplan

Terraform apply - With plan

terraform apply tfplan

Terraform destroy - With variables file

terraform destroy -var-file=variables-local.tfvars