Skip to content
Snippets Groups Projects
Commit 368ffc65 authored by Nathan RODET's avatar Nathan RODET
Browse files

Develop

parent 66993024
Branches main
No related tags found
No related merge requests found
...@@ -15,6 +15,7 @@ before_script: ...@@ -15,6 +15,7 @@ before_script:
- export TF_VAR_INSTANCES_COUNT=${TF_VAR_INSTANCES_COUNT} - export TF_VAR_INSTANCES_COUNT=${TF_VAR_INSTANCES_COUNT}
- export TF_VAR_USER_PASSWORD=${TF_VAR_USER_PASSWORD} - export TF_VAR_USER_PASSWORD=${TF_VAR_USER_PASSWORD}
- export TF_VAR_LETSENCRYPT_EMAIL=${TF_VAR_LETSENCRYPT_EMAIL} - export TF_VAR_LETSENCRYPT_EMAIL=${TF_VAR_LETSENCRYPT_EMAIL}
- export TF_VAR_PROJECT_DOMAIN=${TF_VAR_PROJECT_DOMAIN}
- export TF_VAR_ENVIRONMENT=${ENVIRONMENT} - export TF_VAR_ENVIRONMENT=${ENVIRONMENT}
stages: stages:
......
...@@ -9,7 +9,7 @@ such as credentials to Terraform only during the running state to improve securi ...@@ -9,7 +9,7 @@ such as credentials to Terraform only during the running state to improve securi
On run state, Terraform will create, modify and destroy infrastructure resources in Scaleway to match the configuration described in the configuration files. On run state, Terraform will create, modify and destroy infrastructure resources in Scaleway to match the configuration described in the configuration files.
Resources deployed : **Resources deployed :**
| resource-type | plan | inbound port | | resource-type | plan | inbound port |
|----------------------------------|--------|--------------| |----------------------------------|--------|--------------|
...@@ -19,21 +19,112 @@ Resources deployed : ...@@ -19,21 +19,112 @@ Resources deployed :
| scaleway_instance_security_group | | 22, 443 | | scaleway_instance_security_group | | 22, 443 |
| scaleway_instance_server | DEV1-L | | | scaleway_instance_server | DEV1-L | |
After the resources provisionned with Terraform, Cloud-init will configure the instances by running bash scripts, creating files... And 3 services will be running for each instance : After the resources provisionned with Terraform, Cloud-init will configure the instances by running bash scripts, creating files... And 3 services will be running for each instance.
**Running services :**
- Atrium (Reverse proxy, TLS encryption and HTTPS to the others hosted services) - Atrium (Reverse proxy, TLS encryption and HTTPS to the others hosted services)
- Code-server - Code-server
- Webtop - Webtop
After deployment, each resource can be accessed though HTTPS depending their count number : After deployment, each resource can be accessed though HTTPS depending on their count number :
- https://desktop-0.rust-0.daag.alpha.grandlyon.com/ - https://desktop-0.rust-0.daag.alpha.grandlyon.com/
- https://code-0.rust-0.daag.alpha.grandlyon.com/ - https://code-0.rust-0.daag.alpha.grandlyon.com/
- https://rust-0.daag.alpha.grandlyon.com/
- https://desktop-1.rust-1.daag.alpha.grandlyon.com/ - https://desktop-1.rust-1.daag.alpha.grandlyon.com/
- https://code-1.rust-1.daag.alpha.grandlyon.com/ - https://code-1.rust-1.daag.alpha.grandlyon.com/
- https://rust-1.daag.alpha.grandlyon.com/
### Credentials
**Desktop username : abc**
**Atrium username : admin**
**Password : <Gitlab Stored>**
## How to add new resources
You can add Terraform resources directly to the main.tf file.
Template are available to the link below.
**Terraform Registry for Scaleway provider : https://registry.terraform.io/providers/scaleway/scaleway/latest/docs**
## How to setup configuration file for services
### cloud-init : file creation
In this example, you will find how to create a file to a defined path with cloud-init.
Content inside `%` are meant to be replaced with sed command to be able to use environment variables or user-data.
**example:**
```yaml
write_files:
- content: |
hostname: %atrium_hostname%.daag.alpha.grandlyon.com
debug_mode: false
letsencrypt_email: %atrium_letsencrypt_email%
tls_mode: Auto
apps: # optional : applications served by atrium
- id: 1
name: Code %atrium_count_index%
icon: web_asset
color: 4292030255
is_proxy: true
host: code-%atrium_count_index%
target: localhost:8080
- id: 2
name: Desktop %atrium_count_index%
icon: web_asset
color: 4292030255
is_proxy: true
host: desktop-%atrium_count_index%
target: localhost:8081
path: /root/atrium.yaml
```
**Bash commands example to replace `%` content :**
```bash
sed -i "s/%atrium_hostname%/$(scw-userdata atrium_hostname)/g" /root/atrium.yaml
sed -i "s/%atrium_letsencrypt_email%/$(scw-userdata atrium_letsencrypt_email)/g" /root/atrium.yaml
sed -i "s/%atrium_count_index%/$(scw-userdata atrium_count_index)/g" /root/atrium.yaml
```
User : ## Downloading file
- Username: abc
- Password: <Gitlab Stored> The example show how to curl a configuration file from repository to a defined path with cloud-init.
**example :**
```yaml
runcmd:
#
# cloud-init execute cmd before HOME and USER are set.
#
- "export HOME=/root"
- "export USER=root"
#
# Install Atrium
#
- "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/config-files/atrium.yaml -o root/atrium.yaml"
- "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/config-files/atrium.sh | bash"
```
## Use Gitlab variables in instances
### Example
From my instances, I should be able to access the variables as user-data using the command `scw-userdata <my-variable>` after declaring them in Terraform.
**Some documentation : https://blog.scaleway.com/introducing-scaleway-cloud-init-support/**
```HCL
user_data = {
atrium_count_index = count.index
atrium_hostname = scaleway_domain_record.subdomain_record[count.index].name
atrium_letsencrypt_email = var.LETSENCRYPT_EMAIL
user_password = var.USER_PASSWORD
cloud-init = file("../instance-scripts/cloud-init.yml") // this is not a variable, but the declaration of cloud-init file.
}
```
## Setup Terraform Locally ## Setup Terraform Locally
...@@ -74,7 +165,7 @@ SCW_PROJECT_ID = "" ...@@ -74,7 +165,7 @@ SCW_PROJECT_ID = ""
SCW_ACCESS_KEY = "" SCW_ACCESS_KEY = ""
SCW_SECRET_KEY = "" SCW_SECRET_KEY = ""
INSTANCES_COUNT = "2" INSTANCES_COUNT = "2"
ENVIRONMENT = "devrust" ENVIRONMENT = ""
``` ```
### Terraform init - Gitlab remote tfstate ### Terraform init - Gitlab remote tfstate
...@@ -85,9 +176,9 @@ Command should look like : ...@@ -85,9 +176,9 @@ Command should look like :
```bash ```bash
export GITLAB_ACCESS_TOKEN=<YOUR-ACCESS-TOKEN> export GITLAB_ACCESS_TOKEN=<YOUR-ACCESS-TOKEN>
terraform init \ terraform init \
-backend-config="address=https://forge.grandlyon.com/api/v4/projects/875/terraform/state/devrust" \ -backend-config="address=https://forge.grandlyon.com/api/v4/projects/<project-id>/terraform/state/<tfstate-name>" \
-backend-config="lock_address=https://forge.grandlyon.com/api/v4/projects/875/terraform/state/devrust/lock" \ -backend-config="lock_address=https://forge.grandlyon.com/api/v4/projects/<project-id>/terraform/state/<tfstate-name>/lock" \
-backend-config="unlock_address=https://forge.grandlyon.com/api/v4/projects/875/terraform/state/devrust/lock" \ -backend-config="unlock_address=https://forge.grandlyon.com/api/v4/projects/<project-id>/terraform/state/<tfstate-name>/lock" \
-backend-config="username=xxxxxxx" \ -backend-config="username=xxxxxxx" \
-backend-config="password=$GITLAB_ACCESS_TOKEN" \ -backend-config="password=$GITLAB_ACCESS_TOKEN" \
-backend-config="lock_method=POST" \ -backend-config="lock_method=POST" \
......
...@@ -8,81 +8,38 @@ package_upgrade: true ...@@ -8,81 +8,38 @@ package_upgrade: true
packages: packages:
- curl - curl
- git - git
- argon2
write_files:
#
# Atrium config file
#
- content: |
hostname: %atrium_hostname%.daag.alpha.grandlyon.com
debug_mode: false
letsencrypt_email: %atrium_letsencrypt_email%
tls_mode: Auto
apps: # optional : applications served by atrium
- id: 1
name: Code %atrium_count_index%
icon: web_asset
color: 4292030255
is_proxy: true
host: code-%atrium_count_index%
target: localhost:8080
- id: 2
name: Desktop %atrium_count_index%
icon: web_asset
color: 4292030255
is_proxy: true
host: desktop-%atrium_count_index%
target: localhost:8081
path: /root/atrium.yaml
#
# Code-server config file
#
- content: |
bind-addr: 0.0.0.0:8080
auth: password
password: %user_password%
cert: false
path: /root/config.yaml
runcmd: runcmd:
# #
# Cloud-init execute cmd before HOME and USER are set.
#
- "export HOME=/root"
- "export USER=root"
#
# Install Rust # Install Rust
# #
- "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y" - "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/scripts/rust.sh | bash"
- "source \"$HOME/.cargo/env\""
# #
# Install Docker # Install Docker
# #
- "curl -fsSL https://get.docker.com -o get-docker.sh" - "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/scripts/docker.sh | bash"
- "sh get-docker.sh"
# #
# Install Atrium # Install Atrium
# #
- "mkdir -p /root/atrium/data && mkdir -p /root/atrium/letsencrypt_cache" - "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/scripts/atrium.sh | bash"
- 'sed -i "s/%atrium_hostname%/$(scw-userdata atrium_hostname)/g" /root/atrium.yaml'
- 'sed -i "s/%atrium_letsencrypt_email%/$(scw-userdata atrium_letsencrypt_email)/g" /root/atrium.yaml'
- 'sed -i "s/%atrium_count_index%/$(scw-userdata atrium_count_index)/g" /root/atrium.yaml'
- "mv /root/atrium.yaml /root/atrium/atrium.yaml"
- "chown -Rf 1000:1000 /root/atrium"
# #
# Run Atrium Container # Install Code Server
# #
- "docker run -d --name atrium --restart unless-stopped --net=host -v /root/atrium/atrium.yaml:/app/atrium.yaml -v /root/atrium/letsencrypt_cache:/app/letsencrypt_cache -v /root/atrium/data:/app/data nicolaspernoud/atrium:latest" - "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/scripts/code-server.sh | bash"
# #
# Install And Run Code Server # Install Webtop
# #
- "export HOME=/root" - "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/scripts/webtop.sh | sh"
- "export USER=root"
- "mkdir -p /root/.config/code-server/"
- 'sed -i "s/%user_password%/$(scw-userdata user_password)/g" /root/config.yaml'
- "mv /root/config.yaml /root/.config/code-server/config.yaml"
- "curl -fsSL https://code-server.dev/install.sh | sh"
- "sudo systemctl enable --now code-server@$USER"
# #
# Run Webtop Container # Install ttyd
# #
- "docker run -d --name=webtop --security-opt seccomp=unconfined -e PUID=0 -e PGID=0 -e TZ=Europe/Paris -e SUBFOLDER=/ -e KEYBOARD=fr-fr-azerty -e AUTO_LOGIN=false -p 8081:3000 -v /root/webtop:/config -v /var/run/docker.sock:/var/run/docker.sock --shm-size=8gb --cap-add=NET_ADMIN --privileged --restart unless-stopped --sysctl net.ipv6.conf.all.disable_ipv6=0 lscr.io/linuxserver/webtop:ubuntu-xfce" - "curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/scripts/ttyd.sh | bash"
- 'echo "$(scw-userdata user_password)\n$(scw-userdata user_password)" | docker exec -i webtop passwd abc'
# #
# Install Repository # Install Repository
# #
......
# The content of this file is recreated by cloud init. # The content of this file is recreated by cloud init.
hostname: %atrium_hostname%.daag.alpha.grandlyon.com hostname: %atrium_hostname%.%project_domain%
debug_mode: false debug_mode: false
letsencrypt_email: %atrium_letsencrypt_email% letsencrypt_email: %atrium_letsencrypt_email%
tls_mode: Auto tls_mode: Auto
...@@ -17,4 +17,19 @@ apps: # optional : applications served by atrium ...@@ -17,4 +17,19 @@ apps: # optional : applications served by atrium
color: 4292030255 color: 4292030255
is_proxy: true is_proxy: true
host: desktop-%atrium_count_index% host: desktop-%atrium_count_index%
target: localhost:8081 target: localhost:8081
\ No newline at end of file - id: 3
name: Terminal %atrium_count_index%
icon: terminal
color: 4292030255
is_proxy: true
host: terminal-%atrium_count_index%
target: localhost:8082
secured: true
login: admin
password: %user_password%
users:
- login: admin
password: %user_password_hash%
roles:
- ADMINS
\ No newline at end of file
[Unit]
Description=ttyd web server to allow remote access to terminal
After=network.target
[Service]
ExecStart=/root/ttyd/ttyd.x86_64 -p 8082 -i lo -c admin:%user_password% -T xterm bash
[Install]
WantedBy=multi-user.target
\ No newline at end of file
#!/bin/bash
#
# Install Atrium
#
#
# Example : Generate user_password Argon2 encoded hash for Atrium
# echo $(scw-userdata user_password) | argon2 $(echo $RANDOM | md5sum | head -c 10) -id -l 32 -e
#
export USER_PASSWORD_HASH=$(echo "$(scw-userdata user_password)" | argon2 "$(echo $RANDOM | md5sum | head -c 10)" -id -l 32 -e)
mkdir -p /root/atrium/data && mkdir -p /root/atrium/letsencrypt_cache
curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/config-files/atrium.yaml -o root/atrium/atrium.yaml
sed -i "s|%atrium_hostname%|$(scw-userdata atrium_hostname)|g" /root/atrium/atrium.yaml
sed -i "s|%project_domain%|$(scw-userdata project_domain)|g" /root/atrium/atrium.yaml
sed -i "s|%atrium_letsencrypt_email%|$(scw-userdata atrium_letsencrypt_email)|g" /root/atrium/atrium.yaml
sed -i "s|%atrium_count_index%|$(scw-userdata atrium_count_index)|g" /root/atrium/atrium.yaml
sed -i "s|%user_password_hash%|$USER_PASSWORD_HASH|g" /root/atrium/atrium.yaml
sed -i "s|%user_password%|$(scw-userdata user_password)|g" /root/atrium/atrium.yaml
chown -Rf 1000:1000 /root/atrium
#
# Run Atrium Container
#
docker run -d --name atrium \
--restart unless-stopped \
--net=host \
-v /root/atrium/atrium.yaml:/app/atrium.yaml \
-v /root/atrium/letsencrypt_cache:/app/letsencrypt_cache \
-v /root/atrium/data:/app/data nicolaspernoud/atrium:latest
#!/bin/bash
#
# Install Code Server
#
mkdir -p /root/.config/code-server/
curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/config-files/config.yaml -o root/.config/code-server/config.yaml
sed -i "s/%user_password%/$(scw-userdata user_password)/g" /root/.config/code-server/config.yaml
curl -fsSL https://code-server.dev/install.sh | sh
#
# Run Code Server
#
sudo systemctl enable --now code-server@$USER
\ No newline at end of file
#!/bin/bash
#
# Install Docker
#
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh
rm get-docker.sh
#!/bin/bash
#
# Install Rust
#
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source "$HOME/.cargo/env"
\ No newline at end of file
#!/bin/sh
#
# Run ttyd service
#
mkdir -p /root/ttyd
curl https://github.com/tsl0922/ttyd/releases/download/1.7.2/ttyd.x86_64 -o root/ttyd/ttyd.x86_64
chmod +x /root/ttyd/ttyd.x86_64
curl https://forge.grandlyon.com/systemes-dinformation/alpha/projects/labo-rust/-/raw/develop/instance-scripts/config-files/ttyd.service -o etc/systemd/system/ttyd.service
sed -i "s|%user_password%|$(scw-userdata user_password)|g" /etc/systemd/system/ttyd.service
systemctl enable ttyd.service && systemctl start ttyd.service
\ No newline at end of file
#!/bin/sh
#
# Run Webtop Container
#
docker run -d --name=webtop \
--security-opt seccomp=unconfined \
-e PUID=0 \
-e PGID=0 \
-e TZ=Europe/Paris \
-e SUBFOLDER=/ \
-e KEYBOARD=fr-fr-azerty \
-e AUTO_LOGIN=false \
-p 8081:3000 \
-v /root/webtop:/config \
-v /var/run/docker.sock:/var/run/docker.sock \
--shm-size=8gb \
--cap-add=NET_ADMIN \
--privileged \
--restart unless-stopped \
--sysctl net.ipv6.conf.all.disable_ipv6=0 lscr.io/linuxserver/webtop:ubuntu-xfce
#
# Modify default password
#
echo "$(scw-userdata user_password)\n$(scw-userdata user_password)" | docker exec -i webtop passwd abc
\ No newline at end of file
...@@ -65,6 +65,7 @@ resource "scaleway_instance_server" "user_instance_server" { ...@@ -65,6 +65,7 @@ resource "scaleway_instance_server" "user_instance_server" {
security_group_id = scaleway_instance_security_group.www.id security_group_id = scaleway_instance_security_group.www.id
user_data = { user_data = {
project_domain = var.PROJECT_DOMAIN
atrium_count_index = count.index atrium_count_index = count.index
atrium_hostname = scaleway_domain_record.subdomain_record[count.index].name atrium_hostname = scaleway_domain_record.subdomain_record[count.index].name
atrium_letsencrypt_email = var.LETSENCRYPT_EMAIL atrium_letsencrypt_email = var.LETSENCRYPT_EMAIL
......
...@@ -18,6 +18,12 @@ variable "SCW_SECRET_KEY" { ...@@ -18,6 +18,12 @@ variable "SCW_SECRET_KEY" {
description = "Scaleway secret key" description = "Scaleway secret key"
} }
variable "PROJECT_DOMAIN" {
type = string
sensitive = true
description = "External domain from Scaleway"
}
variable "LETSENCRYPT_EMAIL" { variable "LETSENCRYPT_EMAIL" {
type = string type = string
sensitive = true sensitive = true
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment