Skip to content
Snippets Groups Projects
README.md 9.21 KiB
Newer Older
  • Learn to ignore specific revisions
  • Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    > Linux Debug Tools For Ops
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    Mainly specific client protocol to test specific server protocol, like curl to test an http server, ldapsearch for a ldap server or mail to test a SMTP relay etc.
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    But you will also find many ways to set proxy settings on your tools.
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    There are usefull tools during the deployment stages in qualification, staging, or production environment (remove them after troubleshooting).
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    Tags: DevOps, Debug, tools, linux, ops, deploy, interface, testing
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    [TOC]
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # TLDR; Install all this interfaces debug tools
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    dnf install -y  net-tools \
                    bind-utils \
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
                    openldap-clients \
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
                    mailx \
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
                    wget \
                    git \
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
                    curl \
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
                    telnet
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # Client TCP
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## TCP Parameters 
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    ```shell
    TCP_HOST=my.domain.tld      # or : "localhost", or ip like 192.168.0.10.
    TCP_PORT=80                 # ex: port 22 (ssh) ou 80 (http) etc.
    ```
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## TCP Test with Telnet
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    ```shell
    # dnf install -y telnet # apt install -y telnet # yum install -y telnet 
    
    telnet $TCP_HOST $TCP_PORT
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    telnet my.hostname.tld 443 # example
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    ```
    
    > For UDP use nc : 
    > `nc -z -v -u $UDP_HOST $UDP_PORT`
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # Client (s)FTP
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## (s)FTP parameters
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    FTP_USER=remote_username
    FTP_HOST=hostname
    ```
    
    ## sFTP Testing 
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    sftp $FTP_USER@$FTP_HOST
    ```
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # Client HTTP(s)
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    ## HTTP(s) Parameter
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    HTTP_PROTOCOL=http # default http; https if secured.
    HTTP_HOST=www.mydomain.com # ex: www.domain.tld
    HTTP_PORT=80 # default 80 not secure; 443
    HTTP_ROOT_URL=$HTTP_PROTOCOL://$HTTP_HOST:$HTTP_PORT
    ```
    
    ## HTTP Testing with Curl
    
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # dnf install -y curl # yum install -y curl # apt install -y curl
    
    curl -v $HTTP_ROOT_URL
    
    curl -v http://httpbin.org/get  # example
    
    # or if tls/ssl certifate not valid you can temporary ignore that with -k option
    
    curl -kv $HTTP_ROOT_URL
    
    curl -kv https://httpbin.org/get # example
    ```
    
    
    > You can also use `wget` to request a http server
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # Client DNS 
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## DNS Parameters
    
    
    > If you don't know the following values : Ask your administrator, the SMTP manager or your integrator.
    
    DNS variables are : 
    
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    DNS_DOMAIN=mydomain.com     # ex: mydomain.com
    
    HOST=sub.mydomain.com          # ex: mydomain.com or subdomain.mydomain.tld (=FQDN) 
    ```
    
    ## DNS discovery
    
    Retrieve DNS serveur : 
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    dnf install -y bind-utils # yum install -y bind-utils 
    
    host -t ns $DNS_DOMAIN 
    
    host -t ns domain.com # example
    
    # or 
    dig ns $DNS_DOMAIN
    
    host -t ns domain.com # example
    ```
    
    ## DNS Test : Ip resolution from Host  
    
    To get ip address from an host or FQDN :
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    # With Host command
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    host $HOST
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    host  sub.domain.com # example 
    
    # or with NSlookup
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    nslookup $HOST
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    nslookup sub.domain.com # example
    
    # or with Dig
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    dig $HOST
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    dig sub.domain.com # example
    dig sub.domain.com +short # example
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # Client LDAP
    
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## LDAP Parameters
    
    > If you don't know the following values : Ask your administrator, the SMTP manager or your integrator.
    
    LDAP variables are : 
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    LDAP_PROTOCOL=ldap              # or ldaps
    LDAP_HOST=                      # ex: ldap.domain.tld
    LDAP_PORT=389                   # default : 389 (LDAP), 636 (LDAPS)
    LDAP_URL=$LDAP_PROTOCOL://$LDAP_HOST:$LDAP_PORT
    LDAP_USERS_BASE=
    LDAP_BIND_FORMAT=
    LDAP_GROUPE_ATTRIBUT=
    LDAP_USERS_CONNECTION_STRING=
    LDAP_USER_CONNECTION_STRING=
    LDAP_BIND_READONLY_USER_DN=
    LDAP_USER_DN=
    
    LDAP_FILTER=
    
    LDAP_ATTR=*                     # * for "For all attributs", ex: givenName, sn, mail, uid
    ```
    
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    Base:         "dc=example,dc=com",
    Host:         "ldap.example.com",
    Port:         389,
    UseSSL:       false,
    BindDN:       "uid=readonlysuer,ou=People,dc=example,dc=com",
    BindPassword: "readonlypassword",
    UserFilter:   "(uid=%s)",
    GroupFilter: "(memberUid=%s)",
    Attributes:   []string{"givenName", "sn", "mail", "uid"},
    ```
    
    ## LDAP Test
    
    
    ```shell
    # Installation
    ### RHEL 7
    # yum install openldap-client -y 
    ### RHEL 8
    # dnf install openldap-clients -y
    
    
    ldapsearch -x -h $LDAP_HOST \
    -D $LDAP_BIND_READONLY_USER_DN \
    -W \
    -b $LDAP_USERS_BASE \
    -s sub "(${LDAP_FILTER})" $LDAP_ATTR
    ```
    
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # Client SMTP
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## SMTP Parameters
    
    > If you don't know the following values : Ask your administrator, the SMTP manager or your integrator.
    
    SMTP variables are : 
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    SMTP_HOST=                              # default
    SMTP_PORT=                              # default port 25 not secure, and 587 for encrypted (secure)serveur
    # SMTP_USERNAME=                        # optional
    # SMTP_PASSWORD=                        # optional
    
    SMTP_SUBJECT="Test Subject"
    SMTP_BODY="Test Body"
    SMTP_RECIPIENT="recipient@domain.tld"   # Example
    # SMTP_FROM="sender@domain.tld"         # Optional
    ```
    
    ## SMTP Test with Mail command
    
    Test without SMTP Authentication : 
    
    > Troubleshooting : mail command sometimes use /tmp folder, if you don't have permission to write inside it, mail command could fail.
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    dnf install -y mailx # yum install -y mailx # RHEL installation of mail command
    
    echo $SMTP_BODY | mail -S smtp=$SMTP_HOST -s $SMTP_SUBJECT $SMTP_RECIPIENT
    
    echo "body" | mail -S smtp=hostname -s "suject" "dest@domain.org"
    ```
    
    or
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    echo $SMTP_BODY | mail -S smtp=$SMTP_HOST -s $SMTP_SUBJECT $SMTP_RECIPIENT
    ```
    
    or
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    echo $SMTP_BODY | mail -S smtp=$SMTP_HOST -s $SMTP_SUBJECT -r $SMTP_FROM $SMTP_RECIPIENT
    ```
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## SMTP Testing with Telnet
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    With authentication :
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # Get base64 Login 
    perl -MMIME::Base64 -e 'print encode_base64("LOGIN_HERE");'
    # => Z2NpcaxaaWx4ZZzzV2
    
    # Get base64 Password 
    perl -MMIME::Base64 -e 'print encode_base64("PWD_HERE");'
    # => Z2NpNOTaREALexE4ZGV2
    
    telnet $SMTP_HOST $SMTP_PORT
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    EHLO mydomain.com
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    AUTH LOGIN
    Z2NpcmVjdWx4ZGV2
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    Z2NpFAUSE_CHAINEcmVmdWx4ZGV2
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # => "235 2.7.0 Authentication successful"
    
    MAIL FROM:$SMTP_FROM
    RCPT TO:<${SMTP_RECIPIENT}> NOTIFY=success,failure
    DATA
    Subject: $SMTP_SUBJECT
    <empty line, press "return" command >
    $SMTP_BODY
    .
    # Press "." at on the last line to send email.
    ```
    
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # Client GIT 
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## Git Parameter
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    GIT_URL=https://git.domain.com/repository # 
    ```
    
    ## Git Testing
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    git clone $GIT_URL
    ```
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # Show Listened ports
    
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    dnf install -y net-tools # yum install -y net-tools 
    netstat -planteu
    ```
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # PROXY HTTP : Reach a HTTP serveur Through a Proxy
    
    > How to set proxy which allow to reach a http serveur on another network (commonly on internet)
    
    Proxy parameters:
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    PROXY_HOST=proxy.domain.com
    PROXY_PORT=8080              # ex: 8080, or 443, etc.
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    # PROXY_USERNAME=__YOUR_PROXY_USERNAME__ 
    # PROXY_PASSWORD=__YOUR_PROXY_PASSWORD__
    
    URL=https://httpbin.org/get # Example
    ```
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## Set proxy globaly on linux
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    vi /etc/profile.d/http_proxy.sh
    
    export http_proxy=$PROXY_HOST:$PROXY_PORT
    export https_proxy=$PROXY_HOST:$PROXY_PORT
    
    ```
    
    Example :
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    vi /etc/profile.d/http_proxy.sh
    
    export http_proxy=proxy.domain.com:443
    export https_proxy=proxy.domain.com:443
    
    ```
    
    You can check if this variables are loaded :
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    env | grep -i "proxy"
    ```
    
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## Curl behind a proxy
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    curl -v --proxy $PROXY_HOST:$PROXY_PORT  $URL
    
    # Exemple 
    curl -kv --proxy http://myproxy.domain.tld:8080  https://httpbin.org/get
    
    ```
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## Git behind a proxy
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    Globaly :
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    git config --global http.proxy http://$PROXY_HOST:$PROXY_HOST
    git config --global https.proxy http://$PROXY_HOST:$PROXY_HOST 
    
    # unset
    # git config --global --unset http.proxy
    # git config --global --unset https.proxy
    ```
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    Specific for a repository :
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    git config --global http.https://domain.com.proxy http://proxyUsername:proxyPassword@proxy.server.com:port
    ```
    
    or edit the ~/.gitconfig :
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    [http]
    [http "https://domain.com"]
    	proxy = http://proxyUsername:proxyPassword@proxy.server.com:port
    ```
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## Docker behind a proxy
    
    ### Docker daemon add proxy settings
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    cat /etc/systemd/system/docker.service.d/http-proxy.conf
    [Service]
    Environment="HTTP_PROXY=http://$PROXY_HOST:$PROXY_PORT"
    Environment="HTTPS_PROXY=http://$PROXY_HOST:$PROXY_PORT"
    Environment="NO_PROXY=localhost,mydomain.com"
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ...
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    sudo systemctl daemon-reload
    sudo systemctl restart docker
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```
    
    ### Docker build behind Proxy
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    docker build --build-arg http_proxy=http://$PROXY_HOST:$PROXY_PORT --build-arg https_proxy=http://$PROXY_HOST:$PROXY_PORT . 
    
    # Example for MAVEN inside a container 
    # Add at top of Dockerfile : ARG MAVEN_OPTS
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    docker build --build-arg MAVEN_OPTS="-Dhttp.proxyHost=$PROXY_HOST -Dhttp.proxyPort=$PROXY_PORT -Dhttps.proxyHost=$PROXY_HOST -Dhttps.proxyPort=$PROXY_PORT" .
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## NPM (Javascript packager) behind a Proxy
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    npm config set proxy http://$PROXY_HOST:$PROXY_PORT
    npm config set https-proxy http://$PROXY_HOST:$PROXY_PORT
    ```
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ## PIP (Python packager) behind a Proxy
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    pip install --proxy=http://$PROXY_HOST:$PROXY_PORT -r requirements.txt
    ```
    
    ## Gitlab Runner behind a proxy
    
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    ```shell
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    mkdir /etc/systemd/system/gitlab-runner.service.d
    vi /etc/systemd/system/gitlab-runner.service.d/http-proxy.conf
    
    [Service]
    Environment="HTTP_PROXY=http://$PROXY_HOST:$PROXY_PORT"
    Environment="HTTPS_PROXY=http://$PROXY_HOST:$PROXY_PORT"
    # save & quit ESC +  ":wq"
    
    systemctl daemon-reload
    sudo systemctl restart gitlab-runner
    systemctl show --property=Environment gitlab-runner
    gitlab-runner stop
    gitlab-runner start
    ```
    
    Benoit MARECHAL's avatar
    Benoit MARECHAL committed
    
    ## Maven (Java Packaging)
    
    ```shell
    export MAVEN_OPTS="-Dhttp.proxyHost=$PROXY_HOST -Dhttp.proxyPort=$PROXY_PORT -Dhttps.proxyHost=$PROXY_HOST -Dhttps.proxyPort=$PROXY_PORT"
    # set MAVEN_OPTS.
    ```