KKE - Linux Challenges P2
Web Server Security
- During a recent security audit, the application security team of xFusionCorp Industries found security issues with the Apache web server on Nautilus App Server 1 server in Stratos DC. They have listed several security issues that need to be fixed on this server. Please apply the security settings below:
- a. On Nautilus App Server 1 it was identified that the Apache web server is exposing the version number. Ensure this server has the appropriate settings to hide the version number of the Apache web server.
- b. There is a website hosted under /var/www/html/blog on App Server 1. It was detected that the directory /blog lists all of its contents while browsing the URL. Disable the directory browser listing in Apache config.
- c. Also make sure to restart the Apache service after making the changes.
Solution
#ssh to the app server 1 sshpass -p Ir0nM@n -o StrictHostKeyChecking=no tony@stapp01 #check the httpd status sudo systemctl status httpd #open the httpd.conf file with vi sudo vi /etc/httpd/conf/httpd.conf #a. to disable the version discolusre you can append the following two lines to the httpd.conf .> ServerTokens Prod .> ServerSignature Off #b. to disable the directory browser listing, make sure to remove Indexes from options Before > Options Indexes FollowSymLinks After > Options FollowSymLinks #save and quit #c. restart the server and curl to verify sudo systemctl start httpd && sudo systemctl status httpd curl -I stapp01:8080/ #TIP: you can press "/" and write "Options" in vi to search for that line. press n to move to the next matched item. #ref: https://www.tecmint.com/hide-apache-web-server-version-information/ #ref: https://stackoverflow.com/questions/2530372/how-do-i-disable-directory-browsing
Configure Local Yum repos
- The Nautilus production support team and security team had a meeting last month in which they decided to use local yum repositories for maintaing packages needed for their servers. For now they have decided to configure a local yum repo on Nautilus Backup Server. This is one of the pending items from last month, so please configure a local yum repository on Nautilus Backup Server as per details given below.
- a. We have some packages already present at location /packages/downloaded_rpms/ on Nautilus Backup Server.
- b. Create a yum repo named epel_local and make sure to set Repository ID to epel_local. Configure it to use package’s location /packages/downloaded_rpms/.
- c. Install package vim-enhanced from this newly created repo.
Solution:
#Connect to the backup server sshpass -p H@wk3y3 ssh -o StrictHostKeyChecking=no clint@stbkp01 #Switch to the root user sudo su - #there is actually 2 ways to create yum repos, either you do it manually or you can do it with "yum-config-manager --add-repo=<url>" cat <<\ EOF > /etc/yum_repos.d/epel_local.repo [epel_local] name=epel_local baseurl=file:///packages/downloaded_rpms/ enabled=1 gpgcheck=0 EOF #clean and list the repos yum clean all yum repo list #install the vim-enhanced yum install vim-enhanced -y
Setup SSL for Nginx
- The system admins team of xFusionCorp Industries needs to deploy a new application on App Server 1 in Stratos Datacenter. They have some pre-requites to get ready that server for application deployment. Prepare the server as per requirements shared below:
- Install and configure nginx on App Server 1.
- On App Server 1 there is a self signed SSL certificate and key present at location /tmp/nautilus.crt and /tmp/nautilus.key. Move them to some appropriate location and deploy the same in Nginx.
- Create an index.html file with content Welcome! under Nginx document root.
- For final testing try to access the App Server 2 link (either hostname or IP) from jump host using curl command. For example curl -Ik https:///.
Solution
- edit the /etc/nginx/nginx.conf to look like this
#ssh to the app server 1 sshpass -p Ir0nM@n ssh -o StrictHostKeyChecking=no tony@stapp01 #Switch to root user sudo su - #search for the nginx package yum whatprovides nginx yum search nginx #install epel-release & nginx yum install epel-release -y yum install nginx #copy the self-signed cert cp /tmp/nautilus.crt /etc/pki/CA/certs/ && cp /tmp/nautilus.key /etc/pki/CA/private/
- Now lets add index.html file and start the nginx service
#For more information on configuration, see: #* Official English Documentation: http://nginx.org/en/docs/ #* Official Russian Documentation: http://nginx.org/ru/docs/ user nginx; worker_processes auto; error_log /var/log/nginx/error.log; pid /run/nginx.pid; #Load dynamic modules. See /usr/share/doc/nginx/README.dynamic. include /usr/share/nginx/modules/*.conf; events { worker_connections 1024; } http { log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 4096; include /etc/nginx/mime.types; default_type application/octet-stream; # Load modular configuration files from the /etc/nginx/conf.d directory. # See http://nginx.org/en/docs/ngx_core_module.html#include # for more information. include /etc/nginx/conf.d/*.conf; server { listen 80; listen [::]:80; server_name <your_server_ip_here>; root /usr/share/nginx/html; # Load configuration files for the default server block. include /etc/nginx/default.d/*.conf; error_page 404 /404.html; location = /404.html { } error_page 500 502 503 504 /50x.html; location = /50x.html { } } #Settings for a TLS enabled server. server { listen 443 ssl http2; listen [::]:443 ssl http2; server_name <your_server_ip_from_here>; root /usr/share/nginx/html; ssl_certificate "/etc/pki/CA/certs/nautilus.crt"; ssl_certificate_key "/etc/pki/CA/private/nautilus.key"; ssl_session_cache shared:SSL:1m; ssl_session_timeout 10m; ssl_ciphers HIGH:!aNULL:!MD5; ssl_prefer_server_ciphers on; # Load configuration files for the default server block. include /etc/nginx/default.d/*.conf; error_page 404 /404.html; location = /40x.html { } error_page 500 502 503 504 /50x.html; location = /50x.html { } }
#Now, you will have to remove the index.html under the nginx root because is linked to another file. rm -rf /usr/share/nginx/html/index.html #create new index.html under nginx root contains "welcome!" echo "Welcome!" > /usr/share/nginx/html/index.html #start nginx systemctl start nginx systemctl status #verify curl localhot #going back to the jump host lets curl curl -Ik https://172.16.238.10 curl -k https://172.16.238.10
Application Security
- We have a backup management application UI hosted on Nautilus backup server in Stratos DC. That backup management application code is deployed under Apache on the backup server itself, and Nginx is running as a reverse proxy on the same server. Apache and Nginx ports are 6100 and 8093, respectively. We have iptables firewall installed on this server. Make the appropriate changes to fulfill the requirements mentioned below:
- a. We want to open all incoming connections to Nginx’s port and block all incoming connections to Apache’s port. Also make sure rules are permanent.
Solution:
#ssh to the backup server sshpass -p H@wk3y3 ssh -o StrictHostKeyChecking=no clint@stbkp01 #switch to root user sudo su - #verify the ports of nginx and apache ss -lntp | grep nginx ss -lntp | grep http #configure the firewall to match the desired behavior iptables -A INPUT -p tcp --dport 8093 -m conntrack --cstate NEW,ESTABLISHED -j ACCEPT iptables -A INPUT -p tcp --dport 6100 -m conntrack --cstate NEW -j REJECT #save the rules to remain permanent sudo iptables-save > /etc/sysconfig/iptables service iptables save #veriffy cat /etc/sysconfig/iptables iptables -L -n -v #verify from jump host telnet stkbp01 8093 telnet stkbp01 6100
Bash Script
- The production support team of xFusionCorp Industries is working on developing some bash scripts to automate different day to day tasks. One is to create a bash script for taking websites backup. They have a static website running on App Server 1 in Stratos Datacenter, and they need to create a bash script named official_backup.sh which should accomplish the following tasks. (Also remember to place the script under /scripts directory on App Server 1)
- a. Create a zip archive named xfusioncorp_official.zip of /var/www/html/official directory.
- b. Save the archive in /backup/ on App Server 1. This is a temporary storage, as backups from this location will be clean on weekly basis. Therefore, we also need to save this backup archive on Nautilus Backup Server.
- c. Copy the created archive to Nautilus Backup Server server in /backup/ location.
- d. Please make sure script won’t ask for password while copying the archive file. Additionally, the respective server user (for example, tony in case of App Server 1) must be able to run it.
Solution:
- After opening the officila_backup.sh file, copy the following script, save and quit.
#ssh to app server 1 sshpass -p Ir0nM@n ssh -o StrictHostKeyChecking=no tony@stapp01 #cd to /scripts folder and create a official_backup.sh file cd /scripts vi official_backup.sh
- now, to make sure the script won’t ask for password while copying the archive file, lets generate a ssh key
#!/bin/bash #make an archive for the official folder under /var/www/html/ "-r for recurisve" zip -r /backup/xfusioncorp_official.zip /var/www/html/official #copy the archive file to the backup server scp /backup/xfusioncorp_official.zip clint@stbkp01:/backup/
#generate an ssh key ssh-keygen #copy the generated key to the backup server ssh-copy-id clint@stbkp01 #ssh to verify ssh clint@stkp01 #add exec perm to the official_backup.sh file chmod +x official_backup.sh #run the script ./official_backup #ssh to backserver and verify ssh clint@stkp01 ls /backup/
Linux Configure sudo
- We have some users on all app servers in Stratos Datacenter. Some of them have been assigned some new roles and responsibilities, therefore their users need to be upgraded with sudo access so that they can perform admin level tasks.
- a. Provide sudo access to user james on all app servers.
- b. Make sure you have set up password-less sudo for the user.
Solution :
#ssh to all app servers one by one ssh -p <password> ssh -o StrictHostKeyChecking=no <name>@<hostname> #switch to root user sudo su - #Provide sudo access to user james echo "james ALL=(ALL) NOPASSWD:ALL" > /etc/sudoer.d/james #verify su james sudo id sudo su
Apache Redirects
- The Nautilus devops team got some requirements related to some Apache config changes. They need to setup some redirects for some URLs. There might be some more changes need to be done. Below you can find more details regarding that:
- httpd is already installed on app server 3. Configure Apache to listen on port 5003.
- Configure Apache to add some redirects as mentioned below: a.) Redirect http://stapp03.stratos.xfusioncorp.com:/ to http://www.stapp03.stratos.xfusioncorp.com:/ i.e non www to www. This must be a permanent redirect i.e 301 b.) Redirect http://www.stapp03.stratos.xfusioncorp.com:/blog/ to http://www.stapp03.stratos.xfusioncorp.com:/news/. This must be a temporary redirect i.e 302.
Solution :
- Configure redirection
#ssh to app server 3 sshpass -p BigGr33n ssh -o StrictHostKeyChecking=no banner@stapp03 #Switch to the root user sudo su - #verify httpd is alr installed rpm -qa | grep httpd #Configure apache to listen on port 5003 vi /etc/httpd/conf/httpd.conf Look for the line that starts with Listen and change it to > Listen 5003
vi /etc/httpd/conf.d/main.conf #copy the following lines <VirtualHost *:5003> ServerName http://stapp03.stratos.xfusioncorp.com:5003/ Redirect 301 / http://www.stapp03.stratos.xfusioncorp.com:5003/ </VirtualHost> <VirtualHost *:5003> ServerName http://www.stapp03.stratos.xfusioncorp.com:5003/blog/ Redirect 302 / http://www.stapp03.stratos.xfusioncorp.com:5003/news/ </VirtualHost>
#Restart htttpd systemctl restartd httpd #Verify curl http://stapp03.stratos.xfusioncorp.com:5003/ curl http://www.stapp03.stratos.xfusioncorp.com:5003/ curl http://www.stapp03.stratos.xfusioncorp.com:5003/blog/ curl http://www.stapp03.stratos.xfusioncorp.com:5003/news
Linux GPG Encryption
- We have confidential data that needs to be transferred to a remote location, so we need to encrypt that data.We also need to decrypt data we received from a remote location in order to understand its content.
- On storage server in Stratos Datacenter we have private and public keys stored /home/*_key.asc. Use those keys to perform the following actions.
- Encrypt /home/encrypt_me.txt to /home/encrypted_me.asc.
- Decrypt /home/decrypt_me.asc to /home/decrypted_me.txt. (Passphrase for decryption and encryption is kodekloud).
Solution
#ssh to the storage server sshpass -p Bl@ckW ssh -o StrictHostKeyChecking=no natasha@ststor01 #swith to the root user sudo su - #check for the keys ls /home/ cat /home/encrypt_me.txt cat /home/decrypt_me cd /home #Import the keys gpg --import public_key.asc gpg --import private_key.asc #verify gpg --list-keys gpg --list-secret-keys #encryp the encrypt_me.txt file gpg --encrypt -r kodekloud@kodekloud.com --armor < encrypt_me.txt -o encrypted_me.asc #decrypt the message, you will be prompted with a passphrase which is "kodekloud" gpg --decrypt decrypt_me.asc > decrypted_me.txt #validate cat decrypted_me.txt cat decrypt_me.asc cat encrypt_me.txt cat encrpyed_me.asc
Linux LogRotate
- The Nautilus DevOps team is ready to launch a new application, which they will deploy on app servers in Stratos Datacenter. They are expecting significant traffic/usage of httpd on app servers after that. This will generate massive logs, creating huge log files. To utilise the storage efficiently, they need to compress the log files and need to rotate old logs. Check the requirements shared below:
- a. In all app servers install httpd package.
- b. Using logrotate configure httpd logs rotation to monthly and keep only 3 rotated logs. (If by default log rotation is set, then please update configuration as needed)
Solution :
- copy this into the http file under logrotate.d/
#ssh to the app server sshpass -p <passwd> ssh -o StrictHostKeyChecking=no <user>@<hostname> #switch to root user sudo su - #Install httpd package yum install httpd -y #configure logrotate to rotate httpd logs vi /etc/logrotate.d/http #start httpd service systemctl start httpd systemctl status httpd
/var/log/httpd/*log { monthly missingok rotate 3 notifempty sharedscripts compress postrotate /bin/systemctl reload httpd.service > /dev/null 2>/dev/null || true endscript }
Apache heading
- We are working on hardening Apache web server on all app servers. As a part of this process we want to add some of the Apache response headers for security purpose. We are testing the settings one by one on all app servers. As per details mentioned below enable these headers for Apache:
- Install httpd package on App Server 3 using yum and configure it to run on 3003 port, make sure to start its service.
- Create an index.html file under Apache’s default document root i.e /var/www/html and add below given content in it.
- Welcome to the xFusionCorp Industries!
- Configure Apache to enable below mentioned headers:
- X-XSS-Protection header with value 1; mode=block
- X-Frame-Options header with value SAMEORIGIN
- X-Content-Type-Options header with value nosniff => Note: You can test using curl on the given app server as LBR URL will not work for this task
Solution:
#ssh to server app 3 sshpass -p BigGr33n ssh -o StrictHostKeyChecking=no ssh banner@stapp03 #switch to root user sudo su - #install httpd yum install httpd -y #configure httpd to listen on port 3003, and enable the mentioned headers vi /etc/httpd/conf/httpd.conf EDIT > Listen 3003 INSERT > Header set X-XSS-Protection "1; mode=block" INSERT > Header set X-Content-Type-Options nosniff INSERT > Header always append X-Frame-Options SAMEORIGIN #create index.html echo "Welcome to the xFusionCorp Industries!" > /var/www/html/index.html #start the http service systemctl start httpd systemctl status httpd #verify curl -i localhost:3003 #verify again from jumphost curl -i http://stapp03:3003
Install package
- As per new application requirements shared by the Nautilus project development team, serveral new packages need to be installed on all app servers in Stratos Datacenter. Most of them are completed except for git.
- Therefore, install the git package on all app-servers.
Solution
#ssh to app server 1,2 and 3 sshpass -p <pass> ssh -o StrictHostKeyChecking=no <user>@<hostname> #switch to the root user sudo su - #list all the installed packages rpm -qa | grep git #install git yum install git -y #verify yum list installed | grep git git version
Linux Find Command
- During a routine security audit, the team identified an issue on the Nautilus App Server. Some malicious content was identified within the website code. After digging into the issue they found that there might be more infected files. Before doing a cleanup they would like to find all similar files and copy them to a safe location for further investigation. Accomplish the task as per the following requirements:
- a. On App Server 1 at location /var/www/html/official find out all files (not directories) having .js extension.
- b. Copy all those files along with their parent directory structure to location /official on same server.
- c. Please make sure not to copy the entire /var/www/html/official directory content.
Solution:
#connect to app server 1 sshpass -p Ir0nM@n ssh -o StrictHostKeyChecking=no tony@stapp01 #check if /official exists ls /official ls /var/www/html/ #find all the js files under /var/www/html/official find /var/www/html/official -name "*.js" -type f -exec cp --parents {} /official/ \; #verify ls /official/
Linux Postfix Mail
- xFusionCorp Industries has planned to set up a common email server in Stork DC. After several meetings and recommendations they have decided to use postfix as their mail transfer agent and dovecot as an IMAP/POP3 server. We would like you to perform the following steps:
- Install and configure postfix on Stork DC mail server.
- Create an email account javed@stratos.xfusioncorp.com identified by BruCStnMT5.
- Set its mail directory to /home/javed/Maildir.
- Install and configure dovecot on the same server.
Solution:
## ssh to the mail server sshpass -p Gr00T123 ssh -o StrictHostKeyChecking groot@stmail01 #switch to the root user sudo su - #Confirm that yum is installed rpm -qa | grep postfix #Install postfix on the mail server yum install postfix -y #Configure postfix on the server vi /etc/postfix/main.cf #Find the line with #myhostname & #mydomain and set it as follows >myhostname = stmail01.stratos.xfusioncorp.com >mydomain = stratos.xfusioncorp.com #Uncomment the '#myorigin=$mydomain' line >myorigin = $mydomain #Uncomment the '#inet_interfaces = all' line > inet_interfaces = all #Uncomment the '#mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain' line > mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain #Uncomment the '#mynetworks = host IP address, => localhost' line and replace it accordingly >mynetworks = {host IP address}/24, 127.0.0.0/8 #Uncomment the '#home_mailbox = Maildir/' line > home_mailbox = Maildir/ #Save and quit the configuration file #Start postfix and confirm it is working systemctl start postfix systemctl status postfix #Create the user account for javed useradd javed passwd javed telnet stmail01 25 #Enter the following settings: => EHLO localhost => mail from: javed@stratos.xfusioncorp.com => rcpt to: javed@stratos.xfusioncorp.com => DATA => test mail => quit #Install dovecot on mail server yum install dovecot -y #Configure dovecot vi /etc/dovecot/dovecot.conf #tip>:set nu #Uncomment '#protocols = imap pop3 lmtp' > save and quit :wq #modify 10-mail.conf vi /etc/dovecot/conf.d/10-mail.conf #Uncomment the line '#mail_location = maildir:~/Maildir' > save and quit :wq #modify 10-auth.conf vi /etc/dovecot/conf.d/10-auth.conf #Uncomment the line '#disable_plaintext_auth = yes' > disable_plaintext_auth = yes #Set the 'auth_mechanisms' line > auth_mechanisms = plain login > save and quit :wq #modfiy master.conf vi 10-master.conf #Uncomment and set the line '#user = ' > user = postfix > group = postfix #Start dovecot service systemctl start dovecot #Test the configuration telnet stmail01 110 > user javed > pass {given-password} > retr 1 > quit > ss -tulnp
Linux Process Troubleshooting
- The production support team of xFusionCorp Industries has deployed some of the latest monitoring tools to keep an eye on every service, application, etc. running on the systems. One of the monitoring systems reported about Apache service unavailability on one of the app servers in Stratos DC.
- Identify the faulty app host and fix the issue. Make sure Apache service is up and running on all app hosts. They might not hosted any code yet on these servers so you need not to worry about if Apache isn’t serving any pages or not, just make sure service is up and running. Also, do not try to change the Apache port on any host.
Solution:
#First, we need to identify the faulty app. lets use curl from the jump host curl http://stapp01:6200 curl http://stapp02:6200 curl http://stapp03:6200 #In my case, stapp01 was the faulty app, lets ssh to it and find out whats the problem sshpass -p Ir0nM@n ssh -o StrictHostKeyChecking=no tony@stapp01 systemctl start httpd systemctl status httpd httpd -t journalctl -xe cat /etc/httpd/conf/httpd_conf | grep Listen #lets install netstat yum install netstat -y netstat -tulnp |grep 6200 #get the id of the current proccess using port 6200 and kill it, in my case it was 525 kill -9 525 #start the service and verify systemctl start httpd systemctl status httpd curl http://stapp01:6200
Install and Configure PostgreSQL
- The Nautilus application development team has shared that they are planning to deploy one newly developed application on Nautilus infra in Stratos DC. The application uses PostgreSQL database, so as a pre-requisite we need to set up PostgreSQL database server as per requirements shared below:
- a. Install and configure PostgreSQL database on Nautilus database server.
- b. Create a database user kodekloud_sam and set its password to B4zNgHA7Ya.
- c. Create a database kodekloud_db8 and grant full permissions to user kodekloud_sam on this database.
- d. Make appropriate settings to allow all local clients (local socket connections) to connect to the kodekloud_db8 database through kodekloud_sam user using md5 method (Please do not try to encrypt password with md5sum).
- e. At the end its good to test the db connection using these new credentials from root user or server’s sudo use
Solution:
#ssh to the database server sshpass -p Sp\!dy ssh -o StrictHostKeyChecking=no peter@stdb01 #switch to root user sudo su - #install postgresql yum search all postgresql yum whatprovides postgresql yum install postgresql-server postgresql-contrib #iniate the db postgresql-setup initdb #enable and start postgresql systemctl enable postgresql && systemctl start postgresql #create user and grant full permissions sudo -u postgres psql > CREATE USER <username> WITH PASSWORD <password>; > CREATE DATABASE <database_name>; > GRANT ALL PRIVILEGES ON DATABASE <databse_name> TO <username>; >\q #configure to postgres to allow local socket cnx using md5 method vi /var/lib/psql/data/pg_hba.conf #edit the following lines >local all all md5 >host all 127.0.0.1/32 md5 #open postgresql.conf and uncomment listen_addresses line vi /var/lib/psql/data/postgresql.conf >listen_addresses = 'localhost' #restart psq service systemctl restart postgresql systemctl status postgresql #verifying psql -U <username> -d <database_name> -h 127.0.0.1 -W psql -U <username> -d <database_name> -h localhost -W
Tomcat
- The Nautilus application development team recently finished the beta version of one of their Java-based applications, which they are planning to deploy on one of the app servers in Stratos DC. After an internal team meeting, they have decided to use the tomcat application server. Based on the requirements mentioned below complete the task:
- a. Install tomcat server on App Server 2 using yum.
- b. Configure it to run on port 6300.
- c. There is a ROOT.war file on Jump host at location /tmp. Deploy it on this tomcat server and make sure the webpage works directly on base URL i.e without specifying any sub-directory anything like this http://URL/ROOT .
Solution:
#ssh to app server 2 sshpass -p Am3ric@ ssh -o StrictHostKeyChecking=no steve@stapp02 #switch to the root user sudo su - #install tomcat yum install tomcat -y #run tomcat on port 6300 vi /etc/tomcat/server.xml > look for Port and change it to <Connector Port = "6300"> systemctl start tomcat systemctl status tomcat #from the jumphost copy the ROOT.war to app server2 scp ROOT.war steve@lstapp02:/tmp/ #go back to the app server2 mv /tmp/ROOT.war /var/lib/tomcat/webapps/ #verify curl localhost:6300
Install and configure SFTP
- Some of the developers from Nautilus project team have asked for SFTP access to at least one of the app server in Stratos DC. After going through the requirements, the system admins team has decided to configure the SFTP server on App Server 2 server in Stratos Datacenter. Please configure it as per the following instructions:
- a. Create an SFTP user mariyam and set its password to BruCStnMT5.
- b. Password authentication should be enabled for this user.
- c. Set its ChrootDirectory to /var/www/appdata.
- d. SFTP user should only be allowed to make SFTP connections.
Solution:
#ssh to app server2 sshpass -p Am3ric@ ssh -o StrictHostKeyChecking=no steve@stapp02 #switch to the root user sudo su - #make the folder appdata mkdir -p /var/www/appdata #change perms and the owner of appdata folder chmod 755 /var/www/appdata/ chown root:root /var/www/appdata/ #add user mariyam with appdata as its chrootDir useradd -d /var/www/appdata/ mariyam #verify cat /etc/passwd #configure sshd vi /etc/ssh/sshd_config >PasswordAuthentication yes >ChrootDirectory %h >AllowTCPForwarding no >X11Forwarding no >ForceCommand internal-sftp >AllowAgentForwarding no >PermitTunnel no #restart sshd systemctl restart sshd #verify sftp mariyam@stapp02
IPtables Installation And Configuration
- We have one of our website up and running on our Nautilus infrastructure in Stratos DC. Our security team has raised a concern that right now Apache’s port i.e 3000 is open for all since there is no firewall installed on these hosts. So we have decided to add some security layer for these hosts and after discussions and recommendations we have come up with below given requirements:
- Install iptables and all its dependencies on each app host.
- Block incoming port 3000 on all apps for everyone except for LBR host.
- Make sure the rules should persist even after system reboot.
Solution:
#ssh to the app servers sshpass -p <password> ssh -o StrictHostKeyChecking=no <user>@<host> #switch to the root user sudo su - #install iptables yum install iptables-services -y iptables --help #start the iptables systemctl start iptables systemctl enable iptables systemctl status iptables iptables --list #allow LBR host iptables -R INPUT 5 -p tcp --dport 3000 -s 172.16.238.14 -j ACCEPT #-s is for source ip of the LBR, dport is destination port. #block the incoming reqs iptables -A INPUT -p tcp --dport 3000 -j DROP #save service iptables save #verify from the jump host curl 172.16.238.10:3000 #ssh to the LBR host sshpass -p Mischi3f ssh -o StrictHostKeyChecking=no loki@stlb01 #verify curl 172.16.238.10:3000
Install Ansible
- During the weekly meeting, the Nautilus DevOps team discussed about the automation and configuration management solutions that they want to implement. While considering several options, the team has decided to go with Ansible for now due to its simple setup and minimal pre-requisites. The team wanted to start testing using Ansible, so they have decided to use jump host as an Ansible controller to test different kind of tasks on rest of the servers.
- Install ansible version 4.7.0 on Jump host using pip3 only. Make sure Ansible binary is available globally on this system, i.e all users on this system are able to run Ansible commands. Install Ansible
Solution:
pip3 install -U pip sudo pip3 install ansible==4.7.0
Apache Troubleshooting
- xFusionCorp Industries uses some monitoring tools to check the status of every service, application, etc running on the systems. Recently, the monitoring system identified that Apache service is not running on some of the Nautilus Application Servers in Stratos Datacenter.
- Identify the faulty Nautilus Application Server and fix the issue. Also, make sure Apache service is up and running on all Nautilus Application Servers. Do not try to stop any kind of firewall that is already running.
- Apache is running on 3002 port on all Nautilus Application Servers and its document root must be /var/www/html on all app servers.
- Finally you can test from jump host using curl command to access Apache on all app servers and it should be reachable and you should get some static page. E.g. curl http://172.16.238.10:3002/
Soluiton:
$ sshpass -p <password> ssh -o StrictHostKeyChecking=no <username>@<hostname> $ sudo su - $ systemctl start httpd $ httpd -t $ vi +<line> <configfile> #e.g vi +34 /etc/httpd/conf/httpd.conf and fix the line! $ systemctl start httpd $ systemctl status httpd $ systemctl enable --now httpd $ curl <hostname>:3002
Linux Firewalld Setup
- To secure our Nautilus infrastructure in Stratos Datacenter we have decided to install and configure firewalld on all app servers. We have Apache and Nginx services running on these apps. Nginx is running as a reverse proxy server for Apache. We might have more robust firewall settings in the future, but for now we have decided to go with the given requirements listed below:
- a. Allow all incoming connections on Nginx port, i.e 80.
- b. Block all incoming connections on Apache port, i.e 8080.
- c. All rules must be permanent.
- d. Zone should be public.
- e. If Apache or Nginx services aren’t running already, please make sure to start them
Solution
$ sshpass -p <password> ssh -o StrictHostKeyChecking=no <user>@<hostname> $ systemctl status httpd #start if not $ systemctl status nginx #start if not $ sudo yum install firewalld -y $ sudo systemctl enable --now firewalld $ sudo systemctl status firewalld $ sudo firewall-cmd --permanent --zone=public --add-port=80/tcp $ sudo firewall-cmd --zone=public --add-rich-rule='rule family="ipv4" source address="0.0.0.0/0" port port=8080 protocol="tcp" reject' $ sudo firewall-cmd --get-default-zone $ sudo firewall-cmd --list-all #verify from jump host $ curl stapp01:80 #you should get "Working!" $ curl stapp02:8080 #you should get "connection refused msg"
Install and Configure HaProxy LBR
- There is a static website running in Stratos Datacenter. They have already configured the app servers and code is already deployed there. To make it work properly, they need to configure LBR server. There are number of options for that, but team has decided to go with HAproxy. FYI, apache is running on port 3002 on all app servers. Complete this task as per below details.
- a. Install and configure HAproxy on LBR server using yum only and make sure all app servers are added to HAproxy load balancer. HAproxy must serve on default http port (Note: Please do not remove stats socket /var/lib/haproxy/stats entry from haproxy default config.).
- b. Once done, you can access the website using StaticApp button on the top bar
Solution
$ ssh loki@stlb01 $ sudo yum install -y haproxy $ sudo vi /etc/haproxy/haproxy.cfg #make sure to bind on port 80, and to add all app servers within the backend app. "frontend main bind *:80 acl url_static path_beg -i /static /images /javascript /stylesheets acl url_static path_end -i .jpg .gif .png .css .js use_backend static if url_static default_backend app backend app balance roundrobin server app1 172.16.238.10:3002 check server app2 172.16.238.11:3002 check server app3 172.16.238.12:3002 check" #save and quit $ sudo systemctl restart haproxy $ curl localhost:80 #verify $ curl stlb01:80
Haproxy LBR Troubleshooting
- xFusionCorp Industries has an application running on Nautlitus infrastructure in Stratos Datacenter. The monitoring tool recognised that there is an issue with the haproxy service on LBR server. That needs to fixed to make the application work properly.
- Troubleshoot and fix the issue, and make sure haproxy service is running on Nautilus LBR server. Once fixed, make sure you are able to access the website using StaticApp button on the top bar.
Solution
$ sshpass -p Mischi3f ssh -o StrictHostKeyChecking=no loki@stlb01 $ sudo systemctl start haproxy $ sudo systemctl status haproxy $ haproxy -f /etc/haproxy/haproxy.cfg -c #I got misspelling errors in line 51,58 and 32 $ sudo vi +51 /etc/haproxy/haproxy.cfg > "bind *:80" #line 32 within the frontend main > "roundrobin" #line 51 and 58 within the backend static/app $ haproxy -f /etc/haproxy/haproxy.cfg -c $ sudo systemctl restart haproxy $ sudo systemctl status haproxy
Linux Network Services
- Our monitoring tool has reported an issue in Stratos Datacenter. One of our app servers has an issue, as its Apache service is not reachable on port 3000 (which is the Apache port). The service itself could be down, the firewall could be at fault, or something else could be causing the issue.
- Use tools like telnet, netstat, etc. to find and fix the issue. Also make sure Apache is reachable from the jump host without compromising any security settings.
- Once fixed, you can test the same using command curl http://stapp01:3000 command from jump host.
Solution
$ for i in {tony@stapp01,steve@stapp02,banner@stapp03}; do ssh -o StrictHostKeyChecking=no $i "sudo -S systemctl enable httpd"; done $ for i in {1..3}; do curl -I stapp0$i:3000; done $ sshpass -p Ir0nM@n ssh -o StrictHostKeyCheckng=no tony@stapp01 $ sudo su $ apachectl configtest $ httpd -t $ netstat -lnutp | grep 3000 $ kill -p <PID> $ iptables -I INPUT -p tcp --destination-port 3000 -j ACCEPT $ service iptables save $ systemctl enable --now iptables $ systemctl status iptables $ systemctl enable httpd && systemctl start httpd $ curl localhost:3000 #from the jumphost $ curl -I stapp01:3000 $ telnet stapp01 3000
Linux Nginx as Reverse Proxy
- Nautilus system admin’s team is planning to deploy a front end application for their backup utility on Nautilus Backup Server, so that they can manage the backups of different websites from a graphical user interface. They have shared requirements to set up the same; please accomplish the tasks as per detail given below:
- a. Install Apache Server on Nautilus Backup Server and configure it to use 8085 port (do not bind it to 127.0.0.1 only, keep it default i.e let Apache listen on server’s IP, hostname, localhost, 127.0.0.1 etc).
- b. Install Nginx webserver on Nautilus Backup Server and configure it to use 8097.
- c. Configure Nginx as a reverse proxy server for Apache.
- d. There is a sample index file /home/thor/index.html on Jump Host, copy that file to Apache’s document root.
- e. Make sure to start Apache and Nginx services.
- f. You can test final changes using curl command, e.g curl http://:8097.
Solution:
$ scp index.html clint@stbkp01:/tmp/index.html $ sshpass -p H@wk3y3 ssh -o StrictHostKeyChecking=no $ sudo su $ yum install -y nginx httpd $ vi /etc/httpd/conf/httpd.conf > Listen 8085 $ vi /etc/nginx/nginx.conf "user apache #change user to apache server { listen 8097 default_server; #change the listening port listen [::]:80 default_server; server_name _; root /var/www/html; #chage root # Load configuration files for the default server block. include /etc/nginx/default.d/*.conf; location / { proxy_pass http://127.0.0.1:8085/; #pass req to port 8085 }" $ nginx -t $ httpd -t $ systemctl start nginx $ systemctl start httpd $ curl localhost:8097 $ curl stbkp01:8097 #from the jump host.
Configure protected directories in Apache
- xFusionCorp Industries has hosted several static websites on Nautilus Application Servers in Stratos DC. There are some confidential directories in the document root that need to be password protected. Since they are using Apache for hosting the websites, the production support team has decided to use .htaccess with basic auth. There is a website that needs to be uploaded to /var/www/html/itadmin on Nautilus App Server 1. However, we need to set up the authentication before that.
- Create /var/www/html/itadmin direcotry if doesn’t exist.
- Add a user javed in htpasswd and set its password to B4zNgHA7Ya.
- There is a file /tmp/index.html present on Jump Server. Copy the same to the directory you created, please make sure default document root should remain /var/www/html. Also website should work on URL http://:8080/itadmin/
Solution
scp /tmp/index.html tony@stapp01:/tmp/index.html sshpass -p Ir0nM@n ssh -o StrictHostKeyChecking=no mkdir /var/www/html/itadmin/ cd $_ cp /tmp/index.html . vi .htaccess >AuthType Basic >AuthName "Password Required" >Require valid-user >AuthUserFile /etc/httpd/.htpasswd htpasswd -c /etc/httpd/.htpasswd javed > passwd:B4zNgHA7Ya curl -u javed stapp01:8080/itadmin/
Install and Configure Nginx as an LBR
Day by day traffic is increasing on one of the websites managed by the Nautilus production support team. Therefore, the team has observed a degradation in website performance. Following discussions about this issue, the team has decided to deploy this application on a high availability stack i.e on Nautilus infra in Stratos DC. They started the migration last month and it is almost done, as only the LBR server configuration is pending. Configure LBR server as per the information given below:
Install nginx on LBR (load balancer) server.
Configure load-balancing with the an http context making use of all App Servers.
Make sure you do not update the apache port that is already defined in the apache configuration on all app servers, also make sure apache service is up and running on all app servers.
Once done, you can access the website using StaticApp button on the top bar.
Solution
sshpass -p Mischi3f ssh -o StrictHostKeyChecking=no loki@stlb01 sudo yum install -y nginx sudo vi /etc/nginx/nginx.conf > "http { upstream staticapp { server stapp01:8083; server stapp02:8083; server stapp03:8083; } server { listen 80; location / { proxy_pass http://staticapp; } } }" sudo nginx -t sudo systemctl start nginx sudo systemctl enable nginx sudo systemctl status nginx
- We have a requirement where we want to password protect a directory in the Apache web server document root. We want to password protect http://:<apache_port>/protected URL as per the following requirements (you can use any website-url for it like localhost since there are no such specific requirements as of now). Setup the same on App server 3 as per below mentioned requirements:
- a. We want to use basic authentication.
- b. We do not want to use htpasswd file based authentication. Instead, we want to use PAM authentication, i.e Basic Auth + PAM so that we can authenticate with a Linux user.
- c. We already have a user siva with password Rc5C9EyvbU which you need to provide access to
- d. You can test the same using a curl command from jump host curl http://:<apache_port>/protected
Solution
sshpass -p BigGr33n ssh -o StrictHostKeyChecking=no banner@stapp03 yum --enablerepo=epel -y install mod_authnz_external pwauth cat /etc/httpd/conf/httpd.conf | grep Listen curl localhost:8080/protected/index.html vi /etc/httpd/conf.d/authnz_external.conf vi /etc/pam.d/apache > auth required pam_sss.so > account required pam_sss.so httpd -t systemctl start httpd && systemctl enable --now httpd curl localhost:8080/protected curl -u siva:Rc5C9EyvbU localhost:8080/protected/ #from jumphost curl -u siva:Rc5C9EyvbU stapp03:8080/protected/
#/etc/httpd/conf.d/authnz_external.conf DefineExternalAuth pwauth pipe /usr/bin/pwauth <Location "/protected"> AuthType Basic AuthName "PAM Authentication" AuthBasicProvider external AuthExternal pwauth require user siva </Location>
Bash scripts if/else statements
Solution;
#!/bin/bash db_name="kodekloud_db01" db_user="kodekloud_roy" db_pass="asdfgdsd" existing_db=$(mysql -uroot -e "SHOW DATABASES" | grep ${db_name}) if [ -n "${existing_db}" ]; then echo "Database already exists" else mysql -u root -e "CREATE DATABASE ${db_name}" echo "Database kodekloud_db01 has been created" fi mysql -u root -e "CREATE USER '${db_user}'@'%' IDENTIFIED BY '${db_password}'; GRANT ALL PRIVILEGES ON ${db_name}.* TO '${db_user}'@'%'; FLUSH PRIVILEGES;" t_count=$(mysql -uroot -e "USE ${db_name}; SHOW TABLES;" | wc -l) if [ "${t_count}" -gt 0 ]; then echo "databse is not empty" else mysql -uroot ${db_name} < /opt/db_backups/db.sql echo "imported database dump into kodekloud_db01 database." fi mysqldump -uroot ${db_name} > /opt/db_backups/${db_name}.sql