Scenario: "Gitega": Find the Bad Git Commit

Level: Easy

Type: Do

Access: Paid

Description: The directory at /home/admin/git has a Git repository with a Golang program and a test for it.

To execute the test, from this "git" directory run: go test. The last (current HEAD) commit fails the test. Suppose the first commit passed the test.

Find the (long hash) commit that first broke the test and enter it in the /home/admin/solution file. For example: echo 9e80a7eb1b09385e93ab4a76cb2c93beec48fd9f > /home/admin/solution

Test: Doing md5sum /home/admin/solution returns f7db1bb6b7bfcd66a4eb66782804b39d.

The "Check My Solution" button runs the script /home/admin/agent/check.sh, which you can see and execute.

Time to Solve: 15 minutes.

Scenario: "Rio de Janeiro": Do we have another option?

Level: Easy

Type: Fix

Access: Email

Description: This scenario server is dedicated to Jenkins, a Java application managed by systemd. Jenkins is failing to start. Troubleshoot and find the problem, then apply the solution so Jenkins runs properly.

Test: The service must return the string "Sign in - Jenkins" amongst some other html code. You can check with the command curl -s localhost:8888/login | grep Jenkins | head -n1

The "Check My Solution" button runs the script /home/admin/agent/check.sh, which you can see and execute.

Time to Solve: 15 minutes.

Scenario: "Warsaw": Prometheus can't scrape the webserver

Level: Medium

Type: Fix

Access: Paid

Description: A developer created a golang application that is exposing the /metrics endpoint. They have a problem with scraping the metrics from the application. They asked you to help find the problem.

Full source code of the application is available at the /home/admin/app directory.

Credit Kamil Błaż

Test: The endpoint http://localhost:9000/metrics should return HTTP code 200.

The "Check My Solution" button runs the script /home/admin/agent/check.sh, which you can see and execute.

Time to Solve: 30 minutes.

Scenario: "Bangalore": Envoy Panics

Level: Medium

Type: Fix

Access: Email

Description: You have an Envoy proxy routing traffic to two unhealthy backend services.

When the number of healthy backends falls under a panic threshold (default 50%), Envoy enters a panic mode and it will either send traffic to all upstream hosts or to none at all.

We are simulating this condition by returning an HTTP 503 status code from the /health endpoint in both backends. In our case Envoy is sending traffic to all upstream services.

Your objective is to change the panic Envoy behaviour so that it does not route any traffic to unhealthy services and instead Envoy returns "no healthy upstream". (Do not change anything in the backend services). There can also be other Envoy configuration issues you need to fix.

Test: Running curl localhost:10000 should return no healthy upstream.

The "Check My Solution" button runs the script /home/admin/agent/check.sh, which you can see and execute.

Time to Solve: 15 minutes.

Scenario: "Campina Grande": Give me my cert, Vault

Level: Medium

Type: Fix

Access: Paid

Description: A web application running at https://nginx.example.com has an expired certificate. Issue a new certificate using the Hashicorp Vault running on the server.
The Vault instance is already unsealed and initialized, and you have full admin access with the admin user.

Test: Running curl https://nginx.example.com returns Hello!.

The certificate presented by Nginx is issued by the Vault PKI (check using openssl verify -CAfile /usr/local/share/ca-certificates/vault-pki-ca.crt /etc/nginx/ssl/cert.pem).

The "Check My Solution" button runs the script /home/admin/agent/check.sh, which you can see and execute.

Time to Solve: 15 minutes.

Scenario: "Chennai": Pull a Rabbit from a Hat

Level: Hard

Type: Fix

Access: Paid

Description: There is a RabbitMQ (RMQ) cluster defined in a docker-compose.yml file.

Bring this system up and then run the producer.py script in such a way that is able to send messages to RMQ. In particular you have to send the message "hello-lwc".

- RMQ is a queuing system: messages are put in the queue with a "producer" and they are taken out from the other side by a "consumer". The queue name has to be the same for both.

- To send the message "hello-lwc": python3 ~/producer.py hello-lwc. Should return Message sent to RabbitMQ. "IncompatibleProtocolError" means RMQ is not working properly.

- To test consuming it: python3 ~/consumer.py, this will retrieve the next message from the queue and print it. Once everything is working send more than one message so there's at least one in the queue when the validation runs.

- Do not change the consumer.py and producer.py files; if you do the Check My Solution will fail.

Test: python3 ~/consumer.py returns hello-lwc

See /home/admin/agent/check.sh for the exact test.

Time to Solve: 30 minutes.

Scenario: "Amygdala": Do you have enough insight to see the secrets?

Level: Hard

Type: Fix

Access: Paid

Description: Troubleshoot and fix a Kubernetes web application running in the app namespace. Make the deployment run successfully so that it returns Hello handsome! when you curl it.

Fix first your admin user access to the local Kubernetes cluster; the KUBECONFIG environment variable must be set to $HOME/.kube/config.

You have full admin access to a Vault server (containing the secrets you need) from the admin user. All the used manifests for the application are placed on the /home/admin/manifests directory.

Test: Running: POD_IP=$(kubectl get po -n app -l app=app -o jsonpath='{.items[0].status.podIP}') curl http://$POD_IP returns Hello handsome!.

The "Check My Solution" button runs the script /home/admin/agent/check.sh, which you can see and execute.

Time to Solve: 20 minutes.

Scenario: "Cabedelo": Harbor full of issues

Level: Hard

Type: Fix

Access: Email

Description: You need to build and push a docker image without changing the Dockerfile to your company's Harbor registry, which is running at harbor.sadservers.local, with its home directory at /opt/harbor. You have full admin access with admin:Harbor12345 credential. The source code and the Dockerfile are in the ~/app directory. The image name must be harbor.sadservers.local/images/app:1.0.0. It is also expected that the application will be up and running at localhost:5000 in a container named app.

IMPORTANT. Do not:
1. Generate new internal certificates
2. Change the Dockerfile
3. Change the /opt/harbor.yml file

Test: You are able to pull the application image from Harbor:
docker rmi harbor.sadservers.local/images/app:1.0.0
docker pull harbor.sadservers.local/images/app:1.0.0


You can access the application; curl localhost:5000 returns Hello world!

The "Check My Solution" button runs the script /home/admin/agent/check.sh, which you can see and execute.

Time to Solve: 20 minutes.