🚀 DevSecOps Journey — Day 11

Building a Website Health Monitoring & Log Archival Automation with Shell Scripting

✨ From basic scripting to production-style automation

Today, I worked on a hands-on automation project that closely mirrors how monitoring and log management are handled in real DevOps and DevSecOps environments.

The focus was on:

• 📊 Website health monitoring

• 🧾 Structured logging

• 🗂 Log archival & cleanup automation

---

🌐 Project 1: Website Health Monitoring Script

The goal of this script is to continuously monitor a website’s availability and performance and store the data in daily logs.

---

🔍 What the Script Does

• Takes a target URL

• Creates a dedicated log directory

• Generates date-based log files

• Makes an HTTP request using curl

• Extracts:

  • ✅ HTTP status code

  • ⏱ Response time

• Logs everything with a timestamp into a daily log file

This approach is commonly used in:

• Uptime monitoring

• Synthetic health checks

• SRE probes

• Custom monitoring agents

---

🛠 Key Concepts Used

• Shell scripting fundamentals

• Variables & directory management

• curl for HTTP/API calls

• Parsing output using awk

• Timestamp generation

• Output redirection (>>)

---

🧪 Website Health Script

#!/bin/bash

URL="https://github.com"

LOG_DIR="$HOME/website_health_log"
mkdir -p "$LOG_DIR"

TODAY=$(date +"%Y-%m-%d")
LOG_FILE="$LOG_DIR/health_log_$TODAY.log"

response=$(curl -o /dev/null -s -A "Mozilla/5.0" -w "%{http_code} %{time_total}" "$URL")

status_code=$(echo "$response" | awk '{ print $1 }')
response_time=$(echo "$response" | awk '{ print $2 }')

timestamp=$(date +"%Y-%m-%d %H:%M:%S")

echo "$timestamp | status : $status_code | response : ${response_time}s" >> "$LOG_FILE"

📌 Why this matters:
This simulates how real monitoring systems capture availability and performance metrics in production.

---

🗂 Project 2: Log Archival & Cleanup Automation

Monitoring generates logs continuously. Without cleanup, storage can quickly grow out of control.

To solve this, I created a log archival script.

---

📦 What the Archival Script Does

• Identifies yesterday’s log file

• Compresses it into a .tar.gz archive

• Deletes the original log file

• Maintains an archive_history.log

• Prevents duplicate archiving

This mirrors real log retention strategies used in production systems.

---

🧪 Log Archival Script

#!/bin/bash

LOG_DIR="$HOME/website_health_logs"
YESTERDAY=$(date -d "yesterday" +"%Y-%m-%d")

YESTERDAY_LOG="$LOG_DIR/health_log_$YESTERDAY.log"
YESTERDAY_ARCHIVE="$LOG_DIR/health_log_$YESTERDAY.tar.gz"
ARCHIVE_HISTORY="$LOG_DIR/archive_history.log"

if [ -f "$YESTERDAY_LOG" ] && [ ! -f "$YESTERDAY_ARCHIVE" ]; then
    tar -czf "$YESTERDAY_ARCHIVE" "$YESTERDAY_LOG"
    rm "$YESTERDAY_LOG"
    echo "$(date) | Archived $YESTERDAY_LOG" >> "$ARCHIVE_HISTORY"
else
    echo "$(date) | Nothing to archive or already archived" >> "$ARCHIVE_HISTORY"
fi

---

🔐 Why Log Archival Matters

Proper log lifecycle management helps with:

• 💾 Disk space optimization

• 📜 Audit & compliance readiness

• 🧹 Clean operational environments

• ⚙️ Predictable automation behavior

---

🎯 Why This Project Matters for DevOps / DevSecOps

This mini-project combines multiple production-grade practices:

• 🔄 Automation over manual monitoring

• 📊 Observability fundamentals

• 🗂 Log rotation & archival

• 🛡 Traceability via history logs

• ⚙️ Idempotent scripting logic

These are the same patterns used in:

• CI/CD pipelines

• Cron jobs

• Monitoring agents

• SRE tooling

• Security & compliance workflows

---

✅ Day 11 Summary

Today, I built:

• 🌐 A website health monitoring script

• 🧾 Daily structured logs with timestamps

• 📦 Automated log archival & cleanup

• 📜 Archival history tracking

This hands-on project strengthened my understanding of automation, monitoring, and operational reliability, which are core to modern DevSecOps workflows.

---

📌 Source Code:
👉 GitHub: https://github.com/Jysh06/website_health_logger

---

🏁 Final Thoughts

You don’t need complex tools to start learning DevOps.

Start small.
Understand deeply.
Automate everything. 🔥

If you’re also learning Linux / DevOps / SRE, feel free to connect and share feedback 🙌

Happy Learning! 🚀