Configure GitLab container registry cleanup policies and storage management

Access project cleanup policy settings

Navigate to your GitLab project and access the container registry cleanup configuration through the project settings menu.

In your GitLab project, go to Settings > Packages and registries. Scroll down to the Cleanup policy for tags section. This is where you'll configure all cleanup rules for your project's container registry.

Configure basic cleanup policy

Set up a basic cleanup policy that removes old development and feature branch images while preserving production tags.

Enable the cleanup policy by toggling the Cleanup policy switch. Set the following basic configuration:

• Expiration interval: 7 days (removes images older than 7 days that match other criteria)
• Expiration schedule: Every day at 06:00 UTC
• Number of tags to retain: 10 (keeps the 10 most recent tags that match patterns)
• Tags with names matching this regex will expire: .* (matches all tags)
• Tags with names matching this regex will be preserved: ^(main|master|production|release-.*|v\d+\.\d+\.\d+)$

This configuration removes images older than 7 days while always preserving main, master, production, release, and semantic version tags.

Add advanced cleanup rules

Create more specific rules for different types of images in your registry to optimize storage usage.

Click Add another rule to create additional cleanup policies. Configure separate rules for different image categories:

Rule for feature branches:

• Tags matching: ^feature-.*
• Expiration interval: 3 days
• Keep N tags: 5

Rule for development images:

• Tags matching: ^(dev|develop|test)-.*
• Expiration interval: 5 days
• Keep N tags: 3

Rule for pull request images:

• Tags matching: ^pr-\d+
• Expiration interval: 1 day
• Keep N tags: 2

Configure cleanup via GitLab API

Use the GitLab API to programmatically manage cleanup policies across multiple projects for consistent configuration.

First, create a personal access token with api scope in GitLab under User Settings > Access Tokens. Then use the API to configure cleanup policies:

#!/bin/bash
GITLAB_TOKEN="your_personal_access_token"
GITLAB_URL="https://gitlab.example.com"
PROJECT_ID="123"

Get current cleanup policy
curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
     "$GITLAB_URL/api/v4/projects/$PROJECT_ID/registry/repositories"

Update cleanup policy
curl --request PUT \
     --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
     --header "Content-Type: application/json" \
     --data '{
       "container_expiration_policy_attributes": {
         "enabled": true,
         "cadence": "1d",
         "older_than": "7d",
         "keep_n": 10,
         "name_regex": ".*",
         "name_regex_keep": "^(main|master|production|release-.*|v[0-9]+\.[0-9]+\.[0-9]+)$"
       }
     }' \
     "$GITLAB_URL/api/v4/projects/$PROJECT_ID"

Set up CLI-based cleanup automation

Create a script that manages cleanup policies across multiple projects using GitLab's API for bulk operations.

Create a cleanup management script for consistent policy deployment:

#!/bin/bash

set -euo pipefail

Configuration
GITLAB_TOKEN="${GITLAB_TOKEN:-}"
GITLAB_URL="${GITLAB_URL:-https://gitlab.example.com}"
CONFIG_FILE="${CONFIG_FILE:-/opt/cleanup-config.json}"

if [[ -z "$GITLAB_TOKEN" ]]; then
    echo "Error: GITLAB_TOKEN environment variable is required"
    exit 1
fi

Function to apply cleanup policy to a project
apply_cleanup_policy() {
    local project_id="$1"
    local policy_config="$2"
    
    echo "Applying cleanup policy to project $project_id..."
    
    response=$(curl --silent --request PUT \
        --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
        --header "Content-Type: application/json" \
        --data "$policy_config" \
        "$GITLAB_URL/api/v4/projects/$project_id" || echo "")
    
    if [[ -n "$response" ]]; then
        echo "✓ Policy applied to project $project_id"
    else
        echo "✗ Failed to apply policy to project $project_id"
    fi
}

Function to get registry storage usage
get_registry_usage() {
    local project_id="$1"
    
    curl --silent --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
         "$GITLAB_URL/api/v4/projects/$project_id/registry/repositories" | \
    jq -r '.[] | "Repository: \(.path) | Size: \(.size_bytes // "unknown") bytes"'
}

Read projects and apply policies
if [[ -f "$CONFIG_FILE" ]]; then
    jq -r '.projects[] | "\(.id) \(.cleanup_policy | tostring)"' "$CONFIG_FILE" | \
    while read -r project_id policy_json; do
        apply_cleanup_policy "$project_id" "$policy_json"
        echo "Current registry usage for project $project_id:"
        get_registry_usage "$project_id"
        echo "---"
    done
else
    echo "Config file $CONFIG_FILE not found"
    exit 1
fi

Create the configuration file for your cleanup policies:

{
  "projects": [
    {
      "id": 123,
      "name": "web-application",
      "cleanup_policy": {
        "container_expiration_policy_attributes": {
          "enabled": true,
          "cadence": "1d",
          "older_than": "7d",
          "keep_n": 10,
          "name_regex": ".*",
          "name_regex_keep": "^(main|master|production|release-.*|v[0-9]+\\.[0-9]+\\.[0-9]+)$"
        }
      }
    },
    {
      "id": 456,
      "name": "api-service",
      "cleanup_policy": {
        "container_expiration_policy_attributes": {
          "enabled": true,
          "cadence": "1d",
          "older_than": "3d",
          "keep_n": 5,
          "name_regex": "^(feature-.|dev-.|test-.*)$",
          "name_regex_keep": "^(main|production)$"
        }
      }
    }
  ]
}

Make the script executable and run it:

sudo chmod +x /opt/gitlab-registry-cleanup.sh
export GITLAB_TOKEN="your_token_here"
export GITLAB_URL="https://gitlab.example.com"
/opt/gitlab-registry-cleanup.sh

Monitor registry storage usage

Set up monitoring to track storage usage and cleanup effectiveness across your GitLab container registries.

Create a monitoring script that tracks registry metrics:

#!/bin/bash

set -euo pipefail

GITLAB_TOKEN="${GITLAB_TOKEN:-}"
GITLAB_URL="${GITLAB_URL:-https://gitlab.example.com}"
METRICS_FILE="${METRICS_FILE:-/var/log/registry-metrics.log}"

Function to get project registry statistics
get_project_registry_stats() {
    local project_id="$1"
    local project_name="$2"
    
    # Get project details
    project_info=$(curl --silent --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
                        "$GITLAB_URL/api/v4/projects/$project_id")
    
    # Get registry repositories
    repositories=$(curl --silent --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
                        "$GITLAB_URL/api/v4/projects/$project_id/registry/repositories")
    
    echo "=== Project: $project_name (ID: $project_id) ==="
    echo "Last activity: $(echo "$project_info" | jq -r '.last_activity_at')"
    
    total_size=0
    repo_count=0
    
    echo "$repositories" | jq -c '.[]' | while read -r repo; do
        repo_id=$(echo "$repo" | jq -r '.id')
        repo_path=$(echo "$repo" | jq -r '.path')
        repo_size=$(echo "$repo" | jq -r '.size_bytes // 0')
        
        # Get tags for this repository
        tags=$(curl --silent --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
                    "$GITLAB_URL/api/v4/projects/$project_id/registry/repositories/$repo_id/tags" || echo '[]')
        
        tag_count=$(echo "$tags" | jq 'length')
        
        echo "Repository: $repo_path"
        echo "  Size: $repo_size bytes ($(numfmt --to=iec "$repo_size"))"
        echo "  Tags: $tag_count"
        
        if [[ "$tag_count" -gt 0 ]]; then
            echo "  Recent tags:"
            echo "$tags" | jq -r '.[:5][] | "    - \(.name) (\(.created_at))"'
        fi
        echo ""
        
        total_size=$((total_size + repo_size))
        repo_count=$((repo_count + 1))
    done
    
    echo "Total repositories: $repo_count"
    echo "Total size: $total_size bytes ($(numfmt --to=iec "$total_size"))"
    
    # Log metrics in structured format
    timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
    echo "$timestamp,$project_id,$project_name,$repo_count,$total_size" >> "$METRICS_FILE"
}

Function to generate summary report
generate_summary() {
    if [[ -f "$METRICS_FILE" ]]; then
        echo "=== Storage Usage Summary ==="
        echo "Date,Project_ID,Project_Name,Repositories,Total_Size_Bytes" | head -1
        tail -10 "$METRICS_FILE"
        
        echo ""
        echo "Top 5 projects by storage usage:"
        tail -50 "$METRICS_FILE" | sort -t',' -k5 -nr | head -5 | \
        while IFS=',' read -r date proj_id proj_name repo_count size_bytes; do
            size_human=$(numfmt --to=iec "$size_bytes")
            echo "  $proj_name (ID: $proj_id): $size_human ($repo_count repos)"
        done
    fi
}

Main execution
if [[ $# -gt 0 && "$1" == "--summary" ]]; then
    generate_summary
    exit 0
fi

Create metrics file with header if it doesn't exist
if [[ ! -f "$METRICS_FILE" ]]; then
    echo "timestamp,project_id,project_name,repository_count,total_size_bytes" > "$METRICS_FILE"
fi

Monitor specific projects (add your project IDs here)
projects=(
    "123:web-application"
    "456:api-service"
    "789:mobile-app"
)

for project in "${projects[@]}"; do
    IFS=':' read -r project_id project_name <<< "$project"
    get_project_registry_stats "$project_id" "$project_name"
    echo ""
done

generate_summary

Set up automated cleanup scheduling

Configure systemd timers to run cleanup monitoring and policy enforcement on a regular schedule.

Create a systemd service for registry monitoring:

[Unit]
Description=GitLab Container Registry Monitoring
After=network.target

[Service]
Type=oneshot
User=gitlab-registry
Group=gitlab-registry
Environment=GITLAB_TOKEN=your_token_here
Environment=GITLAB_URL=https://gitlab.example.com
Environment=METRICS_FILE=/var/log/registry-metrics.log
ExecStart=/opt/registry-monitor.sh
ExecStartPost=/opt/registry-monitor.sh --summary
StandardOutput=journal
StandardError=journal

Create the corresponding timer:

[Unit]
Description=Run GitLab Registry Monitoring Daily
Requires=gitlab-registry-monitor.service

[Timer]
OnCalendar=daily
RandomizedDelaySec=1800
Persistent=true

[Install]
WantedBy=timers.target

Create the service user and enable the timer:

sudo useradd --system --shell /bin/false --home-dir /var/lib/gitlab-registry --create-home gitlab-registry
sudo mkdir -p /var/log /var/lib/gitlab-registry
sudo chown gitlab-registry:gitlab-registry /var/log/registry-metrics.log /var/lib/gitlab-registry
sudo chmod 755 /opt/registry-monitor.sh /opt/gitlab-registry-cleanup.sh

Enable and start the timer
sudo systemctl enable --now gitlab-registry-monitor.timer
sudo systemctl status gitlab-registry-monitor.timer

Configure cleanup policy alerts

Set up alerting to notify you when cleanup policies fail or when storage usage exceeds thresholds.

Create an alerting script that checks cleanup policy effectiveness:

#!/bin/bash

set -euo pipefail

GITLAB_TOKEN="${GITLAB_TOKEN:-}"
GITLAB_URL="${GITLAB_URL:-https://gitlab.example.com}"
ALERT_EMAIL="${ALERT_EMAIL:-admin@example.com}"
STORAGE_THRESHOLD_GB="${STORAGE_THRESHOLD_GB:-50}"
METRICS_FILE="/var/log/registry-metrics.log"

Function to send alert email
send_alert() {
    local subject="$1"
    local message="$2"
    
    echo "$message" | mail -s "GitLab Registry Alert: $subject" "$ALERT_EMAIL"
    echo "Alert sent: $subject"
}

Function to check storage thresholds
check_storage_thresholds() {
    if [[ ! -f "$METRICS_FILE" ]]; then
        send_alert "Metrics Missing" "Registry metrics file not found at $METRICS_FILE"
        return 1
    fi
    
    # Get latest metrics (last 24 hours)
    yesterday=$(date -d '1 day ago' -u +"%Y-%m-%d")
    
    while IFS=',' read -r timestamp project_id project_name repo_count size_bytes; do
        # Skip header line
        [[ "$timestamp" == "timestamp" ]] && continue
        
        # Check if entry is from last 24 hours
        entry_date=$(echo "$timestamp" | cut -d'T' -f1)
        [[ "$entry_date" < "$yesterday" ]] && continue
        
        size_gb=$(echo "scale=2; $size_bytes / 1024 / 1024 / 1024" | bc)
        
        if (( $(echo "$size_gb > $STORAGE_THRESHOLD_GB" | bc -l) )); then
            alert_msg="Project '$project_name' (ID: $project_id) is using ${size_gb}GB of registry storage, which exceeds the threshold of ${STORAGE_THRESHOLD_GB}GB.
            
Current statistics:
Repositories: $repo_count
Total size: ${size_gb}GB
            
Consider reviewing cleanup policies or manually removing unused images."
            
            send_alert "High Storage Usage: $project_name" "$alert_msg"
        fi
    done < "$METRICS_FILE"
}

Function to check cleanup policy status
check_cleanup_policies() {
    local config_file="/opt/cleanup-config.json"
    
    if [[ ! -f "$config_file" ]]; then
        send_alert "Configuration Missing" "Cleanup configuration file not found at $config_file"
        return 1
    fi
    
    jq -r '.projects[].id' "$config_file" | while read -r project_id; do
        # Get current policy status
        policy_response=$(curl --silent --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
                              "$GITLAB_URL/api/v4/projects/$project_id" || echo "")
        
        if [[ -z "$policy_response" ]]; then
            send_alert "API Error" "Failed to retrieve policy status for project $project_id"
            continue
        fi
        
        # Check if cleanup policy is enabled
        policy_enabled=$(echo "$policy_response" | jq -r '.container_expiration_policy.enabled // false')
        
        if [[ "$policy_enabled" != "true" ]]; then
            project_name=$(echo "$policy_response" | jq -r '.name')
            send_alert "Policy Disabled" "Cleanup policy is disabled for project '$project_name' (ID: $project_id)"
        fi
    done
}

Main execution
echo "Running registry cleanup alerts..."
check_storage_thresholds
check_cleanup_policies
echo "Cleanup alerts completed"

Add the alerting script to your monitoring timer or create a separate one:

[Unit]
Description=GitLab Container Registry Alerts
After=network.target

[Service]
Type=oneshot
User=gitlab-registry
Group=gitlab-registry
Environment=GITLAB_TOKEN=your_token_here
Environment=GITLAB_URL=https://gitlab.example.com
Environment=ALERT_EMAIL=admin@example.com
Environment=STORAGE_THRESHOLD_GB=50
ExecStart=/opt/cleanup-alerts.sh
StandardOutput=journal
StandardError=journal

Install mail utilities and enable the alert service:

sudo apt install -y mailutils bc
sudo systemctl enable --now gitlab-registry-alerts.service

sudo dnf install -y mailx bc
sudo systemctl enable --now gitlab-registry-alerts.service