Infrastructure tutorials

Production-grade guides for Linux, servers, security and performance. Copy-paste commands, multi-distro support, written by engineers who run this in production.

databases Advanced

Optimize DuckDB performance for large datasets with partitioning

Configure DuckDB with advanced partitioning strategies and memory optimization for processing multi-gigabyte datasets efficiently. Includes Python integration, query optimization techniques, and comprehensive monitoring setup.

45 min 4 distros 51 views
databases Intermediate

Configure DuckDB cluster setup for distributed analytics and high performance workloads

Set up a DuckDB cluster with distributed query processing, network security, and performance optimization for high-throughput analytical workloads across multiple nodes.

45 min 4 distros 31 views
databases Intermediate

Setup DuckDB with Apache Airflow for automated data pipelines

Configure DuckDB as a high-performance analytical database backend for Apache Airflow workflows. Build automated data pipelines that process files, APIs, and databases using DuckDB's columnar engine.

45 min 4 distros 50 views
monitoring Intermediate

Set up Apache Airflow performance monitoring with DataDog agent integration and custom dashboards

Configure comprehensive Apache Airflow monitoring using DataDog agent to track DAG performance, task execution metrics, and resource utilization with custom dashboards and automated alerting for production workflow management.

45 min 4 distros 120 views
devops Advanced

Implement Airflow load balancing with Celery executor for high availability workflows

Set up Apache Airflow with Celery executor, Redis cluster backend, HAProxy load balancing, and auto-scaling workers for production-grade workflow orchestration with high availability.

45 min 4 distros 113 views
devops Advanced

Configure Apache Airflow high availability with CeleryExecutor and Redis clustering for production deployments

Set up Apache Airflow with CeleryExecutor and Redis clustering for high availability production deployments. Configure multiple workers, load balancing, monitoring, and automated failover to handle enterprise-scale workflow orchestration with zero downtime.

45 min 4 distros 187 views
security Intermediate

Implement Airflow DAG security scanning with Bandit and safety checks

Set up automated security scanning for Apache Airflow DAGs using Bandit for Python code analysis and Safety for vulnerability detection. Configure pre-commit hooks and CI/CD integration for continuous security monitoring.

45 min 4 distros 79 views
devops Advanced

Configure Apache Airflow data lineage tracking with OpenLineage for comprehensive workflow observability

Set up OpenLineage with Apache Airflow to track data lineage across workflows, providing comprehensive observability into data transformations, dependencies, and quality issues in production environments.

45 min 4 distros 82 views
security Advanced

Configure Apache Airflow DAG security and secrets management with RBAC policies and encryption

Implement comprehensive security for Apache Airflow DAGs using role-based access control, HashiCorp Vault integration, and encrypted secrets management. Configure granular permissions, audit logging, and isolation policies for production workflows.

45 min 4 distros 130 views
devops Intermediate

Configure Apache Airflow DAG version control with Git and CI/CD pipelines

Set up Git-based version control for Apache Airflow DAGs with automated CI/CD pipelines for testing and deployment. Implement DAG synchronization, validation workflows, and production-ready deployment strategies.

45 min 4 distros 136 views
security Advanced

Configure Airflow DAG security and isolation with RBAC policies

Set up comprehensive security for Apache Airflow with role-based access control, DAG-level permissions, and resource isolation. Configure user authentication, implement fine-grained security policies, and establish monitoring for production-grade workflow orchestration.

45 min 4 distros 263 views
devops Advanced

Set up Apache Airflow high availability with CeleryExecutor and Redis clustering

Deploy Apache Airflow in high availability mode using CeleryExecutor with Redis clustering for task distribution, PostgreSQL connection pooling, and load-balanced webservers for production-grade workflow orchestration.

45 min 4 distros 489 views

Need help?

Don't want to manage this yourself?

We handle infrastructure for businesses that depend on uptime. From initial setup to ongoing operations.

Talk to an engineer