Dagster

Deploying Dagster to AWS ECS, Part 2: Pipelines

Here is the second of two posts on running Dagster in ECS. The first dealt with the platform side: the Daemon, the two Webservers, the ALB and Cognito wiring, the three security groups, the Cloud Map namespace, and the IAM that lets the Daemon launch …

Deploying Dagster to AWS ECS, Part 1: The Platform

Here at Just-Evotec Biologics, the Data Platform team uses Dagster as our orchestration platform of choice. Many of our services are deployed to AWS ECS as Fargate or EC2 tasks. Getting Dagster onto ECS took me an embarrassing number of hours, and …