We are a pretty small team, yet we still have a pretty large and expansive suite of services, dashboards, and backend data systems. When the number of services was small, auditting and logging was quite easy, but over the last few years, it’s …
I’m building tools that help scientists run their day-to-day work. I work in a heavily regulated environment, subject to both U.S. and European digital policy requirements. The scenario we face regularly is: two services talk to each other. …
Here is the second of two posts on running Dagster in ECS. The first dealt with the platform side: the Daemon, the two Webservers, the ALB and Cognito wiring, the three security groups, the Cloud Map namespace, and the IAM that lets the Daemon launch …
Here at Just-Evotec Biologics, the Data Platform team uses Dagster as our orchestration platform of choice. Many of our services are deployed to AWS ECS as Fargate or EC2 tasks. Getting Dagster onto ECS took me an embarrassing number of hours, and …
When I joined Just-Evotec Biologics (first as a data scientist, now as a data platform engineer), I inherited a data ecosystem that probably looked pretty familiar to what lots of others have dealth with: a dozen scientific instruments each …