×

👋 Hello, Anthropic team!

Thanks for checking out my blog. I'm excited about the opportunity to work with you on building safe, beneficial AI systems.

Feel free to explore the posts on AI alignment, verification theory, and software engineering.

— James

Today I Learned

AWS DMS for Simple CDC Pipelines

AWS DMS (Database Migration Service) is often overlooked as a CDC tool because of its migration-focused marketing, but it's actually excellent for simple change data capture pipelines.

Key advantages for CDC use cases:

  1. Native Aurora integration - Built-in support for Aurora Postgres with minimal configuration
  2. Managed service - No infrastructure to maintain, automatic scaling
  3. Direct Lambda integration - Can invoke Lambda functions directly for each change event
  4. Built-in transformation - Supports light transformations during replication
  5. Low operational overhead - Significantly simpler than Debezium/Kafka setups

Simple CDC pipeline pattern:

Aurora → DMS (CDC mode) → Lambda → Target System

When to use:

  • Internal tools with moderate scale requirements
  • Need for simple, maintainable solutions
  • Teams with limited infrastructure support
  • POCs that need to scale gracefully

When NOT to use:

  • Need complex event routing/filtering
  • Require exactly-once delivery guarantees
  • Multi-region event distribution
  • Sub-second latency requirements