How to build real-time pipelines that survive failures, handle late data, and run efficiently in production.