Data engineer who loves to draw workflows.

One thing about me: I am in my era of spotting patterns by trial and error, clocking the pattern, and writing it down.

Hand-drawn workflow: trial and error, then clock the pattern, then write it down or draw it. A red loop says try again.trial and errorclock the patternwrite it downor draw ittry againhow I learn
Project briefYou are asked to design a pipeline that pullspayments from Stripe's API into raw/payments/ inshop-data-lake, with a 3-day lookback window to pickup refunds.draw it out firstOUTBOUNDStripe APIpayments, last 3 dayspaymentsLambdanightly, 22:00S3 BUCKET · 28 OCTOBER1write, 26 Octversion 1raw/payments/2026-10-26.json2refunds, 27 Octversion 23refunds, 28 Octversion 3Lifecycle ruleNoncurrentDays 30S3 BUCKET · FROM 28 NOVEMBERversion 3version 2, deleted 28 Novversion 1, deleted 27 Novraw/payments/2026-10-26.jsonAWS: "To determine the number of days an object is noncurrent, Amazon S3 looks at when its successor was created."Project briefYou are asked to design a pipeline thatpulls payments from Stripe's API intoraw/payments/ in shop-data-lake, with a3-day lookback window to pick up refunds.draw itout firstOUTBOUNDStripe APIpayments, last 3 dayspaymentsLambdanightly, 22:00S3 BUCKET · 28 OCTOBER1write, 26 Octversion 1raw/payments/2026-10-26.json2refunds, 27 Octversion 23refunds, 28 Octversion 3Lifecycle ruleNoncurrentDays 30S3 BUCKET · FROM 28 NOVEMBERversion 3version 2,deleted 28 Novversion 1,deleted 27 Novraw/payments/2026-10-26.jsonAWS: "To determine the number of days an object is noncurrent,Amazon S3 looks at when its successor was created."

Next episode in

Season 1, Episode 6

days
hours
minutes
seconds
Countdown loading