Welcome to System Design Saturday
Stop memorizing the syntax.
In 2020, you could build a career just by being really good at SQL. You were the "Data Guy." People asked you for a CSV, you wrote a SELECT statement, and you got paid $150k.
In 2026, that job is gone. AI writes the SQL faster than you. AI writes the Python boilerplate faster than you.
If your value proposition is "I know the syntax," you are obsolete. But if your value proposition is "I know how to glue the systems together," you are invaluable.
Welcome to the era of the Full-Stack Data Engineer.
1. THE CONCEPT: The Glue is the Gold
The market has shifted. Companies don't want a "Dbt Developer" or a "Spark Specialist." They want an Engineer who can solve business problems with data.
This requires a "T-Shaped" skill set.
Horizontal (The Breadth): You understand the entire lifecycle. You know how the API (Software Eng) sends data to the Stream (Data Eng) which feeds the Model (AI Eng).
Vertical (The Depth): You are dangerous in one specific area (e.g., Vector Search or Rust).
The 2026 Stack looks like this:
Ingest: Not just ETL. You need to know Webhooks, APIs, and Event-Driven systems (Lambda).
The Pivot: Stop calling yourself a "BI Developer." Start building applications that use data.
2. THE ARCHITECTURE: The "Platform" Mindset
The best engineers I know don't build pipelines; they build Platforms.
Junior: "I built a script to move data from A to B."
Senior: "I built a self-service framework so the Marketing team can move data from A to B without calling me."
Sunday Homework: Look at your current project. Are you building a process (a script) or a product (a reusable tool)? Shift to the product mindset. That is how you survive the AI automation wave.
3. THE CEREBRAL GYM: Solutions & Whiteboarding
Yesterday's Solution (The Git Disaster)
The Challenge: You committed AWS keys. You ran git rm, but the keys were still stolen. Why? The Answer: git rm only removes the file from the current working directory (HEAD). The file still exists in the hidden .git history folder (so you can revert to it). Hackers scan the entire history. The Fix: You must rewrite the history tree.
The Old Way:
git filter-branch(Slow, dangerous).The Modern Way: BFG Repo-Cleaner or
git filter-repo.bfg --delete-files secrets.jsonThen
git push --force.
Today's Puzzle (Rate Limiting)
System Design Saturday.
The Scenario: You are building a public API for your LLM app. You want to limit users to 100 requests per minute. You use a simple counter in Redis: INCR user_id. If > 100, reject. You expire the key every 60 seconds.
The Attack: A user sends 100 requests at 09:00:59. (Allowed). The user sends another 100 requests at 09:01:01. (Allowed). The Failure: In that 2-second window, they hit your server with 200 requests, potentially crashing it.
The Question: This is the "Fixed Window" flaw. What is the name of the algorithm (often used by Nginx/Stripe) that smoothes this out using a "drip" concept?
(Reply with the algorithm name!)
4. THE PULSE: RESOURCE OF THE WEEK
Since we are talking about the "New Data Stack," I highly recommend the book "Fundamentals of Data Engineering" by Joe Reis and Matt Housley.
It is the only book that ignores the "Hype" (tools change every year) and focuses on the "Physics" (concepts that never change). If you haven't read it, buy it this weekend.
5. THE LATENT SPACE
"A complex system that works is invariably found to have evolved from a simple system that worked."
Don't start by building a Kubernetes cluster with a multi-hop RAG agent. Start with a Python script and a SQLite file. Make it work. Then make it complex.
Enjoy your weekend.
See you tomorrow.
Harsh Kathiriya - Query & Context