From Substack Overflow to Smart Insights
How AI helped me build an article summarizer in a programming language I had never used before
Ever found yourself drowning in Substack newsletters because you couldn't keep up with the reading? I've been there. I love the in-depth analysis from my favorite Substack writers. But I sometimes finish a 15-minute read, only to realize I could have skipped it. Either the topic wasn't quite what I expected, or it covered ground I was already familiar with. Classic time sink.
At work, I've been watching the AI revolution from afar. Like many developers in regulated industries, compliance and data privacy concerns meant no ChatGPT or GitHub Copilot for me. But curiosity is a powerful motivator, and I needed a playground to experiment with these tools. So why not combine both problems? Build something useful for my Substack reading habit while getting my hands dirty with AI assisted development. The idea was straightforward: create a tool that could give me a quick preview of articles before I commit my time to them. Here's where it gets interesting: I decided to go all in on the AI experiment. Instead of sticking to my usual programming stack, I picked Python—a language I had never used before. The plan? Let the AI agent be my Python mentor and see how far AI-assisted development has come. Here's how I built it, what I learned about AI-assisted development along the way, and why the outcome is useful for other Substack readers, too.
The Development Process: When AI Becomes Your Coding Mentor
After testing various AI coding assistants - GitHub Copilot, JetBrains AI, and Windsurf - I landed on Windsurf. What sold me? It's terminal access. Executing commands through the AI chat proved to be a game-changer. Creating meaningful commit messages - all handled through the AI agent. Instead of using terminal commands like git commit -m "my commit message“, I chatted with the AI agent to commit and push my changes. Goodbye to those "what did I do last hour?" moments when trying to come up with a good commit message. It was that magic "Aha!" moment for me.
You might say this is just a small, irrelevant example. However, it was was just as easy to generate and configure ssh keys for git access or install Python dependencies with PIP via the AI agent. I wouldn’t even have known that I should use PIP for managing Python dependencies. Not sure how long I will remember git or any other terminal commands, though 🤣
With my AI agent chosen, it was time to dive in. One quick chat about Python web development, and Flask emerged as a good choice for a lightweight web framework. The initial version was simple: paste a Substack URL, get a summary. For the actual summarization, the AI suggested using OpenAI's API. It also helped me generate an API key and choose the right model for my use case. I had a small app up and running in an hour. I was impressed. Here is a short demo video of the initial version.
Encouraged by the early success, I aimed higher. Instead of manually pasting a URL to get a summary, why not fetch articles from my Substack inbox? The AI suggested using available RSS feeds—it seemed promising until we hit a wall. Turns out, Substack exposes RSS feeds only per blog, not for your personalized feed of subscriptions (your Substack Inbox). With no public Substack API available, we had to venture into web scraping territory, which was not really my first choice.
We chose the Playwright framework to log in to Substack and scrape dynamic web content. Despite never having written Python before, understanding the generated code wasn't the challenge I expected. After each code generation, I could accept the changes by reviewing a diff file. The AI agent provided a good summary explaining the implementation in the chat. The generated Python code had a lot of inline comments, explaining each block's purpose. I usually prefer clean, self-explaining code over comments. In this case, though, I appreciated these comments while learning Python. The most fascinating part? I wrote very little code myself. My role shifted to more of a code reviewer, only stepping in to fix small details the AI couldn't handle—like tweaking HTML DOM selectors for the web-scraping part. Substack's specific DOM structure was dynamically loaded, and the AI agent wasn’t able to access it. But instead of telling me that it wasn't able to identify proper DOM selectors, it hallucinated and made up new ones.
The Final Product: Your Personal Substack Time-Saver
Think of the app as Blinkist, but for your Substack articles. Instead of forcing yourself to either read every word or skip the article entirely, you get a quick, AI-generated summary that captures the key points. What surprised me most is the quality of these summaries. They're not just bullet points or crude extracts. They're intelligent distillations that help you decide whether to dive into the full article.
The best part? It's already saving me about two hours per week. That's two hours I can spend on the articles I'm genuinely interested in, deep-diving into topics that really matter to me. And depending on your inbox size, you might save more time.
Try it yourself! The code is available on GitHub, and I'd love to hear about your experience. Whether it's feedback, feature requests, or contributions to the codebase—we can make this tool better together.
Lessons Learned: The Pros and Cons of AI-Assisted Development
Let me be clear: I don't want to code without AI assistance ever again. Going back would feel like returning to the Stone Age. Here's what worked well and what didn't during my experiment:
Pros
⚡ Lightning-Fast Development
Built a working app in just a few hours using Python and Flask—both completely new to me
🎮 Fun Factor
Development experience was genuinely enjoyable, with AI providing immense support during design and execution phase🌟 Bias toward Open Source
AI agent mostly recommends free and open-source libraries and development tools
Cons
📝 Mediocre Code Quality
Better than junior-level code, but can't match mid-to-senior level craftsmanship; e.g., backend lacks modularization
🔄 Brittle Refactoring
Larger code refactoring with the AI agent led to a broken app all the time. The AI had a super loose understanding of the word "refactoring". For example, when I wanted to extract CSS and JavaScript code into separate files, the AI broke the integration with the backend. In another case, the AI changed the actual logic and removed UI fields instead of improving code quality. I refactored everything with the AI anyway, but it required extra care to review the diffs.
🧩 Hallucination
The AI agent couldn't access Substack's dynamic Inbox DOM structure without login credentials and required me to step in and provide accurate DOM selectors. Instead of telling me that it can’t come up with good selectors, it started to hallucinate. Even worse, it kept trying to change my well-designed DOM selectors while refactoring despite my instructions not to.🐛 Tool Stability
Windsurf had occasional issues applying code diffs for an hour in my case. This happened to me on two separate days and might be a bug or some throttling logic on their end (I used Windsurf Pro for 15$/month).💡 Technical Expertise Required
Recently, I heard of an awesome idea. It says that less technical folks, like product managers, could build small prototypes on their own using an AI agent. I think we are not there yet. There is still significant technical knowledge required (e.g., understanding the DOM structure of an HTML page).
One final thought: While AI is an incredible development partner, never let it be a black box. You don't want to be that person who can't explain their own code. I made it a habit to ask clarifying questions about every piece of code the AI generated. I wanted to be able to maintain the code if the AI assistant vanished. This was also a great opportunity to learn Python. Understanding what the AI does, rather than just accepting its output, made the whole experience even more enjoyable and educational. The goal isn't only to build something in a short amount of time. It's to grow as a developer and to guide the AI agent more effectively over time.
🛠️ Have an idea for a new feature? Found a bug?
Check out the code on GitHub:
https://github.com/juvodu/substack-summarizer
Whether you're a Python pro or just getting started with AI-assisted development, your contributions are welcome!
💬 And hey, let's keep the conversation going! I've recently enabled Substack's chat functionality. Drop by in Substack to share your experience with the summarizer, ask questions about AI-assisted development, or suggest topics you'd like me to cover in future posts.
