When Your Side Project Actually Needs a Database. A Build Next Stack field guide for learners shipping small projects.
Signals you crossed the spreadsheet line
Many side projects live happily in JSON files, Google Sheets, or a single SQLite file on disk. You need a dedicated database server when multiple users must write concurrently with consistent rules, when queries outgrow load entire file into memory, or when you require transactions (transfer credits, inventory counts) that CSV cannot enforce.
Warning signs you waited too long on files: duplicate keys after two tabs edit the same JSON, merge conflicts in Git every day, or search that scans thousands of lines per request. Warning signs you jumped too early: no user has opened the app yet but Postgres runs on a paid instance.
The spreadsheet line is about concurrency, integrity, and query shape—not about feeling professional.
A solo bookmark list with fifty URLs does not need Postgres. A shared classroom supply tracker where twenty students mark items taken the same afternoon does. Be honest about writers and readers, not about ambition.
Track one metric: how often you manually merge or dedupe data. Twice in a week is a signal; twice in six months is noise.
SQLite, hosted Postgres, and Firebase — when each wins
SQLite (file or embedded): perfect for single-writer apps, local-first tools, prototypes with relational queries. Zero ops. Fails when many writers hit the same file over network storage—you will see locked database errors.
Hosted Postgres (Supabase, Neon, RDS): multiple users, relational data, SQL you want to keep for years. Costs money and mental overhead (migrations, backups). Fails solo learners when they manage schema before UI—empty tables either way.
Firebase/Firestore: rapid mobile/web sync, auth bundled, real-time listeners. Document model fights complex joins; bill surprises if reads explode. Wins hackathon-speed MVPs with logged-in users across devices.
Pick the smallest store that satisfies today’s concurrency—not tomorrow’s TechCrunch spike.
SQLite on a laptop handles surprising relational queries for personal tools. Hosted Postgres shines when friends in another city hit the same API. Firebase fits when you want Google sign-in and realtime listeners without writing auth middleware first.
Price is not only dollars: ORM learning curve, backup drills, and migration files are hours you will not spend on features.
Anti-pattern: database before a single user
Installing Postgres on day one to feel like a real engineer is a common stall. Schema design without user feedback produces columns you delete in month two. Docker Compose for database + admin UI + app before a homepage exists is resume-driven development at hobby scale.
Another anti-pattern: ORM tour before CRUD. You read migration docs instead of showing a friend the feature. Rule: first persistent record can be a SQLite file with one table—promote when pain is measured, not imagined.
Resume-driven schema design produces tables like users, subscriptions, and audit_logs before anyone logs in. Start with the one entity your idea cannot live without—bookmarks, recipes, tasks—and add columns when a screen actually needs them.
Worked example: bookmark saver without overbuilding
Project: save URLs with tags, search later, sync between laptop and phone. Phase 1: local SQLite in a desktop script—prove tagging UX. Phase 2: export/import JSON for manual sync. Phase 3: only after daily use—hosted Postgres + small API + auth.
Each phase taught something: Phase 1 revealed tag sprawl; Phase 2 showed sync urgency was lower than expected; Phase 3 migration used a real export file as test fixture. Skipping to Phase 3 would have optimized sync before validating tagging rules.
Your bookmark app might stop at Phase 1 forever—that is success if it solves your problem.
Phase 2 taught that manual JSON export was enough for two devices because phone use was read-only. That observation saved a month of auth work. Promotion decisions should cite behavior, not anxiety about looking amateur.
Failure modes that waste weekends
Migration fear loop: rewriting schema nightly instead of shipping one screen. Cap schema changes to twice per week early on.
Connection pool mysticism: tuning pool sizes before traffic exists. Default pool settings on managed hosts are fine for side scale.
N+1 queries ignored until slow: fine early; log slow queries when pages feel laggy, not preemptively at zero users.
Backup theater: automated backups you never restore-test. Once a quarter, restore to a scratch instance— fifteen minutes that prevent panic later.
Index premature optimization: adding composite indexes before queries exist. Log slow requests first; index second.
Secret in connection string committed to git: rotate credentials immediately; use host-provided secret stores on deploy.
Side-by-side: same feature, three storage tiers
Feature: taggable notes with full-text search. JSON files: one file per note, grep for search—fine under two hundred notes, painful when tags need AND queries across files. SQLite: single file, SQL WHERE on tags, indexes when search slows—still one writer unless you add a sync layer. Hosted Postgres: multi-user writes, concurrent edits, backup snapshots—worth it when collaborators edit the same notebook daily.
The feature looks identical in UI; the tiers differ in merge pain and query expressiveness. Sketch your feature’s worst-case query on paper before picking storage—if the query needs joins across entities, relational stores earn their keep early.
Solo learners often stop at SQLite for years on personal tools. That is not under-engineering if backup and export scripts exist.
Five-question gate before you install anything
- Do two or more writers touch the same records in the same hour?
- Do you need atomic updates across multiple rows?
- Will queries require filters/joins across thousands of rows?
- Must data survive beyond one machine’s disk without manual export?
- Have you shipped a version with files/Sheet and hit a concrete limit?
Zero yes: stay flat file or Sheet. One to two yes: SQLite or single-user embedded. Three or more yes: hosted relational or Firebase with budget alert. Re-run the gate when user behavior changes—not when a blog post says you should scale.
When you promote tiers, export a snapshot from the old store and import into the new one in a script you can rerun. Promotion day is less scary when rollback means restoring yesterday’s file, not guessing schema state.
Keep a one-page DATA.md describing where data lives, backup schedule, and how to reset dev environment. Solo projects fail when returning after a month and forgetting which SQLite file was canonical.
Store data at the smallest tier that hurts
Start with the boring store, log the first operation that feels wrong, and promote one tier when the pain repeats three times—not when imagination says you might need Google-scale.
Share your promotion story (JSON to SQLite, SQLite to Postgres) at [email protected] — we collect real upgrade triggers for future guides.