NoteBucket: Building an Offline-First Semantic Note Organizer
The all-night rabbit hole that went nowhere, the pivot that saved the project, and what I'd do differently next time.
The Spark
The idea for NoteBucket came from a simple frustration. I had notes scattered everywhere — Google Keep, local text files, a dusty Notion account I opened once and never touched again. Some folders had 200+ files I'd abandoned because organizing them was more work than creating them.
I wanted a note app that just worked. Not a "smart" AI assistant sending my data to the cloud. Not another subscription with a privacy policy I'd never read. Something that sat on my phone, understood what my notes meant, and filed them automatically. No accounts. No sync. No analytics. Just notes and the folders they belonged in.
That was the seed. A note organizer that routes by meaning, not keywords, and does it entirely on-device. The spec was simple enough to fit in one sentence, but the execution turned into a week-long sprint with more wrong turns than right ones.
"I wanted a note app that just worked. No accounts. No sync. No analytics. Just notes and the folders they belonged in."
The Wrong Path
When I started, I had this grand vision: the AI would read a note, understand it, then create a folder and name it appropriately. A fully autonomous organizer. You write, it files, and the taxonomy builds itself.
I spent the first night trying to make this work with a tiny on-device LLM. I tweaked prompts. I tried different models. I restructured the input format. I convinced myself it was almost there — one more parameter adjustment, one more prompt rewrite, one more model swap.
It was 3 AM when I finally admitted the truth: the model was too small. BGE-small is 33MB. A model capable of generating coherent folder names from scratch would be 300MB or more. That's not a prompt problem — it's a physics problem. You can't squeeze generation capability out of a model that size.
The hard lesson: When a small model fails at a task, it's tempting to blame your implementation. But sometimes the model simply lacks the capacity. Recognizing that boundary saved me from wasting days I didn't have.
I scrapped auto-generation entirely. The app would use predefined folders. The model's job shifted from "understand and generate" to "understand and match." That distinction changed everything.
The Pivot
With auto-generation off the table, I needed a new approach. My first attempt was LLM-based classification — feed the note text to a small LLM and ask it which folder it belongs to. It worked, barely. Inference was slow (multiple seconds per note), and I was uncomfortable sending user notes through an LLM pipeline on principle.
The breakthrough came when I shifted from classification to embedding. Instead of asking the model to decide, I asked it to measure. BGE-small produces a 384-dimensional embedding vector for any text. By embedding each folder name once (at creation time) and each note on save, the routing becomes a cosine similarity calculation — not a model inference.
This was faster by orders of magnitude. Embedding a note took milliseconds. Comparing 384 floats against a handful of folder embeddings was trivial. The app became instant instead of sluggish, and it never touched a network.
"Instead of asking the model to decide, I asked it to measure. That one conceptual shift turned a slow, awkward app into something that felt like magic."
The Bug That Taught Me
The embedding system worked well, but I hit a nasty edge case with hidden folders. I wanted hidden folders to still receive matching notes — you hide a folder, not its routing. So I added logic: if no visible folder scores above the threshold, check hidden folders.
This is what shipped. And it was broken.
The "Unsorted" folder was always in the pool. If "Unsorted" won, I triggered the hidden-folder fallback. But here's the flaw: if any visible folder scored above threshold — even barely — it would win immediately, skipping the hidden folder check entirely. Hidden folders never got a fair shot because visible folders had priority by design.
The fix: Hidden folders are always in the similarity pool, every time. No conditional trigger, no fallback chain. Every folder — visible or hidden — competes on equal footing. The routing is simpler, fairer, and works exactly as intended.
This bug stuck with me because it felt so obvious in hindsight. I'd over-engineered the fallback logic when I should have just included everyone in the pool. The simpler solution was the correct one all along.
How It All Clicked
Once the routing pipeline was solid, the rest of the app fell into place quickly. The architecture settled into a rhythm:
BGE Embedding Pipeline
BGE-small-en-v1.5 via llama.cpp JNI. Notes get 384-dim L2-normalized embeddings on save. Folder names are embedded once at creation. Cosine similarity does the routing.
Disambiguation Dialog
When the margin between the top two folders is 0.03 or less, the app shows a dialog instead of guessing. The user picks. This was the UI solution to the ambiguity problem — no threshold tuning could replace it.
Crash-Safe Drafts
Drafts persist to Room on every keystroke (debounced 500ms) and auto-commit after 1 minute in the background via WorkManager. I had lost one too many notes in other apps to skip this.
The disambiguation dialog deserves special mention. When I first prototyped the router, I tried to solve ambiguity with a higher threshold. That just shifted the problem: too high and everything went to Unsorted, too low and notes landed in the wrong folder. The dialog was the actual solution — not a technical one, but a product one.
It took me a while to accept that not every UX problem has a technical fix. Sometimes you just need to ask the user.
"It took me a while to accept that not every UX problem has a technical fix. Sometimes you just need to ask the user."
Room to Grow
NoteBucket works. But there are four things I'd add if I had more time:
Export / Import
Right now your notes are tied to the device. A JSON export/import flow would give users ownership portability — backup their data or move to a new phone without starting from zero.
Quick Capture Widget
The fastest way to lose a thought is to open an app and wait for it to load. A home screen widget with a single text field would lower the capture friction to zero.
Tags System
Folders are one axis of organization. Tags would add a second — cross-cutting labels that span folders. A note about a Kotlin coroutine bug could live in the 'Android' folder and carry the 'kotlin' tag simultaneously.
Folder Embedding Fine-Tuning
When the user corrects a routing decision, that signal could refine the folder embedding over time. The model learns which notes belong where, making routing more accurate with use.
Ship Fast, Ship Private
Eight days. Twenty commits. Zero network permissions. From a vague idea about semantic organization to a working app with Room persistence, on-device ML, crash-safe drafts, hidden folders, search, and attachments.
Looking back, the most important decision I made was the constraint: 100% offline, no accounts, no analytics. Every technical choice flowed from that. BGE-small instead of a larger model because it had to fit in the APK. Embedding routing instead of LLM classification because it had to run without a network. Room instead of Firebase because the data never leaves the device.
The constraints didn't limit the app — they defined it. A version with cloud sync and a 7B model would have been more capable on paper and less useful in practice.
"The constraints didn't limit the app — they defined it. A version with cloud sync and a 7B model would have been more capable on paper and less useful in practice."