NoteBucket
Local-first Android note organizer — BGE-small embeddings + llama.cpp route notes into folders by semantic similarity. 100% offline.
Context & Overview
Note-taking apps rely on cloud sync, require accounts, or depend on keyword-based folders. NoteBucket organizes notes by meaning rather than keywords and runs entirely on-device with zero network calls.
The Challenge
Problem Statement
Manual folder management breaks down at scale: notes pile into a single bucket, or organizing consumes more time than writing.
Why It Matters
Semantic routing removes organizational friction entirely — the user writes; the app files. No manual step remains between capturing a note and storing it correctly.
Constraints
- 100% offline — no cloud, no accounts, no analytics
- On-device inference only (BGE-small via llama.cpp JNI)
- Android arm64-v8a only (emulator not supported)
- Model + app must fit a reasonable APK size
Goals & Metrics
Objectives
- Embed notes on-device and route to folders by cosine similarity
- Semantic search with folder + date range filters
- Crash-safe draft persistence
- File/image attachments
- Hidden folders + bulk operations
Target KPIs
- Routing accuracy (correct folder assignment)
- Inference latency per note
- APK size with bundled model
Research & Discovery
Methodology
Key Insights
- BGE-small delivers 384-dim embeddings at ~33MB — small enough to bundle in the APK, accurate enough for folder routing
- LLM classification was too slow and privacy-incompatible for an offline-first app
- Ambiguous notes (margin ≤ 0.03 between top-2 folders) call for a disambiguation dialog, not a guess
Approach & Strategy
Spike-first development: prove BGE + llama.cpp JNI on Android on day 1, then build the full app around it. Every feature is additive to the core embedding pipeline.
Frameworks
Jetpack Compose + Material 3 (UI), Room (persistence: folders, notes, drafts, attachments, embeddings as BLOB), Hilt (DI), WorkManager + ProcessLifecycleObserver (background draft commits), Coil (image loading), llama.cpp via NDK + CMake (inference runtime)
Collaboration
Solo. PRD-driven — architecture decisions were locked before implementation began.
The Solution
A standalone Android app that embeds each note on-device (BGE-small-en-v1.5, 384-dim, L2-normalized), cosine-scores it against folder name embeddings, and files it automatically. Ambiguous matches surface a disambiguation dialog; unmatched notes land in 'Unsorted'. Drafts persist to Room on every keystroke and auto-commit after 1 minute in the background.
App Visuals
Design Rationale
Kotlin + Jetpack Compose for a native Android experience. Room for structured persistence with embeddings stored as BLOBs. llama.cpp over JNI for on-device inference with no cloud dependency. Hilt for DI; WorkManager for reliable background draft commits.
Execution
Roadmap
Day 1
Spike: BGE-small + llama.cpp JNI scaffold on Android
Day 2
Full app: pages, onboarding, Room persistence, folder/note CRUD
Days 3-4
BGE routing, UI overhaul (dark mode, folder colors, notion-style editor)
Day 5
Search with filters, note input UX, disambiguation dialog, bulk ops
Days 6-7
Hidden folders, settings, UI polish, keyboard fixes
Day 8
v0.3.0: onboarding flow, voice removal, MIT license
Challenges Overcome
- ⚠ Replacing the LLM classifier with BGE embedding routing mid-development required rewriting the core sort pipeline
- ⚠ The JNI bridge between Kotlin and llama.cpp required careful memory management for 384-dim float arrays
- ⚠ Background draft auto-commit demanded ProcessLifecycleObserver + WorkManager coordination to avoid data loss
- ⚠ Disambiguation at cosine margin ≤ 0.03 — too ambiguous to auto-route, too close to ignore
Outcomes & Impact
Quantifiable Results
- 1.20 commits across 8 days (spike → v0.3.0 release)
- 2.4 Room tables: folders, notes, drafts, attachments
- 3.33MB BGE model bundled in APK assets
- 4.Zero network permissions — fully offline
Qualitative Feedback
“Notes route to the correct folder without manual organization”
“Semantic search finds notes by meaning, not keywords”
“The draft system is crash-safe — no lost work”
Learnings & Future
Key Takeaways
- Embedding-based routing is faster and more privacy-preserving than LLM classification for this use case
- On-device ML is viable at the right model size (33MB BGE-small, not a 7B LLM)
- Ambiguity calls for a UI solution (disambiguation dialog), not a higher threshold
Next Steps
- Export/import notes across devices
- Widget for quick note capture
- Tags system alongside folder routing
- Folder embedding fine-tuning from user corrections