NoteBucket
Local-first note organizer — BGE-small embeddings + llama.cpp classify notes into folders by semantic similarity. 100% offline.
Context & Overview
Note-taking apps either rely on cloud sync, require accounts, or use keyword-based folders. I wanted something that organizes notes by meaning — not by keywords — and runs entirely on-device with zero network calls.
The Challenge
Problem Statement
Manual folder management breaks down at scale. Users either dump everything into one bucket or spend more time organizing than writing.
Why It Matters
Semantic routing removes the organizational friction entirely — you write, the app files.
Constraints
- Must run 100% offline — no cloud, no accounts, no analytics
- On-device inference only (BGE-small via llama.cpp JNI)
- Android arm64-v8a only (emulator not supported)
- Model + app must fit in a reasonable APK size
Goals & Metrics
Objectives
- Embed notes on-device and route to folders by cosine similarity
- Semantic search with folder + date range filters
- Crash-safe draft persistence
- File/image attachments
- Hidden folders + bulk operations
Target KPIs
- Routing accuracy (correct folder assignment)
- Inference latency per note
- APK size with bundled model
Research & Discovery
Methodology
Key Insights
- BGE-small gives 384-dim embeddings at ~33MB — small enough to bundle in APK, accurate enough for folder routing
- LLM classification was too slow and privacy-violating for an offline-first app
- Ambiguous notes (margin ≤ 0.03 between top-2 folders) need a disambiguation dialog, not a guess
Approach & Strategy
Spike-first: prove BGE + llama.cpp JNI works on Android in day 1, then build the full app around it. Every feature is additive from the core embedding pipeline.
Frameworks
Jetpack Compose + Material 3 (UI), Room (persistence: folders, notes, drafts, attachments, embeddings as BLOB), Hilt (DI), WorkManager + ProcessLifecycleObserver (background draft commits), Coil (image loading), llama.cpp via NDK + CMake (inference runtime)
Collaboration
Solo. PRD-driven development — architecture decisions locked before code.
The Solution
A standalone Android app where notes are embedded on-device (BGE-small-en-v1.5, 384-dim, L2-normalized), cosine-scored against folder name embeddings, and filed automatically. Ambiguous matches show a disambiguation dialog. Unmatched notes go to 'Unsorted'. Drafts persist to Room on every keystroke and auto-commit after 1 minute in background.
App Visuals
Design Rationale
Kotlin + Jetpack Compose for native Android feel. Room for structured persistence (embeddings stored as BLOB). llama.cpp via JNI for on-device inference — no cloud dependency. Hilt for DI. WorkManager for reliable background draft commits.
Execution
Roadmap
Day 1
Spike: BGE-small + llama.cpp JNI scaffold on Android
Day 2
Full app: pages, onboarding, Room persistence, folder/note CRUD
Days 3-4
BGE routing, UI overhaul (dark mode, folder colors, notion-style editor)
Day 5
Search with filters, note input UX, disambiguation dialog, bulk ops
Days 6-7
Hidden folders, settings, UI polish, keyboard fixes
Day 8
v0.3.0: onboarding flow, voice removal, MIT license
Challenges Overcome
- ⚠ Replacing LLM classifier with BGE embedding routing mid-development required rewriting the core sort pipeline
- ⚠ JNI bridge between Kotlin and llama.cpp needed careful memory management for 384-dim float arrays
- ⚠ Draft auto-commit on background required ProcessLifecycleObserver + WorkManager coordination to avoid data loss
- ⚠ Disambiguation dialog when cosine margin ≤ 0.03 — too ambiguous to auto-route, too close to ignore
Outcomes & Impact
Quantifiable Results
- 1.20 commits across 8 days (spike → v0.3.0 release)
- 2.4 Room tables: folders, notes, drafts, attachments
- 3.33MB BGE model bundled in APK assets
- 4.Zero network permissions — fully offline
Qualitative Feedback
“Notes route to the correct folder without manual organization”
“Semantic search finds notes by meaning, not keywords”
“Draft system is crash-safe — no lost work”
Learnings & Future
Key Takeaways
- Embedding-based routing is faster and more private than LLM classification for this use case
- On-device ML is viable when you pick the right model size (33MB BGE-small, not a 7B LLM)
- Ambiguity needs a UI solution (disambiguation dialog), not a higher threshold
Next Steps
- Export/import notes across devices
- Widget for quick note capture
- Tags system alongside folder routing
- Folder embedding fine-tuning from user corrections