Augmenting human computer interactions throughput with cognitive assistance.

Hypothesis: The insertion and synchronization of an adaptive cognitive assistant in the rapid visual/motor feedback loops involved in computer-based interactions will enable noticeably more efficient human computer interactions. Humans will be willing and capable to co-adapt with their personalized cognitive assistants toward more efficient co-discovered latent languages, enabling them to achieve high-level tasks (such as writing email or coding) faster and more reliably.

Examples:

• Intuitive and adaptive auto-complete for writing plain english. Possible synchronization of a neural net with our own muscle memory.
• Sentence generator based on context and a few input keywords.
• Predictive window management that anticipates upcoming user actions.

Metrics:

• Time and/or number of basic interactions (keystrokes, clicks, reading time) required to achieve high-level tasks.
• Training effort needed to experience tangible gains from using the system.

Exploratory tasks

• [x] LSTM char-level language model on Gmail sent emails
• [x] UI/UX for rapid (instantaneous visual feedback at cursor position) auto-complete while writing plain English
• [ ] Deeper LSTM on 1b word dataset
• [ ] Transformer for word level language model and full sentence autocomplete
• [ ] iGAN applied to written sentences manifold
• [ ] Think about mixing ML based auto-complete with more conventional dictionary based systems