Very cool project! Out of curiosity, after reading the story section in the README, I thought of Sikuli. Would JavaScript bindings to the Sikuli API meet your desired use case, or are you looking to provide other functionality, improve performance, etc?
This looks incredible! I really hope you keep working on this project. I never liked Sikuli or AutoHotKey but I always find myself using AutoIt. Their F1 help popup is by far the best I've ever used in any software - I easily find the answer I need every time.
Its flaws: Windows only afaik, not JS. Would be really interested to see how this project evolves.