Virtual assistants, providing a voice interface to web services and IoTs, can potentially develop into monopolistic platforms that threaten consumer privacy and open competition. This talk presents Almond as an open-source alternative.

We introduce a more cost-effective and robust methodology and tool set for building virtual assistants. We use an end-to-end neural model that translates natural language to directly executable code in our new virtual assistant programming language, called ThingTalk. As annotating real data training is prohibitively expensive, we train our neural network with mostly data synthesized with our Genie data engineering tool. Genie can automatically generate a dialogue agent from a database in given domains. In addition, it is designed to be extensible: developers can supply new domain schemas and APIs, domain-independent sentence templates and dialogue state machines in any language, as well as new ThingTalk constructs. Using this methodology, Almond has been shown to answer complex questions more accurately than existing commercial assistants in the domain of restaurant reservations.

To protect privacy, our Almond virtual assistant prototype has a federated design that lets users own their data and share their digital assets with full control. Almond has been integrated into Home Assistant, an open-source local gateway that can connect to hundreds of different IoT devices. By making all the tools, neural models, training data open-source, we wish to support open-world collaboration to create the smartest and open virtual assistant. Almond and more information are available on http://oval.cs.stanford.edu.

This research is supported by the NSF under grant No. 1900638.