Design of an Interactive A.I. for help desks, and the Internet of things
Sean Wilson and I started a semantic search company over a decade ago. This started my foray into
intelligent systems, big data, and artificial intelligence. We left this company after eight years of hard
work. This company is still operational today and doing well.
intelligent systems, big data, and artificial intelligence. We left this company after eight years of hard
work. This company is still operational today and doing well.
I always felt that there was something missing from a search only solution. First I tried to make the
search more intelligent. I tried many different approaches. Focused on getting better Word Sense
Disambiguation (WSD) using neural networks. WSD can be thought of as being able to tell ambiguous
usage of words apart. “Jaguar”, are we talking about the car or the animal? “bank”, did they mean a
financial institution or the side of a river? This can usually be resolved from the immediate or larger
context of whatever it is you’re looking at.
search more intelligent. I tried many different approaches. Focused on getting better Word Sense
Disambiguation (WSD) using neural networks. WSD can be thought of as being able to tell ambiguous
usage of words apart. “Jaguar”, are we talking about the car or the animal? “bank”, did they mean a
financial institution or the side of a river? This can usually be resolved from the immediate or larger
context of whatever it is you’re looking at.
This only led to better information retrieval, not anything remotely intelligent, clever as it was.
“A shift in paradigm and thinking was needed”
This has been achieved with SimSage.
Diagram 1: Simsage intent neural network
We expose functions in sets. This is so that different users of Simsage can customize their interactive
experience and needs. A user inside the enterprise might want to use our system as a Q/A system with
a semantic search engine, typing on a keyboard. Whereas a user calling into the help desk over a phone
line might not want semantic search as a fall back when functions can’t be matched, but instead talk to a
human operator. Each user can be setup with a profile that selects functionality groupings we call modules.
experience and needs. A user inside the enterprise might want to use our system as a Q/A system with
a semantic search engine, typing on a keyboard. Whereas a user calling into the help desk over a phone
line might not want semantic search as a fall back when functions can’t be matched, but instead talk to a
human operator. Each user can be setup with a profile that selects functionality groupings we call modules.
Simsage is thus a platform that mixes logic with neural networks. We have deep learning neural networks
for Speech to Text (Mozilla DeepSpeech) and Google Cloud’s Speech to Text. We have our own neural
networks for Word Sense Disambiguation, Speech synthesis, and User Intent detection.
for Speech to Text (Mozilla DeepSpeech) and Google Cloud’s Speech to Text. We have our own neural
networks for Word Sense Disambiguation, Speech synthesis, and User Intent detection.
Diagram 3: Simsage Question and Answer pipeline
Different clients connect to our platform using RESTful JSON.
Diagram 4: sample web U.I. for Simsage query interaction
This interface is fully multimedia capable as well. We’ve added some smarts for importing PDF manuals
and marking up data and images in the PDF for interactivity.
and marking up data and images in the PDF for interactivity.
Diagram 5: multimedia data and context sensitive query functionality
Our pure speech to text input doesn’t have a U.I. We process continuous speech using our own threshold
algorithms for silence detection, and stream speech to Google when we hear the user talk. Feedback is
provided through speech and sounds.
algorithms for silence detection, and stream speech to Google when we hear the user talk. Feedback is
provided through speech and sounds.
Listen to the following video, running a speech to text session with Simsage using a faceless Java client.
We are well on our way and have moved beyond search. I hope this article can be of some help to other
fellow explorers.
Feel free to contact Sean or myself at rock@simsage.nz OR try the product at https://simsage.nz
fellow explorers.
Feel free to contact Sean or myself at rock@simsage.nz OR try the product at https://simsage.nz