Monday, January 23, 2017

Natural Language Processing

Natural Language Processing (NLP) is a technology that Gartner reported as a mature and usable technology as early as 2014. Since then, the focus has been on vertical integration. Products like the Amazon Alexa do a remarkable job interpreting spoken phrases and delivering value through skills.  

An area of slower advancement is (and therefore opportunity!): Natural Language Question and Answering (NLQA)

NLQA is currently in the dreaded Trough of Disillusionment (i.e., not meeting user expectations). Vendors like ThoughtSpot are defining the market but so far no one has delivered on the promise (think Hal in 2001: A Space Odyssey ♜ 😎)

The technology behind NLP is mature. If you studied it in school be prepared for a bit of a shock. The days of building and connecting your own parsers, lemma tools and Named Entity Recognizers (NER) are gone.  NLP-as-a-Service (NaaS) is here. Driven by machine learning, modern NaaS easily beats the best academic products, like the venerable Stanford NLP Parser, that were state-of-the-art just a few years ago.

If you want to jump to the head of the NLP class, consider using services like the Google Cloud Natural Language API.  It works remarkably well without custom training for both text ingestion and question parsing. It implements several high level functions that help developers execute commands and answer questions with greater accuracy and precision. One of its best features: dependency graphs that link verb (action) and noun (thing) phrases to create powerful interpretations without custom programming.

Here is an example of dependency parsing using the Google API:



The moral: NLP is more mature than you might think. Start at the top using NLP services and you can meet NLQA challenges faster than you (and your boss) may have thought possible.

No comments:

Post a Comment