Introduction to Natural Language Processing | NLP Tutorial | AI ML Training | Type of Ambiguity

Human Language / Natural Language

Today more than 7000+ varieties language are spoken all..

over the world. Which are driven by their own rules (grammar)...

What is Text Mining ?

Text Mining / Text Analytics is the process of deriving meaningful information from natural language text for further analysis using NLP.

Text mining usually involves the process of structuring the input text, deriving patterns within the structured data, and finally evaluating and interpreting the output.

Need of Text Mining

With the advancement of technology most of the data (approx. 85%) is in unstructured textual form

Better techniques and algorithms are required to extract useful and interesting information from the large amount of generated textual data.

Hence, the area of text mining and information extraction came into picture and helps for extraction interesting and useful information.

What is Natural Language Processing (NLP) ?

"A part of AI which deals with understanding human language / natural language by a program".

Natural language processing (NLP) is a subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of natural language data. 

The result is a computer capable of "understanding" the contents of documents, including the contextual nuances of the language within them.

The technology can then accurately extract information and insights contained in the documents as well as categorize and organize the documents themselves. 

History of Natural Language Processing  

1940s - 1950s : Foundations 

• Development of formal language theory (Chomsky, Backus, Naur, Kleene)
• Probabilities and information (Shannon)

1957 - 1970s :

• Use of formal grammars as basis for natural language processing ( Chomsky, Kaplan)
• Use of logic and logic based programming (Minsky, Winogard, Colmerauer, Kay)

1970s - 1983 :

• Probabilistic methods for early speech recognition (Jelinek, Mereer)
• Discourse modeling (Grosz, Sidner, Hobbs)

1983 - 1993 :

• Finite state models (morphology) (Kaplan, Kay)

1193 - present :

• Strong integration of different technique, different areas.

Demand for Natural Language Processing 

Text Extraction 

The world puts out as much as 2.5 exabytes of data every day which means 90 years of worth high definition videos or 530,000,000 songs or 150,000,000 iPhones.

Companies are frantically looking for effective means of mining the gold rush of data. NLP is one of the most used ways.

Applications of Natural Language Processing 

Type of Ambiguity 

Lexical Ambiguity :

• Presence of two  or more possible meanings for a single word. 
• It's also called semantic ambiguity or homonymy.

Eg:  She is looking for a match.

       The fisherman went to the bank. 

Syntactic Ambiguity :

• Presence of two or more possible meanings within a single sentence or sequence of words.
• Also called structural ambiguity or grammatical ambiguity.

Eg. 

I saw the man with the binoculars.

(Ambiguity: Who has the binoculars me or him??)

I watched her duck

(Ambiguity: Does she have a duck that I watched or did she duck out of the way?)

 Referential Ambiguity :

Occurs when a word or phrase, in the context of a particular sentence, could refer to two r more properties or things.

Eg: She is a big opera star

      (Ambiguity: it is not clear if big means fat or famous)