Computers are great at working with standardized and structured data like database tables and financial records. They are able to process that data much faster than we humans can. But we humans don’t communicate in “structured data” nor do we speak binary! We communicate using words, a form of unstructured data.
Natural Language Processing
Natural Language Learning is a burgeoning field that seeks to bridge the gap between computers and human language. However, computers struggle with unstructured data due to the absence of standardized processing techniques. Defining concrete rules for computers to handle such data is challenging. Fortunately, advancements in Natural Language Learning are changing this landscape. These innovations enable computers to comprehend, analyze, and derive meaning from unstructured data more effectively. This opens up exciting possibilities for leveraging unstructured data, including pattern discovery, prediction, and generating valuable insights from Big Data.
Natural language learning aims to bridge this gap by developing algorithms and models that enable computers to understand and interpret unstructured data, particularly human language. Through techniques such as natural language processing (NLP) and machine learning, computers can learn to analyze and extract meaning from text, speech, and other forms of unstructured data. By leveraging advanced algorithms and models, natural language learning enables computers to comprehend context, identify patterns, and derive insights from unstructured data.
Human vs Computer Understanding of Language
Humans have been writing things down for thousands of years. Over that time, our brain has gained a tremendous amount of experience in understanding natural language. When we read something written on a piece of paper or in a blog post on the internet, we understand what that thing really means in the real world. We feel the emotions that reading that thing elicits and we often visualize how that thing would look in real life. Natural Language Processing (NLP) is a sub-field of Artificial Intelligence that is focused on enabling computers to understand and process human languages, to get computers closer to a human-level understanding of language. Computers don’t yet have the same intuitive understanding of natural language that humans do. They can’t really understand what the language is really trying to say. In a nutshell, a computer can’t read between the lines.
That being said, recent advances in Machine Learning (ML) have enabled computers to do quite a lot of useful things with natural language! Deep Learning has enabled us to write programs to perform things like language translation, semantic understanding, and text summarization. All of these things add real-world value, making it easy for you to understand and perform computations on large blocks of text without manual effort.
The reason why NLP is hard
The process of reading and understanding language is far more complex than it seems at first glance. There are many things that go into truly understanding what a piece of text means in the real world. For example, what do you think the following piece of text means? “Steph Curry was on fire last nice. He totally destroyed the other team”
To a human, it’s probably quite obvious what this sentence means. We know Steph Curry is a basketball player, or even if you don’t we know that he plays on some kind of team, probably a sports team. When we see “on fire” and “destroyed” we know that it means Steph Curry played really well last night and beat the other team. Computers tend to take things a bit too literally. Viewing things literally like a computer, we would see “Steph Curry” and based on the capitalization assume it’s a person, place, or otherwise important thing which is great! But then we see that Steph Curry “was on fire”…. A computer might tell you that someone literally lit Steph Curry on fire yesterday! … yikes. After that, the computer might say that Mr. Curry has physically destroyed the other team…. they no longer exist according to this computer… great…
