Deep Learning: Natural Language Processing in Python with Recursive Neural Networks: Recursive Neural (Tensor) Networks in Theano (Deep Learning and Natural Language Processing Book 3)
The first 2 books in this series focused on word embeddings using 2 novel techniques: Word2Vec and GLoVe.In this book, we return to a classic NLP problem: sentiment analysis. Classification performance on the sentiment analysis task had plateaued for many years, due to not being able to handle negation, which is essentially because existing models failed to account for the structure of language.The bag-of-words vectors for “I love this movie”, “I don’t love this movie”, and “Don’t you love this movie” are very similar.In this book, we return to the fundamentals of language - the parse tree - and structure our neural networks to mirror the tree.It makes sense that a neural network created to classify language would have the same structure as language.These neural networks are called “recursive neural networks” and I will show you how they work both mathematically and with a full implementation in Theano.A naive solution to recursive neural networks would be to use recursion to implement them. This is however, a very poor solution because both Theano and TensorFlow require you to compile a graph of the neural network. If every sentence is a different tree, then every sentence will require a different neural network graph, which would be very inefficient for both Theano or TensorFlow to compute. No one would blame you for attempting this solution first. In fact, I will demonstrate why it’s bad by having you run code that implements it.Once you understand why recursion is not ideal for recursive neural networks, I will show you a “trick” that will help you implement them more efficiently. We will then run the recursive neural net on our sentiment analysis data and achieve state-of-the-art performance.Finally, we discuss a modification to the vanilla recursive neural network called the recursive neural tensor network or RNTN. It was invented by the guys at Stanford, who have created and published many NLP tools throughout the years that are now considered standard.I will show you how the model is structured mathematically and then I will show you how to implement it in Theano. You’ll see that it’s just a simple modification to our recursive neural network.Amazingly, all the technologies we discuss in this book can be downloaded and installed for FREE. That means all you need to invest after purchasing this book is your effort and your time. The only prerequisites are that you are comfortable with Python , Numpy, and Theano coding and you know the basics of deep learning.“Hold up... what’s deep learning and all this other crazy stuff you’re talking about?”If you are completely new to deep learning, you might want to check out my earlier books and courses on the subject, since they are required in order to understand this book. Just go to my profile and look for "Deep Learning in Python", and "Deep Learning in Python Prerequisities".Much like how IBM’s Deep Blue beat world champion chess player Garry Kasparov in 1996, Google’s AlphaGo recently made headlines when it beat world champion Lee Sedol in March 2016.What was amazing about this win was that experts in the field didn’t think it would happen for another 10 years. The search space of Go is much larger than that of chess, meaning that existing techniques for playing games with artificial intelligence were infeasible. Deep learning was the technique that enabled AlphaGo to correctly predict the outcome of its moves and defeat the world champion.Deep learning progress has accelerated in recent years due to more processing power (see: Tensor Processing Unit or TPU), larger datasets, and new algorithms like the ones discussed in this book.
Publication date: 08/20/2016Kindle book details: Kindle Edition, 50 pages
Markov Models are all about learning sequences.A lot of the data that would be very useful for us to model is in sequences. Stock prices are sequences of prices. Language is a sequence of words. Credit scoring involves sequences of borrowing and repaying money, and we can use those sequences to predict whether or not you’re going to default. In short, sequences are everywhere.The easiest way to appreciate the kind of information you get from a sequence is to consider what you are reading right now. If I had printed the previous sentence backwards, it wouldn’t make much sense to you, even though it contained all the same words. So order is important.While the current fad in deep learning is to use recurrent neural networks (LSTM) to model sequences, I want to first introduce you guys to a machine learning algorithm that has been around for several decades now - the Markov Model.This book follows directly from my first course in Unsupervised Machine Learning for Cluster Analysis, where you learned how to measure the probability distribution of a random variable. In this course, you’ll learn to measure the probability distribution of a sequence of random variables. This course is also going to go through the many practical applications of Markov models. We’re going to look at a model of sickness and health, and calculate how to predict how long you’ll stay sick, if you get sick. We’re going to talk about how Markov models can be used to analyze how people interact with your website, and fix problem areas like high bounce rate, which could be affecting your SEO. We’ll build language models that can be used to identify a writer and even generate text - imagine a machine doing your writing for you.We’ll look at what is possibly the most recent and prolific application of Markov models - Google’s PageRank algorithm. It is surprising that the world's leading search engine could have made its money on what is essentially an undergraduate linear algebra problem.If you think Markov models aren't relevant to your life, think again. Even smartphone autosuggestions can be programmed using Markov models.Amazingly, all the technologies we discuss in this book can be downloaded and installed for FREE. That means all you need to invest after purchasing this book is your effort and your time. The only prerequisites are that you are comfortable with Python and the Numpy stack and you know the basics of probability.
Publication date: 09/24/2016Kindle book details: Kindle Edition, 34 pages
Deep Learning in Python Prerequisites: Master Data Science and Machine Learning with Linear Regression and Logistic Regression in Python (Machine Learning in Python)
Do you find deep learning difficult?So you want to learn about deep learning and neural networks, but you don’t have a clue what machine learning even is. This book is for you.Perhaps you’ve already tried to read some tutorials about deep learning, and were just left scratching your head because you did not understand any of it. This book is for you.Believe the hype. Deep learning is making waves. At the time of this writing (March 2016), Google’s AlghaGo program just beat 9-dan professional Go player Lee Sedol at the game of Go, a Chinese board game.Experts in the field of Artificial Intelligence thought we were 10 years away from achieving a victory against a top professional Go player, but progress seems to have accelerated!While deep learning is a complex subject, it is not any more difficult to learn than any other machine learning algorithm. I wrote this book to introduce you to the prerequisites of neural networks, so that learning about neural networks in the future will seem like a natural extension of these topics. You will get along fine with undergraduate-level math and programming skill.All the materials in this book can be downloaded and installed for free. We will use the Python programming language, along with the numerical computing library Numpy.Unlike other machine learning algorithms, deep learning is particularly powerful because it automatically learns features. That means you don’t need to spend your time trying to come up with and test “kernels” or “interaction effects” - something only statisticians love to do. Instead, we will eventually let the neural network learn these things for us. Each layer of the neural network is made up of logistic regression units.Do you want a gentle introduction to this “dark art”, with practical code examples that you can try right away and apply to your own data? Then this book is for you.This book was designed to contain all the prerequisite information you need for my next book, Deep Learning in Python: Master Data Science and Machine Learning with Modern Neural Networks written in Python, Theano, and TensorFlow.There are many techniques that you should be comfortable with before diving into deep learning. For example, the “backpropagation” algorithm is just gradient descent, which is the same technique that is used to solve logistic regression.The error functions and output functions of a neural network are exactly the same as those used in linear regression and logistic regression. The training process is nearly identical. Thus, learning about linear regression and logistic regression before you embark on your deep learning journey will make things much, much simpler for you.Required resourcesFollowing this book does not require any external materials. Everything needed (Python, and some Python libraries) can be obtained for free.
Publication date: 03/19/2016Kindle book details: Kindle Edition, 31 pages
Convolutional Neural Networks in Python: Master Data Science and Machine Learning with Modern Deep Learning in Python, Theano, and TensorFlow (Machine Learning in Python)
This is the 3rd part in my Data Science and Machine Learning series on Deep Learning in Python. At this point, you already know a lot about neural networks and deep learning, including not just the basics like backpropagation, but how to improve it using modern techniques like momentum and adaptive learning rates. You've already written deep neural networks in Theano and TensorFlow, and you know how to run code using the GPU.This book is all about how to use deep learning for computer vision using convolutional neural networks. These are the state of the art when it comes to image classification and they beat vanilla deep networks at tasks like MNIST.In this course we are going to up the ante and look at the StreetView House Number (SVHN) dataset - which uses larger color images at various angles - so things are going to get tougher both computationally and in terms of the difficulty of the classification task. But we will show that convolutional neural networks, or CNNs, are capable of handling the challenge!Because convolution is such a central part of this type of neural network, we are going to go in-depth on this topic. It has more applications than you might imagine, such as modeling artificial organs like the pancreas and the heart. I'm going to show you how to build convolutional filters that can be applied to audio, like the echo effect, and I'm going to show you how to build filters for image effects, like the Gaussian blur and edge detection.After describing the architecture of a convolutional neural network, we will jump straight into code, and I will show you how to extend the deep neural networks we built last time with just a few new functions to turn them into CNNs. We will then test their performance and show how convolutional neural networks written in both Theano and TensorFlow can outperform the accuracy of a plain neural network on the StreetView House Number dataset.All the materials used in this book are FREE. You can download and install Python, Numpy, Scipy, Theano, and TensorFlow with pip or easy_install.Lastly, my goal is to show you that convolutional networks aren’t magical and they don’t require expert-level math to figure out.It’s just the same thing we had with regular neural networks:y = softmax( relu(X.dot(W1).dot(W2) )Except we replace the first “dot product” with a convolution:y = softmax( relu(conv(X, W1)).dot(W2) )The way they are trained is exactly the same as before, so all your skills with backpropagation, etc. carry over.
Publication date: 05/15/2016Kindle book details: Kindle Edition, 41 pages
Unsupervised Machine Learning in Python: Master Data Science and Machine Learning with Cluster Analysis, Gaussian Mixture Models, and Principal Components Analysis
In a real-world environment, you can imagine that a robot or an artificial intelligence won’t always have access to the optimal answer, or maybe there isn’t an optimal correct answer. You’d want that robot to be able to explore the world on its own, and learn things just by looking for patterns.Think about the large amounts of data being collected today, by the likes of the NSA, Google, and other organizations. No human could possibly sift through all that data manually. It was reported recently in the Washington Post and Wall Street Journal that the National Security Agency collects so much surveillance data, it is no longer effective.Could automated pattern discovery solve this problem?Do you ever wonder how we get the data that we use in our supervised machine learning algorithms?Kaggle always seems to provide us with a nice CSV, complete with Xs and corresponding Ys.If you haven’t been involved in acquiring data yourself, you might not have thought about this, but someone has to make this data!A lot of the time this involves manual labor. Sometimes, you don’t have access to the correct information or it is infeasible or costly to acquire.You still want to have some idea of the structure of the data.This is where unsupervised machine learning comes into play.In this book we are first going to talk about clustering. This is where instead of training on labels, we try to create our own labels. We’ll do this by grouping together data that looks alike.The 2 methods of clustering we’ll talk about: k-means clustering and hierarchical clustering.Next, because in machine learning we like to talk about probability distributions, we’ll go into Gaussian mixture models and kernel density estimation, where we talk about how to learn the probability distribution of a set of data.One interesting fact is that under certain conditions, Gaussian mixture models and k-means clustering are exactly the same! We’ll prove how this is the case.Lastly, we’ll look at the theory behind principal components analysis or PCA. PCA has many useful applications: visualization, dimensionality reduction, denoising, and de-correlation. You will see how it allows us to take a different perspective on latent variables, which first appear when we talk about k-means clustering and GMMs.All the algorithms we’ll talk about in this course are staples in machine learning and data science, so if you want to know how to automatically find patterns in your data with data mining and pattern extraction, without needing someone to put in manual work to label that data, then this book is for you.All of the materials required to follow along in this book are free: You just need to able to download and install Python, Numpy, Scipy, Matplotlib, and Sci-kit Learn.
Publication date: 05/22/2016Kindle book details: Kindle Edition, 38 pages
Deep Learning in Python: Master Data Science and Machine Learning with Modern Neural Networks written in Python, Theano, and TensorFlow (Machine Learning in Python)
Deep LearningDeep learning is making waves. At the time of this writing (March 2016), Google’s AlghaGo program just beat 9-dan professional Go player Lee Sedol at the game of Go, a Chinese board game.Experts in the field of Artificial Intelligence thought we were 10 years away from achieving a victory against a top professional Go player, but progress seems to have accelerated!While deep learning is a complex subject, it is not any more difficult to learn than any other machine learning algorithm. I wrote this book to introduce you to the basics of neural networks. You will get along fine with undergraduate-level math and programming skill.All the materials in this book can be downloaded and installed for free. We will use the Python programming language, along with the numerical computing library Numpy. I will also show you in the later chapters how to build a deep network using Theano and TensorFlow, which are libraries built specifically for deep learning and can accelerate computation by taking advantage of the GPU.Unlike other machine learning algorithms, deep learning is particularly powerful because it automatically learns features. That means you don’t need to spend your time trying to come up with and test “kernels” or “interaction effects” - something only statisticians love to do. Instead, we will let the neural network learn these things for us. Each layer of the neural network learns a different abstraction than the previous layers. For example, in image classification, the first layer might learn different strokes, and in the next layer put the strokes together to learn shapes, and in the next layer put the shapes together to form facial features, and in the next layer have a high level representation of faces.On top of all this, deep learning is known for winning its fair share Kaggle contests. These are machine learning contests that are open to anyone in the world who are allowed to use any machine learning technique they want. Deep learning is that powerful.Do you want a gentle introduction to this “dark art”, with practical code examples that you can try right away and apply to your own data? Then this book is for you.Who is this book NOT for?Deep Learning and Neural Networks are usually taught at the upper-year undergraduate level. That should give you some idea of the type of knowledge you need to understand this kind of material.You absolutely need exposure to calculus to understand deep learning, no matter how simple the instructor makes things. Linear algebra would help. I will assume familiarity with Python (although it is an easy language to pick up). You will need to have some concept of machine learning. If you know about algorithms like logistic regression already, this book is perfect for you. If not, you might want to check out my “prerequisites” book, at: http://amzn.com/B01D7GDRQ2On the other hand, this book is more like a casual primer than a dry textbook. If you are looking for material on more advanced topics, like LSTMs, convolutional neural networks, or reinforcement learning, I have online courses that teach this material, for example: https://www.udemy.com/deep-learning-convolutional-neural-networks-theano-tensorflow New libraries like TensorFlow are being updated constantly. This is not an encyclopedia for these libraries (as such a thing would be impossible to keep up to date). In the one (1!!!) month since the book was first published, no less than THREE new wrapper libraries for TensorFlow have been released to make coding deep networks easier. To try and incorporate every little update would not only be impossible, but would continually cause parts of the book to be obsolete. Nobody wants that. This book, rather, includes fundamentals. Understanding these building blocks will make tackling these new libraries and features a piece of cake - that is my goal.
Publication date: 03/11/2016Kindle book details: Kindle Edition, 50 pages
Natural Language Processing in Python: Master Data Science and Machine Learning for spam detection, sentiment analysis, latent semantic analysis, and article spinning (Machine Learning in Python)
Recently, Microsoft’s Twitter bot “Tay” was released into the wild, and quickly began making racist and hateful statements after learning from other Twitter users. The technology behind this? Natural language processing.NLP is the use of machine learning algorithms for problems that involve text.Do you ever wonder why you get much less spam in your inbox these days compared to 10 years ago? What kinds of algorithms are people using to do spam detection? How can they take words in an email and know how to compute whether or not it’s spam? In this book you are going to build your very own spam detector.Did you know people have already used Twitter to determine the current sentiment about a particular company to decide whether or not they should buy or sell stocks? Having a machine that can decide how people feel about something is immensely useful and immediately applicable to revenue optimization. In this course you are going to build your own sentiment analyzer.Are you an Internet marketer or are you interested in SEO? Have you ever wanted to know how you can automatically generate content? In this course we are going to take a first crack at building your own article spinner. You’ll learn to write programs that can take an article as input and spit out a similar article with different words as output. This can save you tons of time and thousands of dollars if you’re paying someone to write content for you.Natural Language Processing, or as it is often abbreviated, NLP - is the use of programming and math to do language-based tasks.If you have Windows or iOS then you have NLP right in front of you! Cortana and Siri are applications that take what you say and turn it into something meaningful that can be done programmatically.The key point: NLP is highly practical. NLP is everywhere.This book is split up into multiple sections based on the various practical tasks that you can do with NLP:Before we do any real programming exercises we’ll look at common NLP tasks (some of these we will actually code ourselves, the others are mentioned so you at least know they exist). We will then look at common data pre-processing techniques used for text. As you’ll see, this preprocessing is what will actually take up a majority of your time when you’re doing NLP.The first programming exercise we’ll do is look at how to build a spam detector. Your email inbox uses this, so it’s clearly very useful and it’s been the subject of study for a long time.Next we’ll look at “sentiment analysis” and you’ll build your own “sentiment analyzer”. This is how a computer can judge how positive or negative some text is based on the words and phrases that are used. This is also immediately practical - some people have analyzed Twitter feeds to predict whether a stock would go up or down.After that we’ll look at the NLTK library. This is a very popular library that solves a lot of fundamental problems in NLP - and you can use it in conjunction with other libraries in your data analysis pipeline.Next we’ll look at “latent semantic analysis”. This is basically doing dimensionality reduction on text - and it helps us solve the problem of 2 words having the same meaning. It also helps us interpret our data and save on computation time.Lastly, we’ll talk about one of the most popular applications of NLP - article spinning. This is very practical for internet marketers and entrepreneurs. As you know, your search rankings in Google and other search engines are affected negatively when you have duplicate content - so it would be great if you could alter an article you wrote just enough, so that you could put it in 2 different places on the web, without being penalized by Google.All of the materials required to follow along with this book can be downloaded for FREE. We use Python, NLTK, Sci-kit learn, and Numpy, all of which can be installed using simple commands.
Publication date: 05/17/2016Kindle book details: Kindle Edition, 38 pages
SQL for Marketers: Dominate data analytics, data science, and big data (Data Science and Machine Learning in Python)
Be data-drivenMore and more companies these days are learning that they need to make DATA-DRIVEN decisions.With big data and data science on the rise, we have more data than we know what to do with.One of the basic languages of data analytics is SQL, which is used for many popular databases including MySQL, Postgres, Microsoft SQL Server, Oracle, and even big data solutions like Hive and Cassandra.I’m going to let you in on a little secret. Most high-level marketers and product managers at big tech companies know how to manipulate data to gain important insights. No longer do you have to wait around the entire day for some software engineer to answer your questions - now you can find the answers directly, by yourself, using SQL!Your existing solutions are inefficientAre you tired of depending on crufty “analytics” software? Do you have to ask an engineer to help you whenever you have a question about the data?This is not ideal and won’t help you do your job efficiently.SQL, short for “structured query language”, is a language that can be used for all kinds of databases - from the tiny databases stored in your iPhone, to large big data databases that span multiple continents.Engineers have done a great job of creating these different types of complex data stores, while still allowing you to use the same language, more or less, for all of them.What does that mean for you?It means as long as you know SQL, you can take advantage of ALL of this software, and gain insights into this data, no matter what kind of database it is stored in, as long as it supports SQL.You can ask questions like:
- How many people are falling into each stage of the sales funnel?
- What is my year over year profit?
- Are there any differences in the demographics between the people who are buying product X and product Y?
- What is our most profitable month?
- What are the seasonal trends in our industry?
Publication date: 03/17/2016Kindle book details: Kindle Edition, 34 pages
Artificial Intelligence: Reinforcement Learning in Python: Complete guide to artificial intelligence and machine learning, prep for deep reinforcement learning
When people talk about artificial intelligence, they usually don’t mean supervised and unsupervised machine learning.These tasks are pretty trivial compared to what we think of AIs doing - playing chess and Go, driving cars, and beating video games at a superhuman level.Reinforcement learning has recently become popular for doing all of that and more.Much like deep learning, a lot of the theory was discovered in the 70s and 80s but it hasn’t been until recently that we’ve been able to observe first hand the amazing results that are possible.In 2016 we saw AlphaGo beat the world Champion in Go.We saw AIs playing video games like Doom and Super Mario.Self-driving cars have started driving on real roads with other drivers and even carrying passengers, all without human assistance.If that sounds amazing, brace yourself for the future because the law of accelerating returns dictates that this progress is only going to continue to increase exponentially.Yet learning about supervised and unsupervised machine learning is no small feat. To date I have over 16 courses just on those topics alone.And still reinforcement learning opens up a whole new world. As you’ll learn in this book, the reinforcement learning paradigm is more different from supervised and unsupervised learning than they are from each other.It’s led to new and amazing insights both in behavioral psychology and neuroscience. As you’ll learn in this course, there are many analogous processes when it comes to teaching an agent and teaching an animal or even a human. It’s the closest thing we have so far to a true general artificial intelligence.What’s covered in this course?The multi-armed bandit problem and the explore-exploit dilemmaWays to calculate means and moving averages and their relationship to stochastic gradient descentMarkov Decision Processes (MDPs)Dynamic ProgrammingMonte CarloTemporal Difference (TD) LearningApproximation Methods (i.e. how to plug in a deep neural network or other differentiable model into your RL algorithm)If you’re ready to take on a brand new challenge, and learn about AI techniques that you’ve never seen before in traditional supervised machine learning, unsupervised machine learning, or even deep learning, then this course is for you.HARD PREREQUISITES / KNOWLEDGE YOU ARE ASSUMED TO HAVE:CalculusProbabilityObject-oriented programmingPython coding: if/else, loops, lists, dicts, setsNumpy coding: matrix and vector operationsLinear regressionGradient descent
Publication date: 03/02/2017Kindle book details: Kindle Edition, 212 pages
Deep Learning: Natural Language Processing in Python with Word2Vec: Word2Vec and Word Embeddings in Python and Theano (Deep Learning and Natural Language Processing Book 1)
Word2VecWord2Vec is a set neural network algorithms that have gotten a lot of attention in recent years as part of the re-emergence of deep learning in AI.The idea that one can represent words and concepts as vectors is not new. The ability to do it effectively and generate noteworthy results is.Word2Vec algorithms are especially interesting because they allow us to perform arithmetic on the word vectors that yield both surprising and satisfying results. We call these “word analogies”.Some popular word analogies Word2Vec is capable of finding:“King” is to “Man” as “Queen” is to “Woman”.“France” is to “Paris” as “Italy” is to “Rome”.“December” is to “November” as “July” is to “June”.Not only can we cluster similar words together, we can make all these clusters have the same “structure”, all by using Word2Vec.Word2Vec was created by a team led by Tomas Mikolov at Google and has many advantages over earlier algorithms that attempt to do similar things, like Latent Semantic Analysis (LSA) or Latent Semantic Indexing (LSI).In this book we cover various popular flavors of the Word2Vec algorithm, including CBOW (continuous bag-of-words), skip-gram, and negative sampling.I show you both their derivations in math (you’ll see that if you already are familiar with deep learning concepts, there is no new math to be learned), and how to implement them in code.Whereas implementation in Numpy is just the straightforward application of the equations in code, Theano is a bit more complex because it requires new array-slicing techniques, namely running gradient descent on only a part of a matrix. It’s not straightforward, but I walk you through all the bits and pieces required to understand the full implementation.Amazingly, all the technologies we discuss in this book can be downloaded and installed for FREE. That means all you need to invest after purchasing this book is your effort and your time. The only prerequisites are that you are comfortable with Python , Numpy, and Theano coding and you know the basics of deep learning.“Hold up... what’s deep learning and all this other crazy stuff you’re talking about?”If you are completely new to deep learning, you might want to check out my earlier books and courses on the subject, since they are required in order to understand this book:Deep Learning in Python https://www.udemy.com/data-science-deep-learning-in-pythonDeep Learning in Python Prerequisities https://www.udemy.com/data-science-logistic-regression-in-pythonMuch like how IBM’s Deep Blue beat world champion chess player Garry Kasparov in 1996, Google’s AlphaGo recently made headlines when it beat world champion Lee Sedol in March 2016.What was amazing about this win was that experts in the field didn’t think it would happen for another 10 years. The search space of Go is much larger than that of chess, meaning that existing techniques for playing games with artificial intelligence were infeasible. Deep learning was the technique that enabled AlphaGo to correctly predict the outcome of its moves and defeat the world champion.Deep learning progress has accelerated in recent years due to more processing power (see: Tensor Processing Unit or TPU), larger datasets, and new algorithms like the ones discussed in this book.Book 2 in the series can be found at: https://www.amazon.com/dp/B01KRBOO4YBook 3 in the series can be found at: https://www.amazon.com/dp/B01KS5AEXO
Publication date: 08/19/2016Kindle book details: Kindle Edition, 47 pages