Thoughts on Theta

Nathanael O'Donnell Neural Networks - An Introduction In comparison with most other machine learning techniques—including linear regression and logistic regression—neural networks are significantly more complex. The simplest way of explaining neural networks is as multi-dimensional logistic regression, in which the "layers" of the neural network representing additional transformations. Somewhat intuitively (because they are based on logistic regression), neural networks, in their more basic forms, are classification models. However, more complex neural networks (those with more neurons) can serve as regression models as well.  So if neural networks are so much more complicated than other machine learning methods, then why do practitioners use them? The simple answer is that they work better than other approaches in many situations. The complexity of neural networks is what makes them such a flexible and powerful approach for more complex problems. Neurons can be...

Nathanael O'Donnell What is Logistic Regression?  Along with linear regression, logistic regression is a widely used machine learning algorithm. In spite of its name, however—and in contrast to linear regression—logistic regression is not actually a regression algorithm. Rather, it is a classification algorithm , meaning that it is useful when categorizing (aka "classifying") observations based on any number of feature variables. Mathematically, logistic regression works by mapping observations to points along a logistic function , whose standard equation and graph are below: Source: Andrew Ng Note that g(z) , called the logistic function or sigmoid function, tends towards 1 as z approaches infinity, and tends towards 0 as z approaches negative infinity. This property of the logistic function is what makes it useful for classification: the output of the function will also be a number between 0 and 1, and more extreme values of z —in either direction...

Applications of Linear Regression Nathanael O'Donnell Linear regression, while one of the simplest machine learning algorithms available to practitioners, can be surprisingly effective and is widely used in research as well as industry. It is perhaps true that the simplicity of linear regression is also part of its strength: if you can visualize a trend line fitting a certain problem (say, SAT scores affecting college admission rates), then linear regression can be of use. And even if the problem in question is not truly linear—for example, if the relationship between SAT scores and college admission rates is logarithmic as opposed to linear—linear regression can still often provide an approximation that is good enough to solve the business problem at hand. Because of its broad set of potential use-cases, there are plenty of real-world examples of situations where linear regression could be useful. I will explore some of these examples, which I have drawn from the article lin...