Applications of Linear Regression

Applications of Linear Regression

Nathanael O'Donnell

Linear regression, while one of the simplest machine learning algorithms available to practitioners, can be surprisingly effective and is widely used in research as well as industry. It is perhaps true that the simplicity of linear regression is also part of its strength: if you can visualize a trend line fitting a certain problem (say, SAT scores affecting college admission rates), then linear regression can be of use. And even if the problem in question is not truly linear—for example, if the relationship between SAT scores and college admission rates is logarithmic as opposed to linear—linear regression can still often provide an approximation that is good enough to solve the business problem at hand.

Because of its broad set of potential use-cases, there are plenty of real-world examples of situations where linear regression could be useful. I will explore some of these examples, which I have drawn from the article linked here: http://intellspot.com/linear-regression-examples/

1) E-Commerce Revenue vs Advertising Spend: In digital marketing, a common and continually relevant question is how every incremental dollar of advertising spend translates into revenue (or margin). Now, as mentioned above, this relationship could very well be nonlinear, but applying linear regression to this problem could give a useful starting point or, depending on the level of precision required, could suffice as a production model. It's worth noting that the accuracy of this model could potentially be improved by adding more variables (and thus more parameters, i.e. coefficients) to this model by including such measurements as the date and/or time of day a particular advertisement is run, the type of advertising target (i.e., impressions vs. clicks vs. conversions), or the average duration an advertisement was kept active.

Image source: Inntellspot


2)  Price of Used Cars vs. Age: It is common knowledge that the market price of a used car will generally decrease as the car becomes older. There are some exceptions to this trend—such as the appreciation enjoyed by owners of collectible vintage and/or limited-edition vehicles—but if we were to collect a large data set that included price vs. age data for a number of cars, we would most likely see a strong negative correlation (price decreasing as age increases). Again, similar to the example of revenue vs. advertising spend, the relationship is likely not perfectly linear, but we can expect to get a decent approximation by applying linear regression. And if the relationship satisfies a logarithmic or exponential trend, we can choose to transform the data (by taking logarithms of the data points) before applying linear regression.

Image source: Intellspot


3) Percent of population that is urban vs. GDP: For the final example I'm sharing this week, I wanted to show how logging variables can allow us to apply linear regression to a nonlinear relationship. In this example, we take the logarithm of GDP to transform a skewed relationship into a "linear-log" relationship, to which we can productively apply our linear regression model. (This example is taken from the following article: https://kenbenoit.net/assets/courses/ME104/logmodels2.pdf)

A scatter plot of the untransformed data appears thusly:

Source: Kenneth Benoit, London School of Economics

While after transforming it, the data exhibits a clear linear (linear-log) trend:

Source: Kenneth Benoit, London School of Economics

From these three examples, it should be clear that linear regression has a wide range of applications, especially in light of the fact that logarithmic transformations can make data more amenable to the algorithm. And while more sophisticated algorithms can often achieve better results, there is much to be said for using linear regression as a quick-and-dirty method that adds value immediately. Thank you for reading, and I hope you can find useful applications of linear regression in your own work!

Comments

Post a Comment

Popular posts from this blog

The Future of Machine Learning

Image
Nathanael O'Donnell This week, I would like to open up the field and contemplate the future of machine learning. In a relatively short period of time—barely one decade—machine learning has gone from being a relatively niche academic domain to becoming an integral part of our global information infrastructure. And the impacts of machine learning are being felt not just in the arena that we traditionally consider "software"; rather, machine learning is enabling intelligent behavior in all sorts of physical devices, leading to the "internet of things" that many have be prophesying. Developments that once sounded like science fiction—such as a computer program beating a world champion in Go—are now science fact. The question is, what comes next? Rather than speculate using only my own imagination, I will survey the plans and predictions of industry leaders and pioneering academics. AI Will Enhance Creativity A great first stop for those interested in how ma...
Read more

Applications of Neural Networks

Image
Nathanael O'Donnell Neural Networks - An Introduction In comparison with most other machine learning techniques—including linear regression and logistic regression—neural networks are significantly more complex. The simplest way of explaining neural networks is as multi-dimensional logistic regression, in which the "layers" of the neural network representing additional transformations. Somewhat intuitively (because they are based on logistic regression), neural networks, in their more basic forms, are classification models. However, more complex neural networks (those with more neurons) can serve as regression models as well.  So if neural networks are so much more complicated than other machine learning methods, then why do practitioners use them? The simple answer is that they work better than other approaches in many situations. The complexity of neural networks is what makes them such a flexible and powerful approach for more complex problems. Neurons can be...
Read more

Applications of Logistic Regression

Image
Nathanael O'Donnell What is Logistic Regression?  Along with linear regression, logistic regression is a widely used machine learning algorithm. In spite of its name, however—and in contrast to linear regression—logistic regression is not actually a regression algorithm. Rather, it is a classification algorithm , meaning that it is useful when categorizing (aka "classifying") observations based on any number of feature variables. Mathematically, logistic regression works by mapping observations to points along a logistic function , whose standard equation and graph are below: Source: Andrew Ng Note that g(z) , called the logistic function or sigmoid function, tends towards 1 as z approaches infinity, and tends towards 0 as z approaches negative infinity. This property of the logistic function is what makes it useful for classification: the output of the function will also be a number between 0 and 1, and more extreme values of z —in either direction...
Read more