In [1]:
import pandas as pd
import numpy as np
import os

import plotly.express as px
import plotly.graph_objects as go
pd.options.plotting.backend = 'plotly'
TEMPLATE = 'seaborn'

Lecture 20 – Modeling and Linear Regression¶

DSC 80, Spring 2023¶

Agenda¶

  • Modeling.
  • Case study: Restaurant tips 🧑‍🍳.
  • Regression in sklearn.

Modeling¶

Reflection¶

So far this quarter, we've learned how to:

  • Extract information from tabular data using pandas and regular expressions.
  • Clean data so that it best represents a data generating process.
    • Missingness analyses and imputation.
  • Collect data from the internet through scraping and APIs, and parse it using BeautifulSoup.
  • Perform exploratory data analysis through aggregation, visualization, and the computation of summary statistics like TF-IDF.
  • Infer about the relationships between samples and populations through hypothesis and permutation testing.
  • We haven't learned how to make predictions.

Modeling¶

  • Data generating process: A real-world phenomena that we are interested in studying.
    • Example: Every year, city employees are hired and fired, earn salaries and benefits, etc.
    • Unless we work for the city, we can't observe this process directly.
  • Model: A theory about the data generating process.
    • Example: If an employee is $X$ years older than average, then they will make $100,000 in salary.
  • Fit Model: A model that is learned from a particular set of observations, i.e. training data.
    • Example: If an employee is 5 years older than average, they will make $100,000 in salary.
    • How is this estimate determined? What makes it "good"?

Goals of modeling¶

  1. To make accurate predictions regarding unseen data drawn from the data generating process.
    • Given this dataset of past UCSD data science students' salaries, can we predict your future salary? (regression)
    • Given this dataset of images, can we predict if this new image is of a dog, cat, or zebra? (classification)
  1. To make inferences about the structure of the data generating process, i.e. to understand complex phenomena.
    • Is there a linear relationship between the heights of children and the heights of their biological mothers?
    • The weights of smoking and non-smoking mothers' babies babies in my sample are different – how confident am I that this difference exists in the population?
  • Of the two focuses of models, we will focus on prediction.
  • In the above taxonomy, we will focus on supervised learning.

Features¶

  • A feature is a measurable property of a phenomenon being observed.
    • Other terms for "feature" include "(explanatory) variable" and "attribute".
    • Typically, features are the inputs to models.
  • In DataFrames, features typically correspond to columns, while rows typically correspond to different individuals.
  • There are two types of features:
    • Features that come as part of a dataset, e.g. weight and height.
    • Features that we create, e.g. $\text{BMI} = \frac{\text{weight (kg)}}{\text{[height (m)]}^2}$.
  • Example: TF-IDF is a feature we've created that summarizes documents!

Example: Restaurant tips 🧑‍🍳¶

About the data¶

What features does the dataset contain?

In [2]:
# The dataset is built into plotly (and seaborn)!
tips = px.data.tips()
tips
Out[2]:
total_bill tip sex smoker day time size
0 16.99 1.01 Female No Sun Dinner 2
1 10.34 1.66 Male No Sun Dinner 3
2 21.01 3.50 Male No Sun Dinner 3
3 23.68 3.31 Male No Sun Dinner 2
4 24.59 3.61 Female No Sun Dinner 4
... ... ... ... ... ... ... ...
239 29.03 5.92 Male No Sat Dinner 3
240 27.18 2.00 Female Yes Sat Dinner 2
241 22.67 2.00 Male Yes Sat Dinner 2
242 17.82 1.75 Male No Sat Dinner 2
243 18.78 3.00 Female No Thur Dinner 2

244 rows × 7 columns

Predicting tips¶

  • Goal: Given various information about a table at a restaurant, we want to predict the tip that a server will earn.
  • Why might a server be interested in doing this?
    • To determine which tables are likely to tip the most (inference).
    • To predict earnings over the next month (prediction).

Exploratory data analysis (EDA)¶

  • The most natural feature to look at first is 'total_bill'.
  • As such, we should explore the relationship between 'total_bill' and 'tip', as well as the distributions of both columns individually.
  • As we do so, try to describe each distribution in words.

Visualizing distributions¶

In [3]:
tips.plot(kind='scatter', 
          x='total_bill', y='tip',
          title='Tip vs. Total Bill',
          template=TEMPLATE)