Feature Selection: A linear regression approach to find the impact of the features of e-commerce sales data

Load the data

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from mywebstyle import plot_style
plot_style('#f4f4f4')
salesdata = pd.read_csv('Ecommerce Customers')
salesdata.head()

	Email	Address	Avatar	Avg. Session Length	Time on App	Time on Website	Length of Membership	Yearly Amount Spent
0	mstephenson@fernandez.com	835 Frank Tunnel\nWrightmouth, MI 82180-9605	Violet	34.497268	12.655651	39.577668	4.082621	587.951054
1	hduke@hotmail.com	4547 Archer Common\nDiazchester, CA 06566-8576	DarkGreen	31.926272	11.109461	37.268959	2.664034	392.204933
2	pallen@yahoo.com	24645 Valerie Unions Suite 582\nCobbborough, D...	Bisque	33.000915	11.330278	37.110597	4.104543	487.547505
3	riverarebecca@gmail.com	1414 David Throughway\nPort Jason, OH 22070-1220	SaddleBrown	34.305557	13.717514	36.721283	3.120179	581.852344
4	mstephens@davidson-herman.com	14023 Rodriguez Passage\nPort Jacobville, PR 3...	MediumAquaMarine	33.330673	12.795189	37.536653	4.446308	599.406092

EDA

Descriptive Statistics

salesdata.describe()

	Avg. Session Length	Time on App	Time on Website	Length of Membership	Yearly Amount Spent
count	500.000000	500.000000	500.000000	500.000000	500.000000
mean	33.053194	12.052488	37.060445	3.533462	499.314038
std	0.992563	0.994216	1.010489	0.999278	79.314782
min	29.532429	8.508152	33.913847	0.269901	256.670582
25%	32.341822	11.388153	36.349257	2.930450	445.038277
50%	33.082008	11.983231	37.069367	3.533975	498.887875
75%	33.711985	12.753850	37.716432	4.126502	549.313828
max	36.139662	15.126994	40.005182	6.922689	765.518462

salesdata.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 500 entries, 0 to 499
Data columns (total 8 columns):
 #   Column                Non-Null Count  Dtype  
---  ------                --------------  -----  
 0   Email                 500 non-null    object 
 1   Address               500 non-null    object 
 2   Avatar                500 non-null    object 
 3   Avg. Session Length   500 non-null    float64
 4   Time on App           500 non-null    float64
 5   Time on Website       500 non-null    float64
 6   Length of Membership  500 non-null    float64
 7   Yearly Amount Spent   500 non-null    float64
dtypes: float64(5), object(3)
memory usage: 31.4+ KB

Visualization

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(7.9, 5))

# Scatter plot with regression line for 'Time on Website' vs 'Yearly Amount Spent'
sns.scatterplot(
    x='Time on Website', y='Yearly Amount Spent', 
    data=salesdata, ax=ax1
    )
sns.regplot(
    x='Time on Website', y='Yearly Amount Spent', 
    data=salesdata, ax=ax1, scatter=False, color='blue'
    )
ax1.set_title('Time on Website vs Yearly Amount Spent')

# Scatter plot with regression line for 'Time on App' vs 'Yearly Amount Spent'
sns.scatterplot(
    x='Time on App', y='Yearly Amount Spent', 
    data=salesdata, ax=ax2
    )
sns.regplot(
    x='Time on App', y='Yearly Amount Spent',
    data=salesdata, ax=ax2, scatter=False, color='blue'
    )
ax2.set_title('Time on App vs Yearly Amount Spent')

plt.tight_layout()
plt.show()

So, from this plot, we see that Time on Website has no significant trend or pattern on Yearly Amount Spent variable. However, Time on App seems to have a linear relationship on Yearly Amount Spent.

Next, we see the relationship between Avg. Session Length vs Yearly Amount Spent, and Length of Membership vs Yearly Amount Spent.

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(7.9, 5))

# Scatter plot with regression line for 'Time on Website' vs 'Yearly Amount Spent'
sns.scatterplot(
    x='Avg. Session Length', y='Yearly Amount Spent', 
    data=salesdata, ax=ax1
    )
sns.regplot(
    x='Avg. Session Length', y='Yearly Amount Spent', 
    data=salesdata, ax=ax1, scatter=False, color='blue'
    )
ax1.set_title('Avg. Session Length vs Yearly Amount Spent')

# Scatter plot with regression line for 'Time on App' vs 'Yearly Amount Spent'
sns.scatterplot(
    x='Length of Membership', y='Yearly Amount Spent', 
    data=salesdata, ax=ax2
    )
sns.regplot(
    x='Length of Membership', y='Yearly Amount Spent', 
    data=salesdata, ax=ax2, scatter=False, color='blue'
    )
ax2.set_title('Length of Membership vs Yearly Amount Spent')

plt.tight_layout()
plt.show()

Both of these features have impact on the dependent variable. However, Length of Membership seems to have the most significant impact on Yearly Amount Spent.

sns.pairplot(salesdata)

Modeling

Training

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

X = salesdata[
    ['Avg. Session Length', 'Time on App',
    'Time on Website', 'Length of Membership']
    ]
y = salesdata['Yearly Amount Spent']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size = 0.30, random_state=123
)

linreg = LinearRegression()
linreg.fit(X_train, y_train)
print('Coefficients: \n', linreg.coef_)

Coefficients: 
 [25.36266491 38.82367921  0.80356799 61.54905291]

Testing

pred = linreg.predict(X_test)
plt.scatter(y_test, pred)
plt.xlabel('y test')
plt.ylabel('predicted y')
plt.show()

Model Evaluation

from sklearn import metrics

print('MAE', metrics.mean_absolute_error(y_test, pred))
print('MSE', metrics.mean_squared_error(y_test, pred))
print('RMSE', metrics.root_mean_squared_error(y_test, pred))
print('R-squared:', metrics.r2_score(y_test, pred))

MAE 7.9880791942451
MSE 102.72313941866005
RMSE 10.135242444986703
R-squared: 0.9845789607829495

Residual Analysis

sns.displot(y_test-pred, bins= 60, kde=True)

Conclusion

coeff = pd.DataFrame({
    'Feature': ['Intercept'] + list(X.columns), 
    'Coefficient': [linreg.intercept_] + list(linreg.coef_) 
})

coeff

	Feature	Coefficient
0	Intercept	-1054.215476
1	Avg. Session Length	25.362665
2	Time on App	38.823679
3	Time on Website	0.803568
4	Length of Membership	61.549053