Data Science with Python

Python has established itself as the go-to language for data science thanks to its rich ecosystem of specialized libraries. This article introduces the essential tools you need to get started with data science in Python.

The Python data science ecosystem

Python offers a complete set of tools for every step of the data science process:

• Data manipulation: Pandas, NumPy
• Visualization: Matplotlib, Seaborn, Plotly
• Machine learning: Scikit-learn, TensorFlow, PyTorch
• Statistical analysis: SciPy, StatsModels
• Natural language processing: NLTK, spaCy
• Development environments: Jupyter Notebook, Google Colab

Installing the essential libraries

Start by installing the core libraries:

pip install numpy pandas matplotlib seaborn scikit-learn jupyter

Data manipulation with Pandas

Pandas is the must-have library for data manipulation in Python:

import pandas as pd
import numpy as np

# Create a DataFrame
data = {
    'Nom': ['Alice', 'Bob', 'Charlie', 'David'],
    'Age': [25, 30, 35, 40],
    'Ville': ['Paris', 'Lyon', 'Marseille', 'Bordeaux'],
    'Salaire': [45000, 55000, 60000, 70000]
}

df = pd.DataFrame(data)
print(df.head())

# Filter the data
jeunes = df[df['Age'] < 35]
print(jeunes)

# Descriptive statistics
print(df.describe())

# Group and aggregate
par_ville = df.groupby('Ville').agg({
    'Salaire': ['mean', 'min', 'max'],
    'Age': 'mean'
})
print(par_ville)

Data visualization

Visualization is essential for understanding your data:

import matplotlib.pyplot as plt
import seaborn as sns

# Visualization style
sns.set(style="whitegrid")

# Age distribution
plt.figure(figsize=(10, 6))
sns.histplot(df['Age'], bins=10, kde=True)
plt.title('Distribution des âges')
plt.xlabel('Âge')
plt.ylabel('Fréquence')
plt.show()

# Relationship between age and salary
plt.figure(figsize=(10, 6))
sns.scatterplot(x='Age', y='Salaire', data=df)
plt.title('Relation entre âge et salaire')
plt.show()

# Average salary by city
plt.figure(figsize=(10, 6))
sns.barplot(x='Ville', y='Salaire', data=df)
plt.title('Salaire moyen par ville')
plt.show()

Machine learning with Scikit-learn

Scikit-learn simplifies the implementation of machine learning algorithms:

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

# Data preparation
X = df[['Age']]  # Independent variable
y = df['Salaire']  # Dependent variable

# Split into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Create and train the model
model = LinearRegression()
model.fit(X_train, y_train)

# Predictions
y_pred = model.predict(X_test)

# Model evaluation
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

print(f"Coefficient: {model.coef_[0]:.2f}")
print(f"Intercept: {model.intercept_:.2f}")
print(f"Erreur quadratique moyenne: {mse:.2f}")
print(f"Coefficient de détermination (R²): {r2:.2f}")

# Visualize the predictions
plt.figure(figsize=(10, 6))
plt.scatter(X_test, y_test, color='blue', label='Données réelles')
plt.plot(X_test, y_pred, color='red', linewidth=2, label='Prédictions')
plt.title('Régression linéaire: Âge vs Salaire')
plt.xlabel('Âge')
plt.ylabel('Salaire')
plt.legend()
plt.show()

Exploratory data analysis (EDA)

EDA is a crucial step for understanding your data before building models:

# Correlation matrix
plt.figure(figsize=(10, 8))
correlation = df.corr()
sns.heatmap(correlation, annot=True, cmap='coolwarm', linewidths=0.5)
plt.title('Matrice de corrélation')
plt.show()

# Pairplot to visualize relationships between variables
sns.pairplot(df)
plt.suptitle('Relations entre les variables numériques', y=1.02)
plt.show()

# Boxplot to detect outliers
plt.figure(figsize=(10, 6))
sns.boxplot(x='Ville', y='Salaire', data=df)
plt.title('Distribution des salaires par ville')
plt.show()

Jupyter Notebook: the ideal environment

Jupyter Notebook is the environment of choice for data scientists:

# Launch Jupyter Notebook
jupyter notebook

Benefits of Jupyter:

• Combines code, visualizations, and explanatory text
• Cell-by-cell execution for iterative development
• Easy sharing of analyses
• Support for rich formats (Markdown, LaTeX, HTML)

Conclusion

Python offers a complete and mature ecosystem for data science. By mastering these core tools, you will be able to:

• Efficiently import and clean your data
• Create informative visualizations
• Build and evaluate machine learning models
• Communicate your results clearly and convincingly

Data science with Python is a vast and constantly evolving field. Keep exploring and learning to stay up to date with the latest advances and techniques.