Data Science Projects

Pick a messy real-world dataset — NYC taxi trips, airline delays, Airbnb listings, or 311 service requests — and produce a polished EDA notebook that answers a specific business question. The key constraint: you must frame every finding as a decision. Not "there are more rides on Friday" but "demand peaks between 5–8 pm Friday in Midtown; surge pricing or supply reallocation would address the 14% drop-off rate during that window."

Build a binary classifier that predicts which customers will churn in the next 30 days. The focus is feature engineering and evaluation — not just fitting a model. Handle class imbalance correctly (SMOTE, class_weight, or threshold tuning), compare at least three model families, and use SHAP to explain which features drive predictions. The Telco Customer Churn dataset is overused; prefer a messier source like the IBM Sample Data or build features from raw event logs.

Work through the Ames Housing dataset end-to-end: handle 79 features including 19 categorical variables, ~15 columns with missing values, and highly skewed distributions. The goal is a production-quality preprocessing pipeline using sklearn Pipelines and ColumnTransformer — not ad hoc transformations that leak between train and test. Tune hyperparameters with Optuna. The pipeline itself is more valuable to show than the final score.

Build a reusable A/B test analysis module that handles the full lifecycle: pre-test power analysis (sample size calculator), frequentist analysis (t-test, chi-square, Mann-Whitney), Bonferroni and Benjamini-Hochberg corrections for multiple metrics, and a Bayesian alternative using PyMC for the same experiment. Apply it to a simulated e-commerce dataset — checkout flow variant test affecting conversion rate, average order value, and session duration simultaneously.

Forecast daily retail sales 28 days ahead using at least three model families: classical (SARIMA), statistical ML (Prophet), and deep learning (LSTM or N-BEATS via the Darts library). Use walk-forward validation — not a single train/test split — to avoid overfitting to seasonal patterns. Compare models on MAE, RMSE, and MAPE. Include a decomposition analysis (trend, seasonality, residuals) as an explainability layer for non-technical stakeholders.

Build a two-stage NLP pipeline: (1) fine-tune a DistilBERT model on a sentiment classification task (SST-2 or Amazon reviews) and compare it against a TF-IDF + logistic regression baseline to quantify the transformer uplift; (2) apply BERTopic to the same corpus to extract latent topics, then correlate topic prevalence with sentiment scores over time. Deploy a Streamlit UI where a user pastes text and sees sentiment + topic classification with confidence scores.

Segment customers using RFM (Recency, Frequency, Monetary) features extracted from transactional data. Apply K-Means clustering with optimal k selection via elbow method and silhouette scores, then validate with hierarchical clustering. Use PCA for 2D visualisation. The critical deliverable is a business interpretation slide for each segment: who they are, what they spend, and what action the business should take (win-back campaign for churned high-value, upsell for frequent low-spend, etc.).

Build a fully reproducible ML pipeline for a classification problem: version your data with DVC, track every experiment (parameters, metrics, artefacts) with MLflow, register the best model in the MLflow Model Registry, and deploy it via a Docker container with a FastAPI inference endpoint. Wire a GitHub Actions CI pipeline that reruns training on data changes and fails if the new model's ROC-AUC regresses below the registered baseline. Use Evidently for data drift monitoring on a held-out test set.

Use the DoWhy and EconML libraries to estimate the causal effect of a treatment on an outcome — not just the correlation. Work through three estimators on the same dataset (propensity score matching, doubly-robust estimator, and meta-learner) and explain why they disagree. Run sensitivity analysis (Rosenbaum bounds, partial R²) to quantify how large an unobserved confounder would need to be to overturn your conclusion. Apply to a policy evaluation problem: minimum wage effect on employment, medication adherence on readmission, or marketing spend on conversion.