VE401/ECE4010J Probabilistic Methods in Engineering - Recitation Classes

This repository contains recitation class materials for UM-SJTU Joint Institute course VE401/ECE4010J, Probabilistic Methods in Engineering, taught by Dr. Horst Hohberger. It aims to visualize and emphasize some key concept in class and illustrate with simple examples using Wolfram Mathematica, a powerful tool.

Keywords are listed in order as follows for easy searching and revisiting. I only list the first appearance of each concept.

Probability Concepts, Random Variables & Distributions

• Recitation Class 1:
  • Concepts: Mutual exclusive, basic counting, probability measures, probability spaces, conditional probability, independence, total probability, Bayes's Theorem, discrete random variable and probability, expectation, probability density function (PDF) and cumulative distribution function (CDF), Bernoulli distribution, Binomial distribution, Geometric distribution.
  • Problems of interest: Banach Matchbox Problem, Two Envelopes Paradox.
• Recitation Class 2:
  • Concepts: Variance and standard variance, moment and moment generating function (MGF), Binomial distribution (approximations of its PDF and CDF), Pascal distribution, negative binomial distribution, Poisson distribution, continuous random variables, exponential distribution, gamma distribution, chi-squared distribution, transformation of random variable, normal distribution, Chebyshev Inequality.
  • Problems of interest: Two Children Paradox, Aircraft Maintenance problem.
• Recitation Class 3:
  • Concepts: Hypergeometric distribution (and approximation of its PDF), summary of discrete random variables, reliability of a system (failure density, reliability function, hazard rate), Weibull distribution, summary of continuous random variables, joint distributions, discrete/continuous multivariate random variables, marginal density, conditional density, covariance, covariance matrix, pearson correlation coefficient, bivariate normal distribution, transformation of variables (generalized)
  • Problems of interest: Sum of two independent random variables, proving weak law of large numbers.

Statistics, Data Visualisations & Hypothesis Tests

• Recitation Class 4:
  • Concepts: percentiles and quartiles, histograms, stem-and-leaf diagrams, box plots, estimation, bias and mean square error, method of moments, method of maximum likelihood, interval estimation, confidence interval, Student T distribution, confidence interval for the mean/variance.
  • Problems of interest: Linear combination of two independent normal distributions.
• Recitation Class 5:
  • Concepts: Fisher's hypothesis test, significance/p-value for one/two-tailed test, Neyman-Pearson Decision Theory, type I (false negative) and type II (false positive) errors, $\alpha$ and critical region, $\beta$ and sample size, OC curve, Null Hypothesis Significance Test (NHST), single sample tests, test statistics, test for mean (Z-test and T-test), test for variance (chi-squared test), test for median (sign test, Wilcoxon signed rank test), confidence interval for proportion, test for proportion (Z-test), comparing two proportions (Z-test, Pool test), F distribution, comparing two variances (F-test).
  • Problems of interest: Test for fairness of a coin (Frequentist vs Bayesian approach)
• Recitation Class 6:
  • Concepts: comparing two means (Z-test, Pooled/paired T-test), comparing locations of two random variables (Wilcoxon rank sum test), test for mean of difference of two random variables (paired T-test), comparison between pooled and paired T-test, estimation of correlation coefficient, test for correlation coefficient (Z-test), multinomial distribution, Pearson statistics, chi-squared goodness-of-fit test, test for independence of categorizations.
  • Problems of interest: Testing hypotheses on an exponential random variable, drawing OC curve for F-test.

Data Mining, Linear Regression

• Recitation Class 7:
  • Concepts: Simple linear regression, least-squares estimators, test for significance of regression, distribution of estimated mean, conficence interval and prediction interval, error sum of squares (SSE), R squared (correlation of determination), test for lack of fit, multiple linear regression, error analysis, distribution of SSE, F-test for significance of regression.
• Final review:
  • Concepts: Distribution and confidence interval of least-squares estimators, distribution of estimated mean, T-test for model sufficiency, Partial F-test for model sufficiency, indicator variable, model selection.