Probability theory basics I. - Random experiment, outcome, sample space, event.
Postulates (axioms) of probability theory and some properties following from
them. Independence and conditional probability. Probability of independent
simultaneous events. Interpretations of prbability: frequency, classical
probability and geometric probability.
Review (in Hungarian) of the book
Paradoxes in Probability Theory and Mathematical Statistics.
Probability theory basics II. - Random variables. Simple events concerning
discrete and continuous random variables. Sampling distributions. Properties of
the probability density function. Relation of the probability distribution
function to the probability density function. Calculation of probabilities based
on the probability distribution function and on the probability density
function. Expectations and their properties. Calculation of the expectation as a
linear operation. Some particularly important expectations. (Distribution mean,
distribution moments, variance, covariance, correlation coefficient, entropy.)
Some important relations for calculating expectations.
Probability theory basics III. - The covariance matrix and its properties.
Relation between independence of random variables and their covariance.
Calculation of (normalised) probability density function, if not known. The law
of large numbers. Stochastic convergence and random walk. Some important
probability distributions. The binomial and hypergeometric distributions.
Poisson process: interrelation of the uniform, the Poisson and the exponential
An Excel worksheet to explore random walk and
fluctuations. A detailed
explanation of the waiting-time paradox.
Probability theory basics IV. - Poisson and the exponential
distributions. Gamma distribution. Normal distribution.
Introduction to statistical methods. Chi-squared, Student's t, and Fisher's F
distribution. Distributions without a maximum. The arcsin distribution.
The aim and methods of statistics. Population vs. sample. Sampling. Estimation
and characteristics of estimators. Estimation methods. Histograms. Sample
statistics. Sample mean, sample variance and covariance.
Auxiliary material: Problems concerning sampling in a
Hungarian election and a U.S. election. A
short appetizing paper and a
on cognitive bias (or "self fooling") from Nature.
Homework: histogram construction.
Estimators, estimation and estimates. Expected properties of estimators. Methods
of estimation. Histograms. The method of maximum likelihood and a few actual
applications. Further examples concerning maximum likelihood.
Auxiliary material: Assignment for histogram
November 11, Thursday:
Estimation: The method of least squares. The method of moments. Other estimation
methods. Estimation of expectation and variance of functions of random
Confidence intervals. Formulation of confidence in a computable form. Confidence
interval for the expectation of a normal distribution with known variance.
Confidence interval for the expectation of a normal distribution with unknown
variance. Confidence interval for the the parameters of a binomial distribution.
Auxiliary material: Assignment for parameter
estimation using the method of moments.
Confidence interval for the the parameters of a binomial distribution.
Confidence interval for the variance. Approximate confidence interval for a
function of random variables. Confidence interval for the difference of the
expectations of two random variables.
Statistical hypothesis testing - general considerations. Null hypothesis and
alternative hypothesis. Types of hypotheses. Statistics underlying decision
making. Type I and type II errors. Power function of the test.
Auxiliary material: Common
concerning hypothesis testing. Assignment concerning confidence intervals:
1. Confidence of mean and variance. 2.
Confidence and comparison of two means.
November 25, Thursday:
9th lecture: Statistical hypothesis testing:
Null hypothesis and alternative hypothesis. Types of hypotheses. Statistics
underlying decision making. Type I and type II errors. Test on the mean of a
normal distribution with known and unknown variance. Test on the parameter p
of a binomial distribution ("test on proportions"). Tests between means drawn
from two samples in the case of normal distributions and binomial distributions
("tests on differences").
Auxiliary material: Assignement on
the confidence interval of proportion differences;
testing a mean.
Tests on matching pairs. Nonparametric tests. The Sign-test; the Mann-Whitney
and Wilcoxon (Rank-Sum) test. Tests on several means: One-way ANOVA and
two-way ANOVA tests. Multivariate analysis of variance (MANOVA).
Homoscedasticity tests (tests on variances). Functional relations between random
variables. Testing the correlation coefficient.
Auxiliary material: A CU Boulder leaflet on
MANOVA; Assignement on testing two
meansns. Assignment on
1-way ANOVA, assignment on 2-way ANOVA.
11th lecture: Estimation of parameters
describing functions of random variables. The general straight line and the
straight line through the origin. Testing the difference between the two cases:
the significance test of the intercept. Weighted least squares (LSQ) estimation.
Optimisation of weights to give MVU estimation. Implicit regression.
Overview of the conditions for the validity of LSQ estimation.
Multivariate analysis: a short overview of multivariate methods.
Last modified: September 28, 2021