Class:	MW 5-6:45, Soc.Sc.2 - 137
Office hours:	Mo,We 12:30-1:30 - E2-357
Prerequisite:	CMPS 242 or my consent
		some probability theory and linear algebra

Rough syllabis
Some of my previous classes

Summary of lectures
1	Notes 1
    	Intro to Machine Learning using curve fitting as an example
	Overfitting, complexity control, regularization
        Experimental setups w. training, validation and test sets

	On-line versus batch
	Definition of regret
	Halfing algorithm and its bound
	Weighted Majority algorithm
	Regret bound for WM via potential function

	Derivation of additive and multiiplicative on-line updates for linear regression
        
 	Bug Machine 

	Hedge algorithm, Follow the Leader algorithm
	Hedge algorithm paper
	
2 	Notes 2
	Analysis of Hedge algorithm via potential function
	Expert framework with a variety of loss functions paper
	Maple file on how to tune Hedge

	Talk re. various Share Updates incl. one that induce longterm memory
 	Long term memory paper
 	Original "Tracking the best expert" paper

 	Homework 1 Due We 4-13-09 at the beginning of class.

3 	Long-term memory with faces
        In long-term memory algs old good experts "glow" for a long time
	Open problem: Other examples of "glowing"
       	 
 	Talk re measuring on-lineness

 	Talk w. more details on Disk Spindown Problem
 	Original Disk Spindown paper 
 	Notes 3
	Information theoretic of relative entropy 
	- motivation of updates and analyses of expert algs
	Online updates for linear regression 
	- and learning linear threshold functions
	Motivation for the GD, EG and EGU
  	Visualizations of relatie entropies    Maple file

4 	Notes 4
	Logistic regression
	Newton updates
	Linearly Least Squares using the SVD decomposition
	Newton type algs for logistic regression

5 	Recall derivation of GD, EG, and EGU in the case of linear regression (Lecture 3)
	Notes 5
  	EG versus GD
	How to prove the regret bounds
	The kernel trick and its limits
	Leaving the span talk 
	Leaving the span paper

6 	Notes 6 Made some corrections
	Optimization
      	- Lagrangians
	- Duality
	How applied to Support Vector Machines
	More on kernels
 	Homework 2 Due Mo 4-27-09 at the beginning of class

7	Bregman divergences, Generalized Pythagorean Thm, Matching Loss, motivation via exponential families
	Matching loss design Thanks Maya!
	Matching loss with piece wise linear transfer function 

8	Rob Schapire's NIPS 07 tutorial on Boosting
	web page with video links
	beautiful paper on Boosting - read by next class
	Talk focused on Game Theory connetion
	Entropy Regularized LPBoost

9	Approximate Newton, Newton with constraints Thanks Karen
	Projection onto the simples 

10	Similarity between Hedge algorithm and Bayes
    	Motivation of Bayes rule
	Hedge and Bayes as Follow the Perturbed Leader
	Notes 10
	KW construction
	Kalai construction

11	Matrix derivative formulas
	Notes 11
	Online PCA talk
	Online PCA paper

12	Hw2 solution, part 1: GD variant of Hedge
	Part 2

	Kernelization with matrix instances

	Homework 3 Due Mo May 25th 5pm

13	PCA, PCA regression, PLS

14	Loss Hedge versus Loss Hedge Notes 14 
	Tuning of eta for Loss Hedge
	Tuning of eta for Gain Hedge maple pdf

15	Mo, May 18 - class cancelled	

16	We, May 20 - David Helmbold teaches class from 4:30-5:30 in the Machine Learning lab, E2-489
	Learning permutations talk paper

	Fr, May 22 - Meeting with all of you at 3pm in ML lab, E2-489
	Discussion of homework and project
	Notes about Dai Fletcher 
	Dai Fletcher paper 

17	Mo, May 25 - Memorial day 
	Homework 3 is due at 5pm

18	Notes 18
	The optimal Hedge alg talk paper more notes

19  	1-2 Ingo Steinwart gives talk
	Engineering 2 Building, Room 375
	Project discussion at 4pm in ML lab
	Bring one-page summary of what you are planning to do

20	Bayesian probability calculus for density matrices talk paper
	Notes 20
	EG+- trick for matrices

 


---------------------------------------------------------