Class:	TTh 2-3:45, E2-506
Office hours:	Mo 10-11, We 11-12 - E2-357
Prerequisite:	CMPS 201, or concurrent enrollment in CMPS 201, or my consent
		some probability theory
		
		
Recommended Textbook by Chris Bishop
"Pattern Recognition and Machine Learning"

Previous CMPS 242 by myself Fall 02
Related Advanced Machine Learning Class CMPS 290C Spring 07


Summary of lectures
1	Notes 1
    	Intro to Machine Learning using curve fitting as an example
	Overfitting, complexity control, regularization
        Experimental setups w. training, validation and test sets

	On-line versus batch
	Definition of regret
	Halfing algorithm and its bound
	Weighted Majority algorithm
	Regret bound for WM via potential function
 	Bug Machine 

	Weighted Majority paper
	
2 	Notes 2
	Randomized Weighted Majority alorithm, Follow the Leader algorithm
	Analysis via potential function
	Talk re. various Share Updates incl. one that induce longterm memory
 	Long term memory paper
 	Original "Tracking the best expert" paper
 	Talk re measuring on-lineness
 	Talk w. more details on Disk Spindown Problem
 	Original Disk Spindown paper 
        
 	Homework 1 Due Th Jan. 17, beginning of class
 	Datasets 

3 	Notes 3
	Information theoretic of relative entropy 
	- motivation of updates and analyses of expert algs
	Online updates for linear regression 
	- and learning linear threshold functions
	Motivation for the GD, EG and EGU
	Logistic regression
  	Visualizations of relatie entropies    Maple file

4 	Notes 4
	More on logistic regression
	Newton updates
	Linearly Least Squares using the SVD decomposition
	Newton type algs for logistic regression

5 	Notes 5
	GD versus EG in the case of linear regression
	Regret bounds w. dual norms
	How to prove the regret bounds
	The kernel trick and its limits
	Leaving the span talk 
	Leaving the span paper

 	Homework 2 Due Tu, Jan. 29, beginning of class 
	Clarified and modified some of the problems
 	Some info about line searches Thanks Maya for scanning it in!

6 	Notes 6 Made some corrections
	Optimization
      	- Lagrangians
	- Duality
	How applied to Support Vector Machines
	More on kernel

7	Finish with Support Vector machines
 	Regularizing logisting regression via clipping&stretching Thanks Dima and Karen!

8	Details re the shrink/stretch alg
 	How moving the labels affects the loss pdf maple
	ROC curve for evaluating a ranking
	ROC curves of perfect and random classifier
	Cross validation
:q
	Bregman divergences, Generalized Pythagorean Thm, Matching Loss, motivation via exponential families
 	Homework 3 Due Tu, Feb. 12, beginning of class 

	Spam data set provided by D. Sculley
 	Visualization of data Ditto permuted  Thanks Nikhila

9	Finish matching loss
	Nodes 9 
	Expert framework with a variety of loss functions paper

10	Finish: Conditional probabilities and Bayes rule
	The expert framework and Bayesian methods
    	Motivation of Bayes rule
   	More about Shrink/Stretch and logistic regression talk 

11 	Notes 11
	ML and MAP estimators
	Naive Bayes and spam application
        filtering spam w. SVMs
        filtering spam w. Naive Bayes

12 	Notes 12 on EM
	Homework 3 reports
 	Homework 4 Due Tu, Feb. 26, beginning of class 
	A problem similar to Problem 2 appears in [Boyd, Vanderbenberghe, p. 228] 

13	Averaging hypotheses can help
	Voted Perceptron
	Rob Schapire's NIPS 07 tutorial on Boosting
	web page with video links
	beautiful paper on Boosting - read by next class

14	More details on Boosting
	Talk focused on Game Theory connetion
	Talk focused on proof techniques via Bregman Projection
	TotalBoost paper
	SoftBoost paper
	Entropy Regularized LPBoost paper With duality proof similar to the one in HW4
	Posted a cleaned up version of above paper

 	Homework 5 Due Th, Feb. 28, beginning of class 

15	Why do relative entropy appear everywhere in nature?
	The blessing and curse of the multiplicative updates
	- Three mechanisms for avoiding the curse
	- Motivating multiplicative updates as relative entropy minimization problems

	For the sake of completeness - here are some of the original papers
	Paper that uses conservative update for learning disjunctions
	Paper that essentially uses lower bounds on 187 the weights
	Paper and talk using capped weights 

	Another application of entropies: Estimating the potential distribution of a species
	Schapire's ICML talk
	Machine learning oriented paper
	More biologically oriented paper

16	Notes 16  Corrected!
	Learning permutations talk paper
	Implementing the fancier Boosting algs based on convex optimization

17	Partial Hw2sols
	The optimal algorithm for the basic expert setting paper partial talk
	Notes 17

18	Applications of Boosting to Dialogue System
	talk by Marylin Walker
	
      
19 	Stock market prediction

20	Variance minimization on the simplex 
	On-line PCA



 


---------------------------------------------------------