CS 512 - Machine Learning
Fall 2009
|
|
|
Mt2 topics: Anything that
was modified between [11/11/2009 – 12/16/2009 ]
= 6-eigen+NNs+7…+8…), however I will not ask more than 10/100pts
worth about SVMs.
Tomorrow (Thursday) we will cover classifier combination.
December 7: I have posted hw 4, under website/hws/
December 3: I have updated 4-backprop-issues to summarize BP
alternatives and include some of the BP alternatives. You are not responsible
of 4-SKIP-BP-alternatives.ppt.
Today we will see
Radial Basis Function networks.
1 December - Syllabus and
Project documents are updated. In particular, Midterm2 will be only 1 hr and 1 week later (on
Dec. 24th), to make up for the lost time. Also, those who want to do
a paper presentation
as a project should select them on a first come first served basis.
5 November:
Hw3 is posted and requires Matlab programming –
due next MONDAY (postponed a bit)
Re:
MT1: Midterm is closed book and I may
give you a little cheat sheet, but only the required information is not very
necessary to memorize (i.e. no cheat sheet for definition of covariance
matrix). I will ask you about finding ML
estimates for certain problems (will use derivation and minimization
knowledge); I will ask you several questions from Bayes
approach (including simple probability calculation)… The exam is up to
(of course not including Neural networks). Depending
on the duration of the exam, I may use the remaining half hour to solve the
exam questions (if it remains).
29
October: Hw2 is posted. Project-2009
document is added (project topic suggestions due Nov 18th, if your
own topic). Happy Republic Day!
-22
October: 1hr lecture on probability and intro. to
Bayesian decision process
-
In the last question of hw1, I had asked you to do the disjunctive question
(Q9) in full and asked how big ..., *in addition*.
But I notice it was not clear.
Please send the full homework by Thursday. (email or
in class is fine). Please see the
explanation
I added for the last question about time
complexity (polynomial time), for non-CS people.
-
HW1 is posted (due
on Oct. 14).
-
Slides is slightly
modified (slide 17) to pose the question asked in class (and give a hint
– its about problem statement/representation and assumptions)
Description:
This is a introductory level course on machine
learning. Topics covered will include theoretical aspects of learning and main
approaches to pattern recognition (Bayesian approaches, Decision trees, NNs,…). The emphasis will be
on what can be learned, rather than how to do it (which is what is covered in
EE566-Pattern Recognition course). Some of the content will be tailored
according to student composition.
This course is especially intended for students working in the area of
pattern recognition or related fields, to deepen their understanding of machine
learning/pattern recognition topics. Students who have already taken
EE566-Pattern Recognition may find a significant material being repeated in
this course (see the syllabus): while there will be a significant overlap, this
course will cover some
topics that were not covered sufficiently in EE566, particularly
theoretical aspects, neural network approaches, support vector machines and
different learning paradigms.
Prereq: none. Undergraduate level Probability
and Linear Algebra helpful. Matlab or other
Toolboxes will be used for homework assignments.
Book:
Machine Learning by T.
Mitchell (ML).
Supplementary book: Introduction
to Machine Learning (Ethem), by Ethem Alpaydin.
Supplementary book: Pattern
Recognition and Machine Learning (PRML), by Christopher Bishop.
We will follow and cover the ML book until the end of Chapter 9.
We will also read important research articles on the topic and some
student presentations/discussions are expected.
Course Schedule: Wed: 10:40-12:30
Thu: 11:40-12:30 in
Instructor: Berrin Yanikoglu (berrin@sabanciuniv.edu, FENS 2056)
Office
Hours: Walk-in.
Grading: Tentative grading: in-class midterm1: %25; in-class midterm2: %25; in-class
final: 30% ; project: 10%; homeworks:
10%
Homeworks may be loosely graded (e.g.
Excellent/Good/OK/Insufficient).
Tentative Syllabus: You may find the lecture slides under Lectures/. In general, they are
sufficient to follow the material, but reading the respective chapters and
looking at more exercises are necessary for making sure you understand
everything.
Note that I consider the lecture
slides as sufficient in terms of content that you should learn in this course
(beware of clearly marked and hidden slides that are extra).
However, you can use any pattern
recognition books (I indicate corresponding chapters next to the week’s
topic) to support your understanding of the lecture material.
Week1: 28 September-2
October
ML1 - Intro to concepts
and course - 2 hrs
Week 2: 5-9 October
ML2 - Concept Learning
Week 3: 12-16 October
ML3 - Decision Trees
Week4:
19-23 October
No class on Wednesday due to meeting
ML6/PRML 1- Bayesian Learning
(Bayes formula, naive Bayes): 1hr
Week5:
26-30 October (29th is a holiday)
ML6/PRML 1 -
Bayesian Learning ctd.
Week6: 2-6 November
Ethem 5/ PRML 1: Parameter Estimation and Multivariate methods
Week7:
9-13 November
Ethem 5/ PRML 1: Parameter Estimation and Multivariate methods – 1hr
ML4 - ANN (also in Ethem
Chp 11 & PRML Chp. 5)
– 2hrs
Week8: 16-20 November
Midterm 1 – 1.5 hrs+ on Wednesday
– no further class
ML4 – Midterm review
+ ANN cont.
Week9: 23-27 November (27 is holiday)
No classes due to extended holiday!
Week10: 30 November-4 December
ML4 - ANN cont.
Week 11: 7-11 December
Lecture slides and
many books: Bias/Variance issue
that was skipped from previous
weeks
Ethem 6, Lecture slides:
Dimensionality reduction introduction (feature selection, extraction, PCA)
Lecture slides, ML8,Ethem8: Non-parametric density estimation (Parzen windows, KNN)
Week 12: 14-18 December
Ethem 7 - Linear Discriminant
Analysis and Support Vector Machines : 2hrs
Ethem15 - Classifier Combination:
1hr
Week 13: 21-25 December
Ethem15 - Classifier Combination:
ctd.
Midterm 2 – 1 hrs on Thursday
- no class
Topics included: NNs onward (what is not covered in Mt1, upto and including SVMs).
Week 14: 28 December-
January 1st (Jan 1st: is holiday)
Ethem14 - Assessing & Comparing
Class. Algorithms
ML7 - Computational learning Theory (PAC learning, VC dimension,…): 2+hrs
Week 15: 4-9 January
Paper/Project presentations
Updated Dec 21,
2009