CS412 \ CS 512  Machine Learning
Fall 2017
Description:
This is a double coded (undergraduate/graduate) course on machine learning, shortly called ML from now on. Topics covered will include theoretical aspects of learning and main approaches to pattern recognition (Bayesian approaches, Decision trees, NNs,…).
At the end of the course, students should be able to:
 Describe different learning paradigms.
 Discuss the goal, assumptions and limitations of inductive learning paradigm.
 List and compare fundamental machine learning techniques, in terms of their strengths and shortcomings.
 Develop several standard machine learning algorithms using a programming environment including Matlab.
 Utilize model selection methods for selecting a learner with the right complexity.
 Analyze and compare algorithms using unbiased test methods.
ML can be taken before CS404 –AI without any problems; they overlap in about 15%
topic.
Prereq. And Requirements: For undergraduates. there is a 4^{th} year requirement (roughly 90 credits) so that there is a certain academic background. No formal prerequisites are listed, but you must know undergraduate level Probability and Linear Algebra. Matlab or other Toolboxes will be used for homework assignments. Also undergraduates are required to attend the lectures (except for 45 lecture hours total in the year).
Course Schedule: Tuesday 10:4012:30, Thursday 2:403:30 Recitation for undergrads (and interested grads – space permitting): Thursday 4:405:30
Instructor: Berrin Yanikoglu (berrin@sabanciuniv.edu, FENS 2056)
Recitation: Recitation is mainly for undergraduate
students but attendance is not required.
It is mainly for going over last week's material (in weeks that may be
necessary or just before exams) and for teaching software and handson tools.
It will be every other week (unless there is demand for more) and will be
announced after/at that week’s lectures. Graduates who have questions about the
material can also go if they wish, but the recitation hour is not designed for advanced questions.
Teaching Assistant: Figen Beken Fikri (fbekenfikri@sabanciuniv.edu)
Office Hours: Friday 9:0010:30 (email for appointment if you cannot attend those hours). Please do not call on the phone. The office hours are not for full fledged tutoring, but short help on material, asking help about homeworks, and for raising your requests about the course etc…
Book: Main Book: Course slides and reading material
Supplementary book: Introduction to Machine Learning (Ethem), latest edition by Ethem Alpaydin. (basic  first edition is OK to have if you have that one)
Supplementary book: Machine Learning by T. Mitchell (ML).
Supplementary book: Pattern Recognition and Machine Learning (PRML), by Christopher Bishop (advanced)
We will follow and cover most chapters of the
Ethem book. In general, the slides will be sufficient to follow the material
but reading the respective chapters and looking at more exercises may be needed
to better understand the material. You can use any pattern recognition books (I
indicate corresponding chapters next to the week’s topic) to support your
understanding of the lecture material. While using the
slides, beware of clearly marked slides and hidden slides that are
uncovered/extra material. They will not show in slideshow mode and you can
choose to exclude the printing of hidden slides from the printing dialog.
Grading: Each
midterm: 25%; inclass final: 25%; group project: 10% & homeworks: 15% (some
hws may be in the form of very small quizzes, about 3 per term at most)
o
You can miss up to one midterm (no proof is
required), in which case the weight of the final will increase (to 50%). If you
miss two midterms, you get an F.
o
To encourage exam participation, some brownie points will be given to those
who attend both midterm exams (otherwise those taking a good grade in Mt1 does
not attend Mt2). For instance, I do consider Mt attendance if someone is at the
border of a grade cluster.
o If you miss the final exam, you MUST have a
legitimate excuse/report.
o
To pass the course you grade as calculated above must be at
least 35 (strict) and Final grade should be above 30/100.
TENTATIVE Syllabus: Can be found below.
Things you are expected to know:
Mt1 Topics:
· Machine learning concepts (incl. diff. learning problems,overfitting, train and generalization errors, error measures,…);
· Decision tree learning (capabilities,attribute selection,issues with,…);
· Entropy
· Concept learning (esp. what is a concept; counting instance and hypothesis spaces, and understanding inductive bias);
· Probability (chain rule, independence, conditional independence, prior, posterior, joint vs marginal distributions, being able to formulate and solve problems…)
· Bayes formula; Bayesian Classification (Bayes optimal and Naïve Bayes, error minimization, general loss function,…);
Mt2 Topics: Will include basics, in addition to the new material after MT1.
· Machine learning concepts (incl. diff. learning problems,overfitting, train and generalization errors, error measures,…);
· Probability (chain rule, independence, conditional independence, prior, posterior, joint vs marginal distributions, being able to formulate and solve problems…)
· Bayes formula; Bayesian Classification (Bayes optimal and Naïve Bayes, error minimization, general loss function,…);
· Parametric methods
· Multivariate Normal Distribution (mean,variance, covariance, correlation, properties of, use in Bayesian decision,…)
Final Topics: All topics included.
In all the
exams, questions from a topic will have roughly as much points as the amount
given to that topic in class (i.e. I won’t ask about something that was not at
all covered in class, or won’t assign too much weight for something that was
barely covered.)
Table 1: Expected syllabus.
Topic 
CS412/CS512 
EE566 
Introduction 
Week 1: One week 

Decision Trees
(Ethem 9/ML 3) 
Week 2: One week 

Concept
Learning (ML 2) 
Week 3: One week 

Probability
Review 
Week 4 (1hr) 

Bayesian
Decision Theory (Ethem 3/ML 6) 
Week 45: One week 

Parametric
Methods (Ethem 4) 
Week 6: One week 


Week 6 or 7 – Midterm 1


Multivariate methods (Ethem 5) 
Week 8: One week 

Syntactic
Pattern Recognition 
X 

Nonparametric
approaches (kNN) 
1 hr 

Linear Discriminant
Functions 
X 

Neural Networks
(Ethem 11, ML 4) 
Week 910: 1.5 weeks 

Preprocessing
and Dimensionality reduction 
Week 10 (1hr)  Basic concept 

Learning Theory
(lecture notes) 
Week 11: One week 

Unsupervised Learning and
Clustering 
X 

Combining
Classifiers (Ethem17) 
Week 12: One week 

Hidden Markov
Models 
X 


Week 12 or 13 –Midterm 2 

Support Vector
Machines (Ethem 13) 
Week 13: 1hr 

Group (23 people) Project presentations 
Week 14 

Assessing &
Comparing Classification Algorithms 
X 

Reinforcement
Learning 
X 

Graphical
Models 
X 
