CS412 \ CS 512  - Machine Learning

Fall 2017

     Description:  

This is a double coded (undergraduate/graduate) course on machine learning, shortly called ML from now on. Topics covered will include theoretical aspects of learning and main approaches to pattern recognition (Bayesian approaches, Decision trees, NNs,…).

At the end of the course, students should be able to:

- Describe different learning paradigms.

- Discuss the goal, assumptions and limitations of inductive learning paradigm.

- List and compare fundamental machine learning techniques, in terms of their strengths and shortcomings.

- Develop several standard machine learning algorithms using a programming environment including Matlab.

- Utilize model selection methods for selecting a learner with the right complexity.

- Analyze and compare algorithms using unbiased test methods.

ML can be taken before CS404 AI  without any problems; they overlap in about 15% topic.

Prereq. And Requirements:  For undergraduates. there is a 4th year requirement (roughly 90 credits) so that there is a certain academic background. No formal prerequisites are listed, but you must know undergraduate level Probability and Linear Algebra. Matlab or other Toolboxes will be used for homework assignments. Also undergraduates are required to attend the lectures (except for 4-5 lecture hours total in the year).

Course Schedule:  Tuesday 10:40-12:30, Thursday 2:40-3:30   Recitation for undergrads (and interested grads – space permitting): Thursday 4:40-5:30

Instructor: Berrin Yanikoglu (berrin@sabanciuniv.edu, FENS 2056)

Recitation: Recitation is mainly for undergraduate students but attendance is not required. It is mainly for going over last week's material (in weeks that may be necessary or just before exams) and for teaching software and hands-on tools. It will be every other week (unless there is demand for more) and will be announced after/at that week’s lectures. Graduates who have questions about the material can also go if they wish, but the recitation hour is not designed for advanced questions.

Teaching Assistant: Figen Beken Fikri (fbekenfikri@sabanciuniv.edu)

Office Hours: Friday 9:00-10:30 (email for appointment if you cannot attend those hours). Please do not call on the phone. The office hours are not for full fledged tutoring, but short help on material, asking help about homeworks, and for raising your requests about the course etc…

Book:        Main Book: Course slides and reading material

                 Supplementary book:  Introduction to Machine Learning (Ethem), latest edition by Ethem Alpaydin. (basic - first edition is OK to have if you have that one)

Supplementary book: Machine Learning by T. Mitchell (ML).

Supplementary book: Pattern Recognition and Machine Learning (PRML), by Christopher Bishop (advanced)

We will follow and cover most chapters of the Ethem book. In general, the slides will be sufficient to follow the material but reading the respective chapters and looking at more exercises may be needed to better understand the material. You can use any pattern recognition books (I indicate corresponding chapters next to the week’s topic) to support your understanding of the lecture material. While using the slides, beware of clearly marked slides and hidden slides that are uncovered/extra material. They will not show in slideshow mode and you can choose to exclude the printing of hidden slides from the printing dialog.

Grading:   Each midterm: 25%; in-class final: 25%; group project: 10% & homeworks: 15% (some hws may be in the form of very small quizzes, about 3 per term at most)

o   You can miss up to one midterm (no proof is required), in which case the weight of the final will increase (to 50%). If you miss two midterms, you get an F.

o   To encourage exam participation, some brownie points will be given to those who attend both midterm exams (otherwise those taking a good grade in Mt1 does not attend Mt2). For instance, I do consider Mt attendance if someone is at the border of a grade cluster.

o   If you miss the final exam, you MUST have a legitimate excuse/report.

o   To pass the course you grade as calculated above must be at least 35 (strict) and Final grade should be above 30/100.

TENTATIVE Syllabus: Can be found below.

Things you are expected to know:

Mt1 Topics:

·      Machine learning concepts (incl. diff. learning problems,overfitting, train and generalization errors, error measures,…);

·      Decision tree learning (capabilities,attribute selection,issues with,…);

·      Entropy

·      Concept learning (esp. what is a concept; counting instance and hypothesis spaces, and understanding inductive bias);

·      Probability (chain rule, independence, conditional independence, prior, posterior, joint vs marginal distributions, being able to formulate and solve problems…)

·      Bayes formula; Bayesian Classification (Bayes optimal and Naïve Bayes, error minimization, general loss function,…);

Mt2 Topics:  Will include basics, in addition to the new material after MT1.

·      Machine learning concepts (incl. diff. learning problems,overfitting, train and generalization errors, error measures,…);

·      Probability (chain rule, independence, conditional independence, prior, posterior, joint vs marginal distributions, being able to formulate and solve problems…)

·      Bayes formula; Bayesian Classification (Bayes optimal and Naïve Bayes, error minimization, general loss function,…);

·      Parametric methods

·      Multivariate Normal Distribution (mean,variance, covariance, correlation, properties of, use in Bayesian decision,…)

Final Topics:  All topics included.

In all the exams, questions from a topic will have roughly as much points as the amount given to that topic in class (i.e. I won’t ask about something that was not at all covered in class, or won’t assign too much weight for something that was barely covered.)

 

      Table 1: Expected syllabus.

Topic

CS412/CS512

EE566

Introduction

Week 1: One week

 

Decision Trees (Ethem 9/ML 3)

Week 2: One week

 

Concept Learning (ML 2)

Week 3: One week

 

Probability Review

Week 4 (1hr)

 

Bayesian Decision Theory (Ethem 3/ML 6)

Week 4-5: One week

 

Parametric Methods (Ethem 4)

Week 6: One- week

 

 

Week 6 or 7 – Midterm 1  

 

Multivariate  methods (Ethem 5)

Week 8: One week

 

Syntactic Pattern Recognition

X

 

Nonparametric approaches (kNN)

1 hr

 

Linear Discriminant Functions

X

 

Neural Networks (Ethem 11, ML 4)

Week 9-10: 1.5 weeks

 

Preprocessing and Dimensionality reduction

Week 10 (1hr) - Basic concept

 

Learning Theory (lecture notes)

Week 11: One week

 

Unsupervised Learning and Clustering

X

 

Combining Classifiers (Ethem17)

Week 12: One week

 

Hidden Markov Models

X

 

 

Week 12 or 13 –Midterm 2

 

Support Vector Machines (Ethem 13)

Week 13: 1hr

 

Group (2-3 people) Project presentations

Week 14

 

Assessing & Comparing Classification Algorithms

X

 

Reinforcement Learning

X

 

Graphical Models

X