Probabilistic Graphical Models

Winter 2025 (COMP588)

Administrative

Class: Tuesdays and Thursdays 2:30 PM- 4 PM
Location: ENGMC 103
Office hours: Tuesdays 1:30 - 2:30 PM
TA: Sékou-Oumar Kaba
TA office hours: TBD

Course Description

Graphical models are powerful probabilistic modeling tools. They can model the complex behavior of systems of interacting variables through local relations specified using a graph. These probabilistic models represent the conditional dependencies between subsets of variables in a compressed and elegant form. The graphical models' framework has achieved remarkable success across various domains, from near-optimal codes for communication to the state-of-the-art in combinatorial optimization; these models are widely used in bioinformatics, robotics, vision, natural language processing, and machine learning. In this course, we study both directed and undirected models, exact and approximate inference procedures, and learning methods for complete and partial observations.


Prerequisites:

Familiarity with probability theory and algorithm design is required. Assigments need familiarity with Python and Numpy. Background in AI and in particular machine learning is highly recommended.

Main textbook:

Koller, Daphne, and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.

Further readings:


Course Material:

Assignments, announcements, slides, project descriptions and other course materials are posted on myCourses.

Tentative Outline

Representation
  • Syllabus, review of the probability theory (chapters 2)
  • Bayesian Networks (chapter 3)
  • Markov Networks (chapter 4)
  • Local and Conditional Probability Models (chapter 5)
  • Gaussian Network Models (chapter 6)
Inference
  • Variable Elimination (chapter 7)
  • Junction Trees and Belief Propagation (chapter 10)
  • Variational Inference (chapters 8, 11, 13)
    • Exponential Family and Variational Inference
    • Loopy Belief Propagation and Bethe Free Energy
    • Naive Mean-Field
  • Maximum a Posteriori Inference (chapter 13)
  • Sampling Based Inference (chapter 12)
    • Monte Carlo Inference in Graphical Models
    • Markov Chain Monte Carlo
Learning
  • Overview: Objectives in Learning (chapter 16)
  • Maximum likelihood and Bayesian Estimation in Directed Models (chapter 17)
  • Structure learning in Directed Models (chapter 18)
  • Parameter-Learning in Undirected Models (chapter 20)
  • Learning with Partial Observations (chapters 19)
  • Causality (chapters 21)

Evaluation

Evaluation will be based on the following components (see myCourses for details):
- Assignments (40%)
- Project (50%)
- Class participation (10%)


Assessments in this course are governed by the Policy on Assessment of Student Learning (PASL), which provides a set of common principles to guide the assessment of students’ learning. Also see Faculty of Science-specific rules on the implementation of PASL.

Accommodations

Legally mandated academic accommodations are handled by Student Accessibility and Achievement. For more information see https://www.mcgill.ca/access-achieve/

Academic considerations for assessments that are missed or late for valid reasons will be provided at the instructor's discretion. As per the new Quebec guidelines, medical notes are not required for absences of less than 5 days. Note that repeated, similar requests for academic considerations in this course are unlikely to be granted.

Language of Submission

In accord with McGill University’s Charter of Students’ Rights, students in this course have the right to submit in English or in French written work that is to be graded. This does not apply to courses in which acquiring proficiency in a language is one of the objectives.” (Approved by Senate on 21 January 2009) Conformément à la Charte des droits de l’étudiant de l’Université McGill, chaque étudiant a le droit de soumettre en français ou en anglais tout travail écrit devant être noté, sauf dans le cas des cours dont l’un des objets est la maîtrise d’une langue. (Énoncé approuvé par le Sénat le 21 janvier 2009)

Academic Integrity

McGill University values academic integrity. Therefore, all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the Code of Student Conduct and Disciplinary Procedures (Approved by Senate on 29 January 2003) (See McGill’s guide to academic honesty for more information).

In the event of extraordinary circumstances beyond the University’s control, the content and/or assessment tasks in this course are subject to change and students will be advised of the change.