COMP 3711H - Honors Design and Analysis of Algorithms
Spring Semester 2023-24
Number of Students: 70
Average Rating by the Students: 4.95/5.0
YouTube Playlist
https://www.youtube.com/watch?v=irzrLdt7V5U&list=PLzZlJT-UOiwD1OSqOs00D0iW93BSJPjBR
Course Files
https://drive.google.com/drive/folders/134ayXCVlOXNQHJznJ4JSQOt24d71uJ2e?usp=sharing
Notes of the first 16 lectures
https://drive.google.com/file/d/1sdARQ6BtpxuPfKIkWsAe35mADBdUaDt6/view?usp=drive_link
Thanks to Zhang Tianrui for preparing these notes.
Course syllabus
Instructor:
Amir Goharshady
Office: CYT 3006
Office Hours: Please make appointments by email
Guest Lecturer:
Dr Harshit Motwani
Office Hours: Please make appointments by email
Teaching Assistants:
Chun Kit "John" Lam
Kerim Kochekov
Office Hours: Please make appointments by email
Times and Venues:
Wednesdays, 15:00 - 16:20, Room 5583, Lifts 27-30
Thursdays, 15:00 - 15:50, Room 2404, Lifts 17-18
Fridays, 15:00 - 16:20, Room 5583, Lifts 27-30
Every session is a mixture of lecture and lab/tutorial.
Guest lectures and extra problem-solving sessions will be scheduled on zoom separately. They will be recorded for students who cannot attend.
Recommended Textbooks:
Introduction to Algorithms (3rd ed)
Cormen, Leiserson, Rivest and Stein (CLRS). MIT Press
E-version available from the university library
Jeff Erikson (free book)
Dasgupta, Papadimitriou, and Vazirani. McGraw Hill.
Bentley. Addison Wesley
Kleinberg and Tardos. Addison Wesley
Problems on Algorithms (2nd ed)
Ian Parberry and William Gasarch (free book)
Acknowledgments:
Much of the syllabus and material, especially slides, were prepared by Prof. Mordecai J. Golin, who kindly agreed to share it with us.
Grading:
Homeworks: 4 x 10%
You will have a chance to resubmit each homework
Midterm Exam: 30%, pen and paper
Time: Friday, 22 March 2024, 19:30-22:30
Place: CYT-LTL
Final Exam: 30%, pen and paper
Time: Thursday, 23 May 2024, 08:30-11:30 am
Place: LG1 Table Tennis Room
Required Coding Practice: 0%
These are required and missing any of them leads to a grade of F
Policy on Plagiarism:
Students are expected to follow the HKUST Academic Honor Code.
All files submitted by the students must be 100% their own individual work.
All work submitted for grading, e.g. assignments and codes, must be your own. You are permitted to discuss problems with other students but you must write-up all solutions by yourself, in your own words. If you got the main idea for a solution from another student or a web-site you must acknowledge that source in your submission.
You are not allowed to copy code from any source, even if you acknowledge it.
Submission of non-acknowledged material will be considered as plagiarism.
You are not allowed to use ChatGPT or similar tools.
We have a zero-tolerance policy on plagiarism. The minimum penalty is a grade of F for the whole course.
Important notes on how to answer questions that involve designing an algorithm: Your solution should consist of all of the following 4 components:
the main ideas of your algorithm (This will help you gain partial credits if your algorithm is not completely correct),
pseudocode or plain language description of the algorithm (must be detailed and exact, to a level that allows implementation with ease),
a tight asymptotic analysis of the running time of your algorithm, and
a proof of correctness for your algorithm (especially for greedy and graph algorithms)
The proofs must be written rigorously and with consistent mathematical notation.
A secondary benefit of working on the assignments is to prepare for the exams, because exam questions are often variants of assignment problems. For this reason we encourage you to spend time alone thinking about each problem and your approach in solving it. You are allowed to and encouraged to discuss assignment problems with your classmates. However, you must write up the solutions on your own. In particular:
Assignment solutions must be written in your own words (not copied or modified from someone else's write-up). If you see someone else's write-up, that's already plagiarism.
You must understand your solution and its derivation. (I may randomly ask you to explain your solution to me.)
You must explicitly acknowledge in writing in the assignment your collaborators (whether or not they are classmates) or any other outside sources on each assignment.
Failing to do any of these will be considered plagiarism, and will result in a failing grade in the course and notification for appropriate disciplinary action.
LaTeX Requirement:
All written assignments must be submitted as high-quality pdf files generated by LaTeX. We do NOT accept any other formats. Handwritten files or files generated by other typesetting software will be rejected without grading. If you are not familiar with LaTeX, https://www.overleaf.com/ is a great resource.
Programming Language:
This course uses standard C++ as its sole programming language. All codes must be submitted in .cpp format and will be compiled using the latest version of g++ on Ubuntu.
Lecture 1: Runtime Analysis (31 January 2024)
Tip: Try to code the algorithms discussed in each lecture on your own (without relying on the video). You are ready to move to the next lecture only if you can do this comfortably.
Lecture 2: Merge Sort (1 February 2024)
Lecture 3: Heaps and Heap Sort (2 February 2024)
Required Reading: Chapter 6 of CLRS
Recommended to Watch: An example of Heap Sort (not the variant we covered)
TA Session 1: Counting Inversions (6 February 2024)
Lecture 4: Quick Sort, Quick Select and Median (7 February 2024)
Lecture 5: Solving Recurrences (8 February 2024)
Required Reading: Chapter 4 of CLRS
Recommended to Watch: Solving Recurrences (Great lecture, terrible video quality)
Lecture 6: Radix Sort and Binary Search Trees (9 February 2024)
Lecture 7: Implementing Binary Search Trees (14 February 2024)
Required Reading: Chapter 12 of CLRS
Recommended Reading: Chapter 14 of CLRS (When you see Red-black trees, just think of BSTs)
Lecture 8: AVL Trees (15 February 2024)
Recommended Reading: Chapter 13 of CLRS
Lecture 9: Treaps and Randomized Quick Sort (16 February 2024)
Coding Practice 1
Link to the problemset (Username: guest, Password: guest)
Lecture 10: Divide and Conquer (21 February 2024)
Lecture 11: Greedy Algorithms (22 February 2024)
TA Session 3: Two Pointers (27 February 2024)
Lecture 13: DFS and BFS (28 February 2024)
Lecture 14: Topological Sort (29 February 2024)
Required Reading: Chapter 22 of CLRS
Lecture 15: Strongly Connected Components and Dijkstra's Algorithm (1 March 2024)
Important: See Lecture 19 for a correction regarding cross edges
Required Reading: Section 24.3 of CLRS
Recommended to Watch: Tarjan's Strongly Connected Components Algorithm
Lecture 16: Let's Code Dijkstra (1 March 2024)
Coding Practice 2
Link to the problemset (Username: guest, Password: guest)
TA Session 4: C++ Standard Library (5 March 2024)
Homework 1
Lecture 17: Disjoint Sets with Union (6 March 2024)
Required Reading: Chapter 19 of CLRS
Lecture 18: Graph Theory Background (6 March 2024)
Recommended to Read: Chapters 1 and 2 of West's Introduction to Graph Theory (available at the HKUST library)
Lecture 19: Let's Code Strongly Connected Components (7 March 2024)
Lecture 20: Minimum Spanning Trees (7 March 2024)
Lecture 21: Let's Code DFS Times, Topological Sort and Kosaraju's SCC Algorithm (7 March 2024)
Lecture 22: Dynamic Programming, Knapsack and Bellman-Ford (8 March 2023)
Required Reading: Chapter 22 of CLRS
Recommended to Watch: A Visualization of the Bellman-Ford Algorithm
Question: What if there are infinitely many copies of each item in the Knapsack problem? In other words, what if we do not just choose whether to include an item or not, but also how many copies to take?
TA Session 5: Minimum Spanning Trees (12 March 2024)
Lecture 23: All-pairs Shortest Path, Johnson and Floyd-Warshall (13 March 2024)
Required Reading: Chapter 23 of CLRS (All-pairs Shortest Paths)
Recommended to Watch: Dynamic Programming - All-pairs Shortest Paths
Recommended to Watch: Floyd-Warshall All-Pairs Shortest Paths
Lecture 24: Dynamic Programming on Strings (14 March 2024)
Lecture 25: Trees (14 March 2024)
Recommended to Read: Chapter 3 (Trees) of West's Introduction to Graph Theory
Lecture 26: Prefix Sums and Dynamic Programming on Trees (15 March 2024)
Practice Midterm Exam (18 March 2024)
Use this practice exam to simulate a real exam. Print it out and work on it for 2.5 hours.
TA Session 6: Dynamic Programming (19 March 2024)
Lecture 27: Tree Diameter, Edit Distance and Set Cover (20 March 2024)
Lecture 28: Dynamic Programming on Subsets (21 March 2024)
Lecture 29: Lowest Common Ancestor and Segment Tree (22 March 2024)
Midterm Exam (22 March 2024)
TA Session 7: Let's Solve Coding Practice 1 (25 March 2024)
Coding Practice 3
Link to the problemset (Username: guest, Password: guest)
Lecture 30: Segment Trees and Binary Indexed Trees (Fenwick Trees)
Homework 2
Homework 3
Coding Practice 4
Link to the problemset (Username: guest, Password: guest)
Lecture 31: Let's Code Prim, Kruskal and Borůvka's Algorithms for Minimum Spanning Tree (1 April 2024)
Lecture 32: Let's Code Bellman-Ford and Floyd-Warshall (1 April 2024)
TA Session 9: Segment Trees (9 April 2024)
Lecture 33: Amortized Analysis and Binomial Heaps (10 April 2024)
Required Reading: Chapter 17 of CLRS (Amortized Analysis)
Required Reading: Sections 7.1 to 7.5 of these lecture notes by Damon Wischik
Recommended to Watch: Binomial Heap (This is in Hindi! but it has English subtitles)
Lecture 34: Ternary Search Trees and Tries (11 April 2024)
Lecture 35: Suffix Trees and Ukkonen's Algorithm (12 April 2024)
TA Session 10: Let's Solve Coding Practices 2 and 3 (16 April 2024)
Lecture 36: Pattern Matching using Rabin-Karp and Knuth-Morris-Pratt (17 April 2024)
Required Reading: Chapter 32 of CLRS (String Matching)
Required Reading: String Matching Appendix to Erickson's Book
Lecture 37: Centroid Decomposition and Heavy-Light Decomposition (18 April 2024)
Lecture 38: Sparse Tables and Suffix Arrays (19 April 2024)
Required Watching: Suffix Arrays Tutorial
You should sign up on codeforces, go to EDU and sign up in the ITMO academy course for the link to work. Make sure you watch the videos of all 5 steps. I strongly recommend solving the problems, too.
Recommended to Watch: The Suffix Tree is in the Suffix Array
Homework 4
TA Session 11: Hints on Writing and Homeworks (13 April 2024)
Lecture 39: Maximum Flow and Ford-Fulkerson (24 April 2024)
Required Reading: Chapter 26 of CLRS (Maximum Flow)
Lecture 40: Maximum Flow Algorithms of Ford-Fulkerson, Edmonds-Karp and Dinitz (25 April 2024)
Lecture 41: Combinatorial Games, Nim and the Sprague-Grundy Theorem (26 April 2024)
Second Midterm Exam
This exam was for students who had missed the midterm exam due to an acceptable reason.
TA Session 12: Let's Solve Coding Practices 3 and 4 (30 April 2024)
Lecture 42: Halting and Finite Automata (2 May 2024)
Lecture 43: Regular Languages and Turing Machines (3 May 2024)
TA Session 13: Let's Solve Coding Practice 4 (7 May 2024)
Lecture 44: P, NP and Reductions (8 May 2024)
Lecture 45: NP-complete Problems (9 May 2024)
Lecture 46 - NP-completeness of Knapsack and 3-colorability (10 May 2024)
The following items are recommended to students who are interested to learn more about algorithms. They are not part of the syllabus of this course and will not appear in the final exam.
Recommended to Read: Randomized Algorithms by Motwani and Raghavan
Recommended to Watch: HKUST COMP 5711 (Advanced Algorithms) Lectures