Computer Vision and Deep Learning

(2019 Undergraduate Course)

Instructor: Prof. Mu Yadong (email:
Location: Room 205, Teaching Building 3
Time: Friday 15:10pm - 17:00pm (every week)
Office Hour: Thursday 4-6pm, Friday 10-12am, ICST Peking University (128 Zhong-Guan-Cun North Road)
Feb 22, 2019 Introduction - I
  • Course Introduction
  • Introduction to Computer Vision: illustrative applications and demos
March 1, 2019 Introduction - II
  • Introduction to Deep Learning: history, key concepts, back-propagation, neural layers etc.
March 8, 2019 Topic: Visual Recognition - I
  • Visual Recognition: Task Definition and Challenges
  • Visual Features: Harris Corner, SIFT, MSER, HOG etc.
  • Bag-of-words Models
  • Spatial Pyramid Matching
  • Pyramid Match Kernel
  • Vocabulary Tree
  • Sparse Coding
March 15, 2019 Topic: Visual Recognition - II
  • Deep Learning for Visual Recognition: LeNet-5, AlexNet, VGG-16, GoogleNet, ResNet
  • Network Visualizatioin
March 22, 2019 Topic: Object Detection - I
  • V-J Face Detector (Integral Image, AdaBoost, Cascade)
  • HOG+SVM with NMS
  • Deformable Part Model (DPM) for Pedestrian Detection
March 29, 2019 Topic: Object Detection - II
  • R-CNN
  • Fast R-CNN
  • Faster R-CNN
  • R-FCN
  • Multi-Scale R-CNN
  • Feature Pyramid Network
April 12, 2019 Topic: Pixel Computing - I
  • Pixel Labeling: Segmentation, Matting, Parsing
  • Unsupervised Image Segmentation: K-means, Mean-Shift, Normalized Cut
  • Interactive Object Cutout: GraphCut, GrabCut, LazySnapping
  • Image Matting: Poisson Matting, Closed-Form Matting, Robust Color Sampling
  • Image Co-segmentation
  • Image Inpainting / Image Completion
  • Scene Parsing: Sparse Coding
April 26, 2019 Topic: Pixel Computing - II
  • Deep Pixel Labeling: FCN, DeepLab, SegNet, CNN-as-RNN
  • Human Pose Estimation: Bottom-Up and Top-Down
May 10, 2019 Topic: Video Computing
  • Introduction of Video Computing Tasks
  • Video Features (STIP, Deep Video, C3D, Trajectory Feature)
  • Deep Learning for Video Classification (multi-stream fusion techniques)
  • Video Event Detection and Action Detection
  • An Illustrative System for Video Classification
May 17, 2019 Topic: Reccurent Deep Networks
  • Unrolling Computational Graph
  • RNN variants (recurrent through output, sequence-input-single-output, teaching forcing, encoder-decoder, bi/quad-directional RNN etc.)
  • Generative RNN modeling
  • Back propagation through time (BPTT)
  • The issue and remedy for long-term dependency in RNN
  • Long short-term memory (LSTM)
  • Applications (image captioning, convLSTM for rainfall prediction, social LSTM)
May 24, 2019 Topic: Invited Talks
  • Wang Changhu (ByteDance)
  • Yu Gang (Megvii)
May 31, 2019 Topic: Subspace Learning and Image Hashing
  • Dimension Reducation: PCA, CCA, Fisher LDA, EigenFace, LDA-Face
  • Nonlinear Methods: MDS, ISOMAP, LLE
  • LPP, graph embedding
  • Locality-Sensitive Hashing: the concept and proof of sublinear complexity in the STOC98 paper
  • LSH schemes for cosine similarity, Jaccard index (minHash)
  • Applications of LSH in computer vision
TBA Course Project Presentation - I TBA
TBA Course Project Presentation - II TBA