RBE 4744: Deep Learning for Perception

This course exposes the students to the mathematical foundations of deep learning applied to images. Perception stacks in state-of-the-art robots are rapidly adapting to the latest advancements in deep learning due to their efficacy and high accuracy. These deep learning-based methods are also accelerable using parallelized hardware such as GPUs that can enable low latency operations of complex tasks such as real-time scene segmentation. The students will be trained in formulation, development, and implementation of deep learning solutions for common computer vision problems in the context of robot perception. The course will cover advanced and state-of-the-art topics such as sim2real, adversarial attacks on neural networks, vision transformers, and diffusion models. Additional topics explored in this course include image formation, linear classifiers, neural networks and backpropagation, Convolutional Neural Networks (CNNs), CNN architectures, data generation for
sim2real, black-and white-box attacks on neural networks as applied to build state-of-the-art robotic stack. Students will gain knowledge about the considerations required to enable a robotic system with the state-of-the-art deep learning toolkit. The course is designed to balance theory with applications through projects.