During summer school, we will discuss quite advanced topics of Bayesian Deep Learning. The school lasts only 6 days, and we do not have enough time for a detailed introduction into basic Bayesian Methods and Deep Learning. That's why we expect you to come with the knowledge of some basic concepts.

Below we present a checklist of concepts which you have to be familiar with before the summer school. We also provide links to the materials on these topics. We recommend studying the materials on concepts you are not familiar with. Otherwise, it will be hard for you to follow the school lectures.

Although there are a few intro lectures on Bayesian approach in the program, they are not enough to understand the specifics of this approach if it is your first acquaintance. But they will be helpful to understand the connections between methods if you studied the materials in advance.

1. Bayesian Methods
   • Concepts
     1. Bayesian reasoning v. s. frequentist reasoning
     2. Conjugate priors and analytical Bayesian inference
     3. EM-algorithm
     4. Variational inference, mean-field approximation, and conditional conjugacy
   • Materials
     1. First 3 weeks of Bayesian Methods for Machine Learning Course on Coursera.
     2. Or, if you prefer reading books, chapters 1, 2, 3, 9, 10 of Pattern Recognition and Machine Learning by Christopher Bishop
     3. Or, if you prefer concise lecture notes of university courses, we'd recommend this course or this course.
   • To test your knowledge, try solving Bayesian methods problems in this document.
2. Deep Learning
   • Concepts
     1. Computational graphs, deep understanding of backpropagation, stochastic gradient descent
     2. Fully-connected, convolutional, recurrent neural networks, their popular architectures (AlexNet, VGG, ResNet, LSTM)
     3. Regularization in deep learning: weight decay, dropout, batch normalization
     4. Generative adversarial networks
   • Materials
     1. Lectures 3-11 of Stanford's CS231n: Convolutional Neural Networks for Visual Recognition (see Syllabus with materials and Videos)
     2. Or, if you prefer reading books, Chapters 6-10 and 20.10.4 of Deep Learning Book
     3. Or first 4 videos from Introduction to Deep Learning by MIT.
   • To test your knowledge, try solving Deep Learning problems in this document.
3. PyTorch
   • Concepts
     1. Tensors, autograd and implementing custom algorithms using basic operations
     2. nn.Module and how it helps to implement any computational graph
     3. Constructing and training neural networks
   • Materials
     1. Parts of an official PyTorch tutorial: Quick 60-min intro, Learning PyTorch with examples, What is torch.nn really?
     2. Or, if you prefer tutorials in Jupiter notebooks where you write missing code, we suggest solving Part Q5 (PyTorch version) of Stanford's assignment.
   • To test your knowledge, try implementing Multi-layer perceptron on the Boston dataset yourself in low-level PyTorch using nn.Module, basic mathematical operations (like matrix multiplication or random tensor initialization) and autograd.

We also recommend taking a look at a the Matrix Cookbook. It is a good collection of useful mathematical facts that will be useful at the summer school. You should understand basic concepts from this book (basic operations with matrices, matrix derivatives, properties of distributions etc).

There are two types of classes at the summer school: lectures and practical sessions. For the practical sessions, please bring your laptop with python 3.7, pytorch (version 1.1) and torchvision installed. For python 3.7, we recommend installing an Anaconda distribution.

Also, please download a MNIST dataset before the school. We recommend running this script to test your installation, and the script will also download MNIST using torchvision wrapper (in the same manner as we will load data at the school).

School materials (lectures and assignments) will be available in the GitHub repository.

• Red Brick Hotel Presnya (Ulitsa Rodchelskaya 15 building 22)
• Winewood Moscow Lounge Hotel (Ulitsa Krasnaya Presnya 30 building 3)
• Chez Michel Hotel (13 Krasnaya Presnya Street, 5 floor)
• MiniHotel Brusnika na Khoroshevskom (Khoroshevskoye Shosse, 23/2)

• Rassvet Hostel (Krasnaya Presnya 32-34 Stroyeniye 3)
• GetCapsule Hostel (Bol'shoy Tishinskiy pereulok, dom 8 apartment 1)

• maps: (online app available, includes online transport monitoring), 2gis.ru (offline app available), maps.google.com
• interactive metro map (offline app available)
• taxi (with apps): Uber, taxi.yandex.ru, Gett
• getting from and to airports: Aeroexpress
• sightseeing: Trip Advisor, Hop on Hop off Moscow
• accommodation: booking.com, Airbnb

• Please check the weather forecast before the school. A warm sweater may be needed.
• Bank cards are accepted almost everywhere, including public transport
• To get from the airport, you can use Aeroexpress, taxi (see above) or bus. When called using a mobile app, the taxi may cost not much more than Aeroexpress. The bus goes to the nearest metro station, and it is the cheapest option.
• To get cheaper tickets for public transport, buy Troika card. Find more information about tickets here.
• Do not forget to purchase medical insurance.
• Mobile data plans are very cheap in Russia compared to the rest of the world. If you'd like to get yourself a local SIM card, this guide might be useful.