Skip to main content

Essential Reads

There are three aspects that are essential to our group---in no particular orders: (i) machine learning (theory and engineering), (ii) mathematical foundations, (iii) interdisciplinary knowledge. Additionally, (iv) meta knowledge (e.g., general self-improvement) should not be neglected.

Machine Learning​

Throughout your study, you must strive to keep improving your knowledge about:

  • ML foundations: Read Bishop, Murphy, and Bach.
  • ML trends: Read papers daily. Use, e.g., https://scholar-inbox.com.
  • ML engineering: Keep improving your PyTorch/JAX skills, and keep honing your general software engineering skills.

Must read ML books​

In this order:

  1. Bishop's Deep Learning (cover-to-cover)
  2. Garnett's Bayesian Optimization (at least Chapter 1)
  3. Agustinus' monograph on decision-making under uncertainty

Skim-is-fine ML books​

Useful as references. But you need to skim them at least once, unless your research topic requires them.

tip

Agustinus has most of the ML books listed above. Feel free to borrow!

Maths​

You must always aim to improve your mathematical maturity. Keep learning new (more and more advanced) math subjects. A possible path below. (This is the path that Agustinus took starting from his MSc. Note that he didn't study intensive math in his software engineering undergrad!)

Assumption

Good knowledge in linear algebra, multivariable calculus, and applied (non-rigorous) probability or stats.

  1. Start with Real Analysis. It can be seen as "rigorous calculus" and is useful to start building up maturity. I used Understanding Analysis book by Abbott, which I think is just perfect given the assumption above.
  2. Learn basic Abstract Algebra. This book is great: A First Course in Abstract Algebra, by Fraleigh. No need to learn too advanced here, maybe only until the topic of Field and Module.
  3. In (1), we will learn basic topology (in metric space). Moreover (2) will give us solid understanding of things like morphisms. So, we can continue with general Topology. Lee's Introduction to Topological Manifolds is great. Just skim the first few chapters.
  4. Rigorous/Measure-Theoretic Probability Theory: it's very important in probabilistic models in general. Jacod and Protter's Probability Essentials is great.
  5. At this point, the path is very wide open. You can choose whatever you like next, as we should already have quite solid foundation. But to give some options:
    1. Functional Analysis: this is very useful. E.g., stochastic processes (including Gaussian processes) live in function spaces. Neural networks, too. I like this book: Kreyszig's Introductory Functional Analysis with Applications.
    2. Differential & Riemannian Geometry: this is a generalization of calculus in curved space/manifold. I would start with this book: Elementary Differential Geometry by O'Neill. Then, you can work on Lee's Introduction to Smooth Manifolds, and then, you can continue with Lee's Introduction to Riemannian Manifolds.
    3. Stochastic Processes: Gaussian processes are stochastic process. Diffusion models, too. Good book: Pavliotis' Stochastic Processes and Applications.
tip

Check the library (Taylor Library @ Science Centre) for physical books. If you use the university WiFi/account, you can download ebooks from Springer for free.

Interdisciplinary​

The goal is to be able to talk to interdisciplinary people (think chemistry, physics, geosciences, etc.).

  • Be able to listen to their jargons and connect them to our foundational, abstract research.
  • Be able to communicate our research in a way that interdisciplinary people understand.

Some interesting books​

Meta​