Thesis

The focus of my thesis is to develop decision-making algorithms to allocate computational resources to ML algorithms in a resource-constrained environment. When developing and deploying ML algorithms, many decisions are effectively trading-off computations with other quantities such as accuracy or speed. For example, this could be the floating point accuracy chosen, the step size (resolution) used in diffusions or the number of inducing points in sparse Gaussian Processes. Those trade-offs also happen in applications such as robotics where the latency of inferring the next action is benchmarked against faster reaction times [6].

Many of those decisions are manually performed by a researcher for each specific application. In my thesis, I study how one could automate those computational related decisions to create dynamic data-driven computational decisions. To do so, I leverage existing work in probabilistic decision-making (RL, experimental design, BO) and extend them to accommodate computational constraints. Practically my most recent projects apply this concept for a large ice-sheet simulator (PDE) [4]. As a long-term goal, one could automatically design compute-efficient ML algorithms by searching through the space of architectures that optimize a given cost-utility trade-off. For example, it is possible to view many of the recent improvements in deep learning as making deep learning “compute-optimal”.