Project title: DTC CSM 1 - Improving Precision and Convergence of Machine Learning Algorithms with application to Lattice QCD and GPUs
1st supervisor: Dr Benjamin Mora
2nd supervisor: Prof Biagio Lucini
Project description: Lattice QCD researchers are used to improve numerical algorithms to understand particles like quarks or gluons better. The algorithms are at the crossroad between Monte Carlo techniques and inverse problem solvers, and usually improved variants of the conjugate gradient algorithm. Similarly, Machine Learning (ML) researchers try to solve (or at least minimise) inverse problems using randomised methods like Stochastic Gradient Descent (SGD). With the advent of better algorithms and concepts (e.g. Generative Adversarial Networks), high performance graphics cards (GPUs) and specialised accelerators (e.g. Google's TPUs), some reasonably good levels of AI can nowadays be obtained with current methods in specific applications. Hence, it is clear that both Physics and ML have common interchangeable areas of research and there has never been a more exciting time to combine Physics and ML knowledge. This project will therefore try to aim at the following problems:
- Can we ensure faster learning with Machine Learning? More precisely, are there efficient methods that can replace algorithms based on SGD? This is an extremely important problem due the huge quantity of calculations needed to train a neural network.
- What is the influence of arithmetic precision in ML applications? One aim is to provide new algorithms that are more robust at calculating dot products from standard types (e.g. float or doubles), especially on GPUs. The new techniques will possibly improve stability of CG methods and reduce the number of times the algorithm needs to be restarted.
- On the opposite direction, can (low complexity) approximation methods for linear algebra operators be useful to neural networks, and in particular in the context of GANs? Can approximation techniques also be useful to lattice QCD?
Project title: DTC CSM 2 - Computationally-bounded agents in game theory
1st supervisor: Dr Arno Pauly
2nd supervisor: Dr Jeffrey Giansiracusa
Project description: Game theory is concerned with the strategic interactions of agents, typically assumed to be rational. It underlies a significant part of economics, but is also central to AI in the context of heterogeneous or multi-agent systems. In its traditional incarnation, game theory puts no a priori limits on the information-processing abilities of the agents. This is problematic, because it leads to results predicting behaviour which is computationally intractible or even non-computable to determine -- which obviously limits any applicability to real-world agents. This project is about starting from a different position: If agents are computationally-bounded (but otherwise "as rational as possible"), what types of interaction would emerge? Potential setups for this could be rooted in theoretical computer science, limiting the agents to execute algorithms in certain complexity classes; in functional analysis, restricting agents to determine their actions by e.g. Lipschitz functions, and using higher-order fixed point theorem to obtain equilibria; or in machine learning, where an agent selects the parameters of the machine learning model in a rational fashion, but is subsequently bound to the chosen learning model. What approaches to focus on would primarily be determined by the interests and qualifications of the student. A key application area could be the attempt to better model stock market interactions than orthodox approaches accomplish.
Project title: DTC CSM 3 - Logical theory of monitoring and its applications in physical and social systems
Project description: Monitoring is now endemic in society, and many interventions or decisions are based on it. This applies to smart factories (Industry 4.0), control of physical systems (e.g. autonomous vehicles) and to monitoring people in organisations. Issues of security and privacy are both Important, and seem to pull in opposite directions, yet often real world systems are neither secure nor ensure privacy. Many fundamental ideas are involved, including the theories of belief and identity, and the current concern with explainable AI. Can people trust systems to be safe and keep information private? The project is about a formal theory of monitoring combined with a modular description of physical or social systems which communicate with the real world via oracles. These are sources of information which may be error prone or compromised (e.g. by cyberattack). Non-explainable methods in AI (e.g. neural nets) can also be considered as oracles. A problem is then to look at formal methods for decision making which integrate many sources of information to both compensate for errors or failed sensors, and deliberate sabotage (cyberattack). We shall look at simplical complexes, a higher dimensional generalisation of graphs, to describe the structure of systems and the possible decisions, and the theory of belief as a method of collating evidence for making decisions.