Eigenvalue initialisation and regularisation for koopman autoencoders and beyond

The Computational Mathematics Seminar presents talks on the development of computational science & mathematics, including the mathematical / computational modelling of complex systems including their implementation issues and theoretical aspects.

schedule Date & time
Date/time
23 Aug 2022 | 4 - 5pm
person Speaker

Speakers

Charles O'Neill, ANU
Jack Miller, ANU
next_week Event series
contact_support Contact

Content navigation

Description

Abstract:

Recent efforts have been made to learn the Koopman operator with predictive autoencoders. However, little attention has been payed to the initialisation of these networks. Noting the importance of eigenvalues to the action of a linear operator, one may ask whether it would be useful to employ them in the initialisation and regularisation of these autoencoders? To answer this, we devise a spectral eigenvalue initialisation and eigenvalue penalty scheme. Having done so, we discover that eigenvalues do in fact have great utility for this purpose. We demonstrate that in learning a Koopman operator for a damped driven pendulum, appropriate initialisation and regularisation can improve initial performance by an order of magnitude. We also show with this system that as the dissipative element of a dynamical system decreases, the utility of unit circle initialisation schemes increase and the utility of different regularisation schemes change. Additionally, we show that the benefits of eigenvalue initialisation and regularisation generalise to real-world cyclone wind data, sea surface temperature prediction and flow over a cylinder.

Bios:

Charles O'Neill is a Tuckwell Scholar and undergrad at the ANU studying mathematics and economics. He has undertaken research in many areas of applied mathematical modelling, with a particular focus on deep learning models. Charles is also in the process of writing two papers, one which achieves state-of-the-art on long-term vision prediction for macular degeneration, and the other which is with Jack Miller on improving Koopman autoencoders using eigenvalue initialisation and regularisation. He is currently working for Macuject, a Melbourne-based start-up who use AI to optimally diagnose and treat eye disease.

Jack Miller is an undergraduate and Tuckwell Scholar pursuing the PhB science degree. His research interests include physics, applied mathematics and machine learning. Currently, Jack is working on two papers: the first is on the feasibility of active cyclone interventions and the second is on the initialisation of Koopman predictive autoencoders. He also works for engineroom.io where he builds ML algorithms for Australian speech recognition.

Recording Details:

https://cloudstor.aarnet.edu.au/plus/s/KZsZUBkcu7AEASZ

Password: f7lyh4GcG7@b

Location

Seminar Room 1.33

Hanna Neumann Building #145

Science Road

The Australian National University

Canberra ACT 2600 

-35.275387198178, 149.11925554276