Eigenvalue initialisation and regularisation for koopman autoencoders and beyond

Abstract:

Recent efforts have been made to learn the Koopman operator with predictive autoencoders. However, little attention has been payed to the initialisation of these networks. Noting the importance of eigenvalues to the action of a linear operator, one may ask whether it would be useful to employ them in the initialisation and regularisation of these autoencoders? To answer this, we devise a spectral eigenvalue initialisation and eigenvalue penalty scheme. Having done so, we discover that eigenvalues do in fact have great utility for this purpose. We demonstrate that in learning a Koopman operator for a damped driven pendulum, appropriate initialisation and regularisation can improve initial performance by an order of magnitude. We also show with this system that as the dissipative element of a dynamical system decreases, the utility of unit circle initialisation schemes increase and the utility of different regularisation schemes change. Additionally, we show that the benefits of eigenvalue initialisation and regularisation generalise to real-world cyclone wind data, sea surface temperature prediction and flow over a cylinder.

Bios:

Charles O'Neill is a Tuckwell Scholar and undergrad at the ANU studying mathematics and economics. He has undertaken research in many areas of applied mathematical modelling, with a particular focus on deep learning models. Charles is also in the process of writing two papers, one which achieves state-of-the-art on long-term vision prediction for macular degeneration, and the other which is with Jack Miller on improving Koopman autoencoders using eigenvalue initialisation and regularisation. He is currently working for Macuject, a Melbourne-based start-up who use AI to optimally diagnose and treat eye disease.

Jack Miller is an undergraduate and Tuckwell Scholar pursuing the PhB science degree. His research interests include physics, applied mathematics and machine learning. Currently, Jack is working on two papers: the first is on the feasibility of active cyclone interventions and the second is on the initialisation of Koopman predictive autoencoders. He also works for engineroom.io where he builds ML algorithms for Australian speech recognition.

Zoom Details:

Topic: Mathematics and Computational Sciences Seminar Series 
Time: This is a recurring meeting Meet anytime

Join Zoom Meeting
https://anu.zoom.us/j/88084900678?pwd=Tmw5NG0xRk1hR2VGdDlvdnQ3ZmdUdz09

Meeting ID: 880 8490 0678
Password: 971872
One tap mobile
+61731853730,,88084900678#,,,,0#,,971872# Australia
+61861193900,,88084900678#,,,,0#,,971872# Australia

Dial by your location
        +61 7 3185 3730 Australia
        +61 8 6119 3900 Australia
        +61 8 7150 1149 Australia
        +61 2 8015 6011 Australia
        +61 3 7018 2005 Australia
Meeting ID: 880 8490 0678
Password: 971872
Find your local number: https://anu.zoom.us/u/ksbYVntAY

Or an H.323/SIP room system:
    Dial: +61262227588 (AUCX)
    or 88084900678@zoom.aarnet.edu.au
    Meeting ID: 88084900678
    H323/SIP Password: 971872

Join by Skype for Business
https://anu.zoom.us/skype/88084900678