Physics-based domain adaptation for dynamical systems forecasting; towards a generalizable and interpretable machine learning for applied engineering

Data-driven models, especially machine learning-based models such as recurrent neural networks, are a popular choice for time-series forecasting because they can capture spatiotemporal structures from timeseries data, without reference to the mechanics governing the underlying phenomenon. However, their ability to generalize robustly depends on how well-represented the governing dynamics are in the data. This is difficult to ensure, often resulting in dependency on large datasets.

In this work we present an approach that combines simplified Physics-based models in the form of linear time-invariant state space models (LTI SSM) with unsupervised subspace learning, in a subspace-based domain adaptation (SDA) framework. SDA is a transfer learning technique to transfer labelled data from one domain to a related but different target domain where labelled data is scarce, by geometrically aligning their subspaces. We introduce a novel SDA approach where instead of labelled data, we leverage the physics-derived structure of the LTI SSM to forecast for unobserved timesteps, by aligning the source and target subspaces derived from physics and data, respectively. In this initial exploration, we evaluate our approach on a demonstrative heat conduction scenario.

This work contributes to an initiative at The Alan Turing Institute, towards developing a machine-learning catered for applied engineering; a machine-learning that does not depend on large datasets to ensure model-generalization (in-time and cross-transfer); a machine-learning that is physically interpretable.

Bio:

Zack is a Turing Research Fellow at The Alan Turing Institute (ATI) and a Visiting Research Fellow within the Department of Civil Engineering at the University of Cambridge. Prior to this, Zack was a Postdoctoral Research Associate within the Data-Centric Engineering programme at the ATI. He completed his PhD within the Architecture and Sustainable Design (ASD) department at the Singapore University of Technology and Design (SUTD), during which period he also conducted research as a visiting scholar at the Harvard Graduate School of Design in Cambridge, USA. Zack's PhD specialised in the application of Bayesian inference at the intersection of architectural design and structural engineering. Zack also holds an MPhil degree in Digital Architectonics from the University of Bath and a Bachelor’s degree in Architecture and Civil Engineering from the University of Malta. He is also a registered architect and civil engineer and has practiced with several architectural design offices.

Zack’s current research interests focus around leveraging Physics in machine-learning for the modeling of dynamical systems across engineering applications. He is interested in developing cutting-edge methodologies to address better model generalization without depending on large datasets while promoting physical interpretability.