Neural ODEs training session (Day 3 11:00-12:30)
Facilitated by: Andrew Ooi
Introduction: Data driven techniques can be used to find models for engineering problems based on given data. However, it is very common that the derived models do not perform well especially when used outside the parameter range of the training data. In order to obtain better predictions, scientific knowledge of the problem needs to be introduced into the mathematical model. The techniques blending classical, well established scientific numerical methodologies with modern day data-driven approaches has been broadly referred to as scientific machine learning (SciML). In this workshop, we will discuss and present some SciML approaches and see how they can be used to understand and predict relatively complex physical scientific systems.
Methodology: In this workshop, a SciML technique, commonly known as the Neural Ordinary Differential Equations (NODEs) will be presented. Participants will be guided on how NODEs can be used to effectively model data from physical systems. Participants will be shown details of the discrete adjoint method. The workshop will be presented in a series Jupyter Notebooks written in the Julia programming language. Using these Notebooks, participants will be guided to test out predictive capability of NODEs on systems of increasing complexity.
Outcomes: At the conclusion of the workshop, participants will gain insight on how NODEs are trained, and when it could be useful to apply NODEs to their own dataset.
Background and preparation: It is anticipated that the audience is comfortable with at least one programming language and is familiar with the basic techniques of using Neural Network to model data. If participants would like to run the Jupyter Notebooks on their laptops, they should have Julia installed in VSCode with the Jupyter plug-in. In addition the following libraries/packages should also be installed
CairoMakie
ComponentArrays
DiffEqFlux
Lux
MLUtils
Optimization
OptimizationOptimisers
OrdinaryDiffEqTsit5
OrdinaryDiffEq
Random
SciMLSensitivity
Further resources about installing Julia, Jupyter notebook extension and packages can be found in the links below)
https://code.visualstudio.com/docs/languages/julia (installing Julia)
https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter (installing the Jupyter Notebook extension)
https://docs.julialang.org/en/v1/stdlib/Pkg/ (installing packages in Julia)