You are viewing a single comment's thread from:
RE: How to learn neural network solving ODE system?
The machine learning in your post is applied to find numerical solutions of an ODE. So the chaos here is just something extra? How does the chaotic phenomena in the Lorenz system relate to the machine learning being applied?
Very good question. The chaotic system is an additional difficulty for the neural network. The chaotic system is characterized by a complete lack of any determinism. Having the solution of the system of equations for the time t=t1, we are not able to determine functions e.g. for time t=t1+1. In the case of Lorenz system, the particle will be on the first or second loop. I do not know if it is clearly visible on the plots (the lines are strongly compacted), but you can see that the trajectory of the neural network in some places makes a mistake regarding to the loop. I created Lorenz system in which we have not chaos. The 1-layer neural network with the logsig transfer function has dealt with it much better. MSE = 3.666672309560181e-07 with only 1-layer (!).
As you can see, it is highly doubtful to say that neural networks are suitable for non-deterministic processes.
Do you want to comment on this statement you made? In my understanding Chaos is completely deterministic.
Could you explain a little further? In my understanding - if chaos were deterministic then it wouldn't be chaos. If we talk about Lorenz attractor, we don't know where the function will go because we have 2 focal points.
Chaos is completely deterministic. Chaotic trajectories follow a proper set of rules. Lorenz attractor is also a deterministic system (unless you add noise to it). Chaos has only to do with initial conditions in my understanding. I am also tagging @mathowl here. Maybe he can simplify things.
I didn't read that comment thoroughly. Thanks for pointing that mistake out.
From a theoretical viewpoint this is incorrect. ODEs are deterministic this means that given an initial condition all futures are fixed. I think you are confusing determinism with sensitive dependence on initial conditions (which is a property of Chaos). This means that if you take two solutions with initial conditions close to each other then they can exhibit different behavior. Which is exactly what you see in the Lorentz system for chaotic parameters.
From a numerically point of view you could say that when chaos is present your algorithm does not give a good determination of the solution for a given initial condition. However, even in that setting we do not call the algorithm non-deterministic since this is all caused by sensitive dependence on initial conditions.
In addition, sincethe ODE exhibits sensitive dependence on intial conditions it comes as no surprise that the neural network does not give satisfying results.
So you might wonder why a deterministic system can be called chaotic. Well in the real world we cannot measure the initial conditions exactly because of the uncertainty principle. So sensitive dependence means that even a small change in intial conditions can lead to different behaviour and since we cannot measure the initial conditions exactly it makes sense that we can call a deterministic system chaotic.
Thank you very much @mathowl and @dexterdev! I didn't investigate the chaos closely. And you are right - chaos isn't nondeterministic. But let me ask one more question - for clarification. Only initial conditions determine chaos in Lorenz system? Because in the code in comment (https://steemit.com/steemstem/@romualdd/how-to-learn-neural-network-solving-ode-system#@romualdd/re-mathowl-re-romualdd-how-to-learn-neural-network-solving-ode-system-20180706t130006093z) I set fixed and not wide initial conditions - the result was a system with one focal point. But if I change "r" parameter in the same system - the second focal point was apperead. So the system has become chaotic again (because of parameter in ODE system, not initial conditions). Am I right?
In the Lorenz system there is a subset of R^3 for which the orbits/solutions exihibit chaos. Not all initial conditions correspond to Chaos, the easiest way to see this is to compute the fixed points of the Lorenz system. For a fixed point the vector field is zero so the solution is stationary so for these points the system is certainly not chaotic.
Thank you very much for clarification! :)