### Isolated invariant sets and isolating blocks

C. Conley, R. Easton
1971 Transactions of the American Mathematical Society
Introduction. The restricted three body problem has motivated a considerable amount of research in ordinary differential equations; in this instance the motivation comes from the nonplanar problem. However, the dimension, or even the specific form of the equations, is not relevant to the things discussed and so an easier problem is used here as an example. Namely, consider given two bowls which are connected by a saddle-like trough and the problem of describing how a point mass slides around in
more » ... ss slides around in this double bowl under the influence of gravity. Assuming the energy given the point mass is enough that it can go from one bowl to the other, one easily guesses that there are solutions which remain in the trough : if it were too close to either bowl the point mass would fall into the nearer one; somewhere between extremes there should be a way to place it so that it falls into neither. In fact if the energy is such that there is no place the point mass can rest, one might expect the existence of an unstable periodic orbit in the trough. In the planar restricted three body problem this would correspond to one of the unstable periodic orbits near the collinear Lagrangian point between the masses. In the nonplanar problem it is not quite so easy to guess the analogue, but a closer look leads one to expect that the role of the periodic orbit is played by a three dimensional sphere of orbits in the five dimensional energy surface. R. Sacker [13] has shown there is a smooth such invariant (composed of orbits) three-sphere for each of an interval of values of the energy. Again one expects the invariant set to be unstable : its existence is surmized on the basis of the instability together with the fact that orbits can leave the vicinity of the invariant set in two essentially different directions-they can fall towards one or the other mass point. This work concerns a general situation in which such heuristic reasoning can be formalized, see also [6], [8] and [9]. In the intended applications the procedure is to first find a region-like that between the bowls above-wherein one can guess the existence of an invariant set. This region we have called an isolating block ; it takes the form of a compact submanifold which has the same dimension as that carrying the flow and which has the property that all orbits tangent to its boundary