The aim of this paper is to construct multi-symplectic structures starting with the geometry of an oriented Riemannian manifold, independent of a Lagrangian or a particular partial differential equation (PDE). The principal observation is that on an n-dimensional orientable manifold M there is a canonical quadratic form Θ associated with the total exterior algebra bundle on M. On the fibre, which has dimension 2n, the form Θ can be locally decomposed into n classical symplectic structures. When concatenated, these n-symplectic structures define a partial differential operator, J∂, which turns out to be a Dirac operator with multi-symplectic structure. The operator J∂ generalizes the product operator J(d/dt) in classical symplectic geometry, and M is a generalization of the base manifold (i.e. time) in classical Hamiltonian dynamics. The structure generated by Θ provides a natural setting for analysing a class of covariant nonlinear gradient elliptic operators. The operator J∂ is elliptic, and the generalization of Hamiltonian systems, J∂Z=∇S(Z), for a section Z of the total exterior algebra bundle, is also an elliptic PDE. The inverse problem—find S(Z) for a given elliptic PDE—is shown to be related to a variant of the Legendre transform on k-forms. The theory is developed for flat base manifolds, but the constructions are coordinate free and generalize to Riemannian manifolds with non-trivial curvature. Some applications and implications of the theory are also discussed.