They also have a book on field theory, which is just about E&M basically.

Are you sure? I don't see such a book at https://mitpress.mit.edu/author/gerald-jay-sussman-2078/. I'd be very interested to read it if there is one.

Functional Differential Geometry is about the maths required for field theories but focuses on relativity as the main example.

You're right: I'm thinking of the theoretical minimum books by Leonard Susskind and Art Friedman.