This book offers supporting material for the comprehensive textbook Mathematical Physics : A Modern Introduction to Its Foundations authored by Sadri Hassani. The book covers mathematical preliminaries and all of Part I in Hassanis textbook. The subjects covered here include the key topics necessary for physicists to form a solid mathematical foundation: vectors and linear maps, algebras, operators, matrices, and spectral decomposition. In particular, the vector space concept is a central unifying theme in later chapters of Hassanis textbook. Detailed solutions are provided to one third of the end-of-chapter exercises in the first six chapters of his text. The present volume helps upper-undergraduate and early postgraduate physics students deepen their understanding of the mathematics that they encounter in physics, learn physics more efficiently, and use mathematics with more confidence and creativity. The content is thus presented rigorously but remains accessible to physics students. New exercises are also proposed, some with solutions, some without, so that the total number of unsolved exercises remains unchanged. They are chosen to help explain difficult concepts, amplify key points in Hassani's textbook, or make further connections with applications in physics. Taken together with Hassani's work, the two form a self-contained set and the solutions make detailed reference to Hassani's text. The solutions also refer to other mathematics and physics textbooks, providing entry points to further literature that finds a useful place in the physicist's personal library.