Equivariance
Many problems have physical constraints, such as invariance to rotations or the choice of reference frame. Incorporating these constraints, or inductive biases to neural architectures is not trivial and requires careful development of the neural architectures, and the resulting networks have significantly reduced model sizes and increased performance.

Relevant papers:
1. Theodosis, Ba, Dehmamy. "Constructing gauge-invariant neural networks for scientific applications", in GRaM and AI4Science Workshops at ICML, 2024.
2. Theodosis, Helwani, Ba. "Learning group representations in neural networks", in arXiv, 2024.

Structured representations
Learning meaningful representations is instrumental for most learning problems. Representations with appropriate properties can reduce model sizes, improve performance. and be more interpretable by being better suited to the task at hand.

Relevant papers:
1. Tasissa, Theodosis, Tolooshams, Ba. "Discriminative reconstruction via simultaneous dense and sparse coding", in TMLR, 2024.
2. Theodosis, Tolooshams, Tankala, Tasissa, Ba. "On the convergence of group-sparse autoencoders", in arXiv, 2020.

Tropical geometry
Linear algebra powers most learning systems today; however, there are nonlinear algebras that are more expressive and better suited for certain studies, such as the expressivity of neural networks. These algebras, through polyhedral geometry, facilitate the concise description of many problems in science and engineering.

Relevant papers:
1. Maragos, Charisopoulos, Theodosis. "Tropical Geometry and Machine Learning", in Proceedings of the IEEE, 2021.
2. Maragos, Theodosis, "Multivariate tropical regression and piecewise-linear surface fitting", in International Conference on Acoustics, Speech, and Signal Processing, 2020.
4. Theodosis, Maragos. "Tropical modeling of weighted transducers algorithms on graphs", in International Conference on Acoustics, Speech, and Signal Processing, 2019.