Guido Montúfar

Critical Points of Linear Self-Attention with Quadratic Loss on a Single Training Example. optimization Ava Berenji, Khang Nguyen, Guido Montufar. Workshop article at NeurIPS 2026 OPT Workshop on Optimization for Machine Learning. [BibTeX]. On Parameter Symmetries and Conservation Laws in Gradient Flow. optimizationimplicit bias Khang Nguyen and Guido Montufar. Preprint [arXiv:2609.34549]. Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse. implicit biasneural network geometry Shuang Liang, Tom Jacobs, Guido Montufar. Preprint [arXiv:2607.03613]. Repo [GitHub]. [BibTeX]. Polyhedral Geometry of Time-to-First-Spike Neural Networks. neural network geometry Manjot Singh, Guido Montufar, Gitta Kutyniok. Preprint [arXiv:2609.11227]. TriSearch: Learning to Optimize Triangulations via Bistellar Flips. AI & mathematics Yiran Wang, Guido Montufar. Preprint [arXiv:2605.30220]. [BibTeX]. Accepted at NeurIPS 2026. A Numerical Study of Robustness Verification for Lightning Self-Attention. attentionverification Yulia Alexandr, Hao Duan, Guido Montufar. Workshop article at ICML 2026 Workshop Combining Theory and Benchmarks. Repo [GitHub]. [BibTeX]. Stress-Testing Neural Network Verifiers with Provably Robust Instances. verification David Troxell, Yulia Alexandr, Sofia Hunt, Stephanie Lei, Guido Montufar. Preprint [arXiv:2605.17153]. Workshop version at ICML 2026 Workshop Combining Theory and Benchmarks. Repo [GitHub]. [BibTeX]. Accepted at NeurIPS 2026, Evaluations & Datasets Track. Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning. implicit bias Tom Jacobs, Guido Montufar. Preprint [arXiv:2605.19458]. [BibTeX]. Accepted at NeurIPS 2026. The Symmetries of Three-Layer ReLU Networks. neural network geometry Johanna Marie Gegenfurtner, Moritz Grillo, Guido Montufar. Preprint [arXiv:2605.18319]. [BibTeX]. Accepted at NeurIPS 2026. Most ReLU Networks Admit Identifiable Parameters. neural network geometry Moritz Grillo, Guido Montufar. Preprint [arXiv:2605.03601]. [BibTeX]. Accepted at NeurIPS 2026. The Value Function Semi-Algebraic Set in Partially Observable Markov Decision Processes. algebraic geometryreinforcement learning Ryan Anderson, Guido Montufar. In Forty-third International Conference on Machine Learning (ICML 2026). Preprint [arXiv:2606.03048]. Repo [GitHub]. [BibTeX]. Differentiable Optimization Layers for Guaranteed Fairness in Deep Learning. optimizationreliable AI David Troxell, Noah Roemer, Guido Montufar. In Forty-third International Conference on Machine Learning (ICML 2026). Preprint [arXiv:2605.17118]. Repo [GitHub]. [BibTeX]. Harmful Overfitting in Sobolev Spaces. Kedar Karhadkar, Alexander Sietsema, Deanna Needell, Guido Montufar. In Forty-third International Conference on Machine Learning (ICML 2026). Preprint [arXiv:2602.00825]. [BibTeX]. Algebraic Invariants of Lightning Self-Attention. attentionalgebraic geometry Yulia Alexandr, Hao Duan, Guido Montufar. Preprint [arXiv:2604.15632]. Repo [GitHub]. [BibTeX]. Algebraic Robustness Verification of Neural Networks. verificationalgebraic geometry Yulia Alexandr, Hao Duan, Guido Montufar. Preprint [arXiv:2602.06105]. Repo [GitHub]. Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Region. optimization Shuang Liang, Guido Montufar. In International Conference on Learning Representations (ICLR 2026). Preprint [arXiv:2509.25351]. Virtual Poster [Video], [Poster]. Repo [GitHub]. [BibTeX].

Supermodular Rank: Set Function Decomposition and Optimization. optimization Rishi Sonthalia, Anna Seigal, Guido Montufar. SIAM Journal on Mathematics of Data Science 7(4), 2025. Preprint [arXiv:2305.14632]. Repo [GitHub]. [BibTeX]. Hadamard ranks of algebraic varieties. algebraic geometry Dario Antolini, Guido Montufar, Alessandro Oneto. Preprint [arXiv:2510.05231]. Repo [zenodo]. [BibTeX]. Accepted at SIAM Journal on Applied Algebra and Geometry (SIAGA). Low Rank Gradients and Where to Find Them. Rishi Sonthalia, Michael Murray, Guido Montufar. Neural Information Processing Systems (NeurIPS 2025). Workshop version presented at ICML 2025 Workshop HiLD. Preprint [arXiv:2510.01303]. Repo [GitHub]. [BibTeX]. Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts. generalization James Chapman, Kedar Karhadkar, Guido Montufar. Neural Information Processing Systems (NeurIPS 2025). Preprint [arXiv:2507.07348]. Repo [GitHub]. [BibTeX]. Enumeration of max-pooling responses with generalized permutohedra. expressivity Laura Escobar, Patricio Gallardo, Javier Gonzalez-Anaya, Jose L Gonzalez, Guido Montufar, Alejandro H Morales. Annals of Combinatorics, 2025. Preprint [arXiv:2209.16978]. Repo [GitHub]. [BibTeX]. Constraining the outputs of ReLU neural networks. neural network geometry Yulia Alexandr, Guido Montufar. Preprint [arXiv:2508.03867]. [BibTeX]. Understanding learning invariance in deep linear networks. Hao Duan, Guido Montufar. Proceedings of the Geometry, Topology, and Machine Learning Workshop, PMLR 325:1–45, 2026. [arXiv:2506.13714]. [BibTeX]. Fisher-Rao Gradient Flows of Linear Programs and State-Action Natural Policy Gradients. optimizationreinforcement learning Johannes Müller, Semih Çaycı, Guido Montufar. SIAM Journal on Optimization 35(2), 2025. Preprint [arXiv:2403.19448]. Repo [GitHub]. [BibTeX]. On the local complexity of linear regions in deep ReLU networks. neural network geometry Niket Patel, Guido Montufar. International Conference on Machine Learning (ICML 2025). Preprint [arXiv:2412.18283]. [BibTeX]. Implicit Bias of Mirror Flow for Shallow Neural Networks in Univariate Regression. implicit bias Shuang Liang, Guido Montufar. International Conference on Learning Representations (ICLR 2025). Preprint [arXiv:2410.03988]. Repo [GitHub]. [BibTeX]. Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passing. Diaaeldin Taha, James Chapman, Marzieh Eidi, Karel Devriendt, Guido Montufar. International Conference on Learning Representations (ICLR 2025). Preprint [arXiv:2506.06582]. Repo [GitHub]. [BibTeX].

Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape. neural network geometry Kedar Karhadkar, Michael Murray, Hanna Tseran, Guido Montufar. Transactions on Machine Learning Research, 2024. Preprint [arXiv:2305.19510]. Repo [GitHub]. [BibTeX]. Function space and critical points of linear convolutional networks. architecture Kathlen Kohn, Guido Montufar, Vahid Shahverdi, Matthew Trager. SIAM Journal on Applied Algebra and Geometry 8(2), 2024. Preprint [arXiv:2304.05752]. [BibTeX]. Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension. NTK Kedar Karhadkar, Michael Murray, Guido Montufar. Neural Information Processing Systems (NeurIPS 2024). Preprint [arXiv:2405:14630]. [BibTeX]. Benign overfitting in leaky ReLU networks with moderate input dimension. neural network geometry Kedar Karhadkar, Erin George, Michael Murray, Guido Montufar, Deanna Needell. Neural Information Processing Systems (NeurIPS 2024). Preprint [arXiv:2403.06903]. [BibTeX]. The Real Tropical Geometry of Neural Networks. algebraic geometry Marie-Charlotte Brandenburg, Georg Loho, Guido Montufar. Transactions of Machine Learning Research, 2024. Preprint [arXiv:2403.11871]. [BibTeX]. Pull-back Geometry of Persistent Homology Encodings. Shuang Liang, Renata Turkes, Jiayi Li, Nina Otter, Guido Montufar. Transactions on Machine Learning Research, 2024. Preprint [arXiv:2310.07073]. Repo [GitHub]. Video [Video]. [BibTeX].

Continuity and additivity properties of information decompositions. Johannes Rauh, Pradeep Kr. Banerjee, Ekehard Olbrich, Guido Montufar, Jürgen Jost. International Journal of Approximate Reasoning vol 161, 2023. Workshop version in 12 Workshop on Uncertainty Processing (WUPES 2022). Preprint [arXiv:2204.10982]. [BibTeX]. Algebraic Optimization of Sequential Decision Problems. optimizationalgebraic geometry Mareike Dressler, Marina Garrote-Lopez, Guido Montufar, Johannes Müller, Kemal Rose. Journal of Symbolic Computation vol 121, 2023. Preprint [arXiv:2211.09439]. Repo [GitHub]. Geometry and convergence of natural policy gradients. reinforcement learning Johannes Müller, Guido Montufar. Information Geometry, 2023. Preprint [arXiv:2211.02105]. Repo [GitHub]. Implicit bias of gradient descent for mean squared error regression with two-layer wide neural networks. implicit biasoptimization Hui Jin and Guido Montufar. Journal of Machine Learning Research JMLR 24(137):1-97, 2023. Repo [GitHub]. Preprint [arXiv:2006.07356]. Critical points and convergence analysis of generative deep linear networks trained with Bures-Wasserstein loss. Pierre Brechet, Katerina Papagiannouli, Jing An, Guido Montufar. In International Conference on Machine Learning (ICML 2023). Preprint [arXiv:2303.03027]. [BibTeX]. Expected Gradients of Maxout Networks and Consequences to Parameter Initialization. expressivity Hanna Tseran, Guido Montufar. In International Conference on Machine Learning (ICML 2023). Preprint [arXiv:2301.06956]. Repo [GitHub]. [BibTeX]. Characterizing the Spectrum of the NTK via a Power Series Expansion. NTK Michael Murray, Hui Jin, Benjamin Bowman, Guido Montufar. In International Conference on Learning Representations (ICLR 2023). Repo [GitHub]. Virtual poster [ICLR]. Preprint [arXiv:2211.07844]. [BibTeX]. FoSR: First-order spectral rewiring for addressing oversquashing in GNNs. Kedar Karhadkar, Pradeep Kr Banerjee, Guido Montufar. In International Conference on Learning Representations (ICLR 2023). Repo [GitHub]. Virtual poster [ICLR]. Preprint [arXiv:2210.11790]. [BibTeX].

Stochastic Feedforward Neural Networks: Universal Approximation. T. Merkh and G. Montufar. In Mathematical Aspects of Deep Learning, Cambridge University Press, 2022. Preprint [arXiv:1910.09763], [RG]. Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums. expressivity Guido Montufar, Yue Ren, and Leon Zhang. SIAM Journal on Applied Algebra and Geometry 6(4), 2022. Preprint [arXiv:2104.08135]. [BibTeX]. Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime. Benjamin Bowman and Guido Montufar. In Advances in Neural Information Processing Systems (NeurIPS 2022). Virtual poster [SlidesLive]. Preprint [arXiv:2206.02927]. [BibTeX]. On the effectiveness of persistent homology. Renata Turkeš, Guido Montufar, and Nina Otter. In Advances in Neural Information Processing Systems (NeurIPS 2022). Repo [GitHub]. Virtual poster [SlidesLive]. Preprint [arXiv:2206.10551]. [BibTeX]. Oversquashing in GNNs through the lens of information contraction and graph expansion. Pradeep Kr. Banerjee, Kedar Karhadkar, YuGuang Wang, Uri Alon, and Guido Montufar. In 58th Annual Allerton Conference on Communication, Control, and Computing (Allerton 2022). Preprint [arXiv:2208.03471]. [BibTeX]. Cell graph neural networks enable the precise prediction of patient survival in gastric cancer. Yanan Wang, YuGuang Wang, Changyuan Hu, Ming Li, Yanan Fan, Nina Otter, Ikuan Sam, Hongquan Gou, Yiquun Hu, Terry Kwon, John Zalcberg, Alex Boussioutas, Roger Dali, Guido Montufar, Pietro Lio, Dakang Xu, Geoffrey I Webb, and Jiangning Song. npj Precision Oncology 6, Article number: 45 (2022). [BibTeX]. Geometry of Linear Convolutional Networks. architecture Kathlen Kohn, Thomas Merkh, Guido Montufar, Matthew Trager. SIAM Journal on Applied Algebra and Geometry 6(3), 2022. Preprint [arXiv:2108.01538]. [BibTeX]. Solving infinite-horizon POMDPs with memoryless stochastic policies in state-action space. Johannes Müller and Guido Montufar. In Reinforcement Learning and Decision Making (RLDM 2022). Repo [GitHub]. Preprint [arXiv:2205.14098]. [BibTeX]. Continuity and additivity properties of information decompositions. Johannes Rauh, Pradeep Kr. Banerjee, Ekehard Olbrich, Guido Montufar, Juergen Jost. In 12 Workshop on Uncertainty Processing (WUPES 2022). Preprint [arXiv:2204.10982]. [BibTeX]. Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks. implicit biasoptimization Benjamin Bowman and Guido Montufar. In The Tenth International Conference on Learning Representations (ICLR 2022). Virtual poster [ICLR]. Preprint [arXiv:2201.04738]. [BibTeX]. Learning curves for Gaussian process regression with power-law priors and targets. generalization Hui Jin, Pradeep Kr. Banerjee, and Guido Montufar. In The Tenth International Conference on Learning Representations (ICLR 2022). Preprint [arXiv:2110.12231]. Slides [GitHub]. Power-law asymptotics of the generalization error for GP regression under power-law priors and targets. Workshop version presented at Workshop on Bayesian Deep Learning NeurIPS 2021. [BibTeX]. The Geometry of Memoryless Stochastic Policy Optimization in Infinite-Horizon POMDPs. optimizationreinforcement learning Johannes Müller and Guido Montufar. In The Tenth International Conference on Learning Representations (ICLR 2022). Virtual poster [ICLR]. Repo [GitHub]. Preprint [arXiv:2110.07409]. [BibTeX].

Weisfeiler and Lehman go cellular: CW networks. Christian Bodnar, Fabrizio Frasca, Nina Otter, Yu Guang Wang, Pietro Lio, Guido Montufar, and Michael Bronstein. Advances in Neural Information Processing Systems 35 (NeurIPS 2021). Repo [GitHub]. Virtual poster [SlidesLive]. Preprint [arXiv:2106.12575]. [BibTeX]. On the expected complexity of maxout networks. expressivity Hanna Tseran and Guido Montufar. Advances in Neural Information Processing Systems 35 (NeurIPS 2021). Repo [GitHub]. Virtual poster [SlidesLive]. Preprint [arXiv:2107.00379]. [BibTeX]. A top-down approach to attain decentralized multi-agents. Alex Tong Lin, Guido Montufar, and Stanley Osher. Handbook of Reinforcement Learning and Control, pp 419-431, Springer, 2021. [BibTeX]. Distributed learning via filtered hyperinterpolation on manifolds. Guido Montufar and Yu Guang Wang. Foundations of computational mathematics 22, 1219-1271, 2021. Preprint [arXiv:2007.09392]. [BibTeX]. How framelets enhance graph neural networks. Xuebin Zheng, Bingxin Zhou, Junbin Gao, Yu Guang Wang, Pietro Lio, Ming Li, and Guido Montufar. In Proceedings of the 38th International Conference on Machine Learning (ICML 2021), PMLR 139:12761-12771, 2021. Repo [GitHub]. Preprint [arXiv:2102.06986]. [BibTeX]. Weisfeiler and Lehman go topological: message passing simplicial networks. Christian Bodnar, Fabrizio Frasca, Yu Guang Wang, Nina Otter, Guido Montufar, Pietro Lio, and Michael Bronstein. In Proceedings of the 38th International Conference on Machine Learning (ICML 2021), PMLR 139:1026-1037, 2021. Virtual poster [ICML]. Preprint [arXiv:2103.03212]. Workshop version presented at Workshop on Geometrical and Topological Representation Learning ICLR, 2021. Virtual poster [SlidesLive]. [BibTeX]. Tight bounds on the smallest eigenvalue of the neural tangent kernel for deep ReLU networks. neural network geometryNTK Quynh Nguyen, Marco Mondelli, and Guido Montufar. In Proceedings of the 38th International Conference on Machine Learning (ICML 2021), PMLR 139:8119-8129, 2021. Preprint [arXiv:2012.11654]. [BibTeX]. Information complexity and generalization bounds. generalization Pradeep Kumar Banerjee and Guido Montufar. In IEEE international symposium on information theory (ISIT 2021). Preprint [arXiv:2105.01747]. [BibTeX]. Wasserstein proximal of GANs. Alex Tong Lin, Wuchen Li, Stanley Osher, and Guido Montufar. In Proceedings of the 5th International Conference Geometric Science of Information (GSI 2021), LNCS, vol 12829, pp 524-533, Springer, 2021. Preprint [arXiv:2102.06862], [CAM report 18-53], [RG]. Poster [pdf]. [BibTeX]. Decentralized multi-agents by imitations of a centralized controller. Alex Tong Lin, Mark J. Debord, Katia Estabridis, Gary Hewer, Guido Montufar, and Stanley Osher. In 2nd Annual Conference on Mathematical and Scientific Machine Learning (MSML 2021). Preprint [arXiv:1902.02311]. Wasserstein distance to independence models. Tuerkue Ozluem Celik, Asgar Jamneshan, Guido Montufar, Bernd Sturmfels, and Lorenzo Venturello. Journal of Symbolic Computation, 104:855-873, 2021. Preprint [arXiv:2003.06725]. [BibTeX]. PAC-Bayes and information complexity. Pradeep Kr. Banerjee and Guido Montufar. Presented at Workshop on neural compression: from information theory to applications ICLR, 2021. Poster [GitHub]. [BibTeX].

Optimization theory for ReLU neural networks trained with normalization layers. optimizationneural network geometry Yonatan Dukler, Quanquan Gu, and Guido Montufar. In Proceedings of the 37th International Conference on Machine Learning (ICML 2020), PMLR 119:2751-2760, 2020. Virtual poster [ICML]. Preprint [arXiv:2006.06878]. [BibTeX]. Haar graph pooling. Yu Guang Wang, Ming Li, Zheng Ma, Guido Montufar, Xiaosheng Zhuang, Yanan Fan. In Proceedings of the 37th International Conference on Machine Learning (ICML 2020), PMLR 119:9952-9962, 2020. Repo [GitHub]. Virtual poster [ICML]. Preprint [arXiv:1909.11580]. [BibTeX]. The variational deficiency bottleneck. Pradeep Kumar Banerjee and Guido Montufar. In Proceedings of the international joint conference on neural networks (IJCNN 2020). Preprint [arXiv:1810.11677]. [BibTeX]. Kernelized Wasserstein Natural Gradient. Michael Arbel, Arthur Gretton, Wuchen Li, Guido Montufar. In International Conference on Learning Representations (ICLR 2020). Repo [GitHub]. Preprint [arXiv:1910.09652]. [BibTeX]. Factorized Mutual Information Maximization. Thomas Merkh and Guido Montufar. Kybernetika 56(5):948-978, 2020. Preprint [arxiv:1906.05460], [RG]. [BibTeX]. Ricci curvature for parametric statistics via optimal transport. W. Li and G. Montufar. Information Geometry 3(1):89-117, 2020. Preprint [arXiv 1807.07095]. [BibTeX]. Can neural networks learn persistent homology features?. Guido Montufar, Nina Otter, and Yu Guang Wang. Presented at Workshop on topological data analysis and beyond NeurIPS, 2020. Preprint [arXiv:2011.14688]. [BibTeX].

Optimal Transport to a Variety. T. O. Celik, A. Jamneshan, G. Montufar, B. Sturmfels, L. Venturello. In International Conference on Mathematical Aspects of Computer and Information Sciences (MACIS 2019), LNCS, vol 11989, pp 364-381, Springer, 2020. Preprint [arxiv:1909.11716], [MPI MIS 7/2021]. [BibTeX]. Affine Natural Proximal Learning. Wuchen Li, Alex Lin and Guido Montufar. In Proceedings of the 4th International Conference Geometric Science of Information (GSI 2019), LNCS, vol 11712, pp 705-714, Springer, 2019. Preprint [MPI MIS 6/2021], [RG]. [BibTeX]. Wasserstein of Wasserstein loss for learning generative models. Y. Dukler, W. Li, A. Lin, and G. Montufar. In Proceedings of the 36th International Conference on Machine Learning (ICML 2019), PMLR 97:1716-1725, 2019. Repo [GitHub]. Preprint [MPI MIS 13/2019]. [BibTeX]. Wasserstein Diffusion Tikhonov Regularization. Alex Tong Lin, Yonatan Dukler, Wuchen Li, and Guido Montufar. Presented at Optimal Transport and Machine Learning Workshop NeurIPS, 2019. Preprint [arxiv:1909.06860], [RG]. A continuity result for optimal memoryless planning in POMDPs. J. Rauh, N. Ay, G. Montufar. Presented at The 4th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM 2019). [pdf]. Preprint [MPI MIS 5/2021], [RG]. [BibTeX]. Task-Agnostic Constraining in Average Reward POMDPs. G. Montufar, J. Rauh, N. Ay. Presented at Task-Agnostic Reinforcement Learning Workshop ICLR, 2019. [pdf]. Preprint [MPI MIS 9/2021], [RG]. [BibTeX]. How Well Do WGANs Estimate the Wasserstein Metric?. Anton Mallasto, Guido Montufar, Augusto Gerolin. Preprint [arXiv:1910.03875]. [BibTeX].

Natural Gradient via Optimal Transport. W. Li and G. Montufar. Information Geometry 1, issue 2, pp 181-214, 2018. Preprint [arXiv:1803.07033]. [BibTeX]. Restricted Boltzmann Machines: Introduction and Review. expressivity G. Montufar. Information geometry and its applications (IGAIA IV), pp 75-115, Springer, 2018. Preprint [arXiv:1806.07066]. [BibTeX]. Computing the Unique Information. P. K. Banerjee, J. Rauh, and G. Montufar. IEEE International Symposium on Information Theory (ISIT), pages 141-145, 2018. Repo [GitHub]. Preprint [arXiv:1709.07487]. [BibTeX]. Mixtures and Products in two Graphical Models. algebraic geometry A. Seigal and G. Montufar. Journal of Algebraic Statistics vol 9 no 1, 2018. Preprint [arXiv:1709.05276]. [BibTeX]. Uncertainty and Stochasticity of Optimal Policies. G. Montufar, J. Rauh, N. Ay. In Proceedings of the 11th Workshop on Uncertainty Processing (WUPES 2018). Preprint [MPI MIS 8/2021], [RG]. [BibTeX].

Geometry of Policy Improvement. reinforcement learning G. Montufar and J. Rauh. In Geometric Science of Information (GSI 2017), LNCS, vol 10589, pp 282-290, Springer, 2017. Preprint [arXiv:1704.01785]. [BibTeX]. Morphological Computation: The good, the bad, and the ugly. K. Ghazi-Zahedi, R. Deimel, G. Montufar, V. Wall, and O. Brock. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017). [BibTex]. Preprint [RG]. [BibTeX]. Dimension of Marginals of Kronecker Product Models. G. Montufar and J. Morton. SIAM Journal on Applied Algebra and Geometry 1(1), 2017. Supplement [JacobianKronecker.m]. Preprint [arXiv:1511.03570], [MPI MIS 75/2015]. [BibTeX]. Hierarchical Models as Marginals of Hierarchical Models. G. Montufar and J. Rauh. International Journal of Approximate Reasoning (IJAR) 88:531-546, 2017. Workshop version in Proceedings of the 10th Workshop on Uncertainty Processing (WUPES 2015). Supplement [starcover.m]. Preprint [arXiv:1508.03606], [MPI MIS 27/2016]. [BibTeX]. Stochasticity of optimal policies for POMDPs. G. Montufar, K. Ghazi-Zahedi, N. Ay. Presented at Reinforcement Learning and Decision Making (RLDM 2017). [BibTeX]. Notes on the number of linear regions of deep neural networks. neural network geometry G. Montufar. Presented at Mathematics of Deep Learning, Sampling Theory and Applications (SampTA 2017). Preprint [RG]. [BibTeX].

Mode Poset Probability Polytopes. algebraic geometry G. Montufar and J. Rauh. Journal of Algebraic Statistics 7(1):1-13, 2016. Workshop version in Proceedings of the 10th Workshop on Uncertainty Processing (WUPES 2015). Preprint [arXiv:1503.00572], [MPI MIS 22/2015]. [BibTeX]. Evaluating Morphological Computation in Muscle and DC-motor Driven Models of Hopping Movements. K. Ghazi-Zahedi, D. Haeufle, G. Montufar, S. Schmitt, and N. Ay. Frontiers in Robotics and AI 3(42):frobt.2016.00042, 2016. Preprint [arXiv:1512.00250]. [BibTeX]. Information Theoretically Aided Reinforcement Learning for Embodied Agents. G. Montufar, K. Ghazi-Zahedi, and N. Ay. Preprint [arXiv:1605.09735], [RG]. Geometry and Determinism of Optimal Stationary Control in Partially Observable Markov Decision Processes. reinforcement learning G. Montufar, K. Ghazi-Zahedi, and N. Ay. Preprint [arXiv:1503.07206], [MPI MIS 22/2016]. [BibTeX].

A Theory of Cheap Control in Embodied Systems. G. Montufar, K. Zahedi, and N. Ay. PLoS Comput Biol 11(9):e1004427, 2015. Preprint [arXiv:1407.6836], [MPI MIS 70/2014]. [BibTeX]. Geometry and Expressive Power of Conditional Restricted Boltzmann Machines. expressivity G. Montufar, N. Ay, and K. Zahedi. Journal of Machine Learning Research JMLR 16(Dec):2405-2436, 2015. Preprint [arXiv:1402.3346], [MPI MIS 16/2014]. [BibTeX]. Discrete Restricted Boltzmann Machines. expressivity G. Montufar and J. Morton. Journal of Machine Learning Research JMLR 16(Apr):653-672, 2015. Conference version in First International Conference on Learning Representations (ICLR 2013). Preprint [arXiv:1301.3529], [MPI MIS 106/2014]. [BibTeX]. When Does a Mixture of Products Contain a Product of Mixtures? G. Montufar and J. Morton. SIAM Journal on Discrete Mathematics (SIDMA) 29(1):321-347, 2015. Preprint [arXiv:1206.0387], [MPI MIS 98/2014]. [BibTeX]. Deep Narrow Boltzmann Machines are Universal Approximators. expressivity G. Montufar. In Third International Conference on Learning Representations (ICLR 2015). Preprint [arXiv:1411.3784], [MPI MIS 113/2014]. [BibTeX]. A Comparison of Neural Network Architectures. G. Montufar. Presented at Deep Learning Workshop ICML, 2015. [pdf], [pdf]. [BibTeX]. Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks. G. Montufar. Preprint [arXiv:1503.07211], [MPI MIS 23/2015]. [BibTeX].

On the Number of Linear Regions of Deep Neural Networks. neural network geometry G. Montufar, R. Pascanu, K. Cho, and Y. Bengio. Neural Information Processing Systems 27 (NeurIPS 2014; formerly NIPS 2014). Preprint [MPI MIS 73/2014], [arXiv 1402.1869]. [BibTeX]. On the Number of Response Regions of Deep Feedforward Networks with Piecewise Linear Activations. R. Pascanu, G. Montufar, and Y. Bengio. In Second International Conference on Learning Representations (ICLR 2014). Preprint [MPI MIS 72/2014], [arXiv 1312.6098]. [BibTeX]. On the Fisher Information Metric of Conditional Probability Polytopes. G. Montufar, J. Rauh, and N. Ay. Entropy 16(6):3207-3233, 2014. Preprint [MPI MIS 87/2014], [arXiv 1404.0198]. [BibTeX]. Scaling of Model Approximation Errors and Expected Entropy Distances. G. Montufar and J. Rauh. Kybernetika 50(2):234-245, 2014. Workshop version WUPES 2012, pp. 137-148. Preprint [arXiv 1207.3399]. [BibTeX]. Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units. G. Montufar. Neural Computation 26(7):1386-1407, 2014. Preprint [MPI MIS 74/2014], [arXiv 1303.7461]. [BibTeX]. Sequential Recurrence-Based Multidimensional Universal Source Coding of Lempel-Ziv Type. T. Krueger, G. Montufar, R. Seiler, and R. Siegmund-Schultze. Preprint [MPI MIS 86/2014], [arXiv 1408.4433]. [BibTeX].

Universally Typical Sets for Ergodic Sources of Multidimensional Data. T. Krüger, G. Montufar, R. Seiler, and R. Siegmund-Schultze. Kybernetika 49(6):868-882, 2013. [BibTeX]. Preprint [MPI MIS 20/2011], [arXiv 1105.0393]. [BibTeX]. Mixture Decompositions of Exponential Families Using a Decomposition of their Sample Spaces. G. Montufar. Kybernetika 49(1):23-39, 2013. Preprint [MPI MIS 39/2010], [arXiv 1008.0204]. [BibTeX]. Maximal Information Divergence from Statistical Models defined by Neural Networks. G. Montufar, J. Rauh, and N. Ay. In Geometric Science of Information (GSI), LNCS, vol 8085, pp 759-766, Springer, 2013. Preprint [MPI MIS 31/2013], [arXiv 1303.0268]. [BibTeX]. Selection Criteria for Neuromanifolds of Stochastic Dynamics. N. Ay, G. Montufar, J. Rauh. In Advances in Cognitive Neurodynamics (III), pp 147-154, 2013. Preprint [MPI MIS 15/2011]. [BibTeX].

Kernels and Submodels of Deep Belief Networks. G. Montufar and J. Morton. Deep Learning and Unsupervised Feature Learning Workshop, NeurIPS 2012; formerly NIPS 2012. Preprint [arXiv 1211.0932]. [BibTeX].

Expressive Power and Approximation Errors of Restricted Boltzmann Machines. expressivity G. Montufar, J. Rauh, and N. Ay. Neural Information Processing Systems 24 (NeurIPS 2011; formerly NIPS 2011). [BibTeX]. Preprint [MPI MIS 27/2011], [arXiv 1406.3140]. [BibTeX]. Refinements of Universal Approximation Results for Restricted Boltzmann Machines and Deep Belief Networks. expressivity G. Montufar and N. Ay. Neural Computation 23(5):1306-1319, 2011. Preprint [MPI MIS 23/2010], [arXiv 1005.1593]. [BibTeX]. Mixture Models and Representational Power of RBMs, DBNs and DBMs. G. Montufar. Deep Learning and Unsupervised Feature Learning Workshop NeurIPS 2010; formerly NIPS 2010. [pdf], [pdf]. [BibTeX].

On the Expressive Power of Discrete Mixture Models, Restricted Boltzmann Machines, and Deep Belief Networks—A Unified Mathematical Treatment. expressivity PhD Thesis, Leipzig University, October 2012. Supervisor: N. Ay. [pdf] (14.4 MB, 155 pages, 30 figures) [BibTeX]. Theory of Transport and Photon-Statistics in a Biased Nanostructure. German Diplom in Physics, Institute for Theoretical Physics, TU-Berlin, December 2008. Supervisor: A. Knorr and T. Brandes. [BibTeX]. Q-Sanov Theorem for d ≥ 2. German Diplom in Mathematics, Institute for Mathematics, TU-Berlin, August 2007. Supervisor: R. Seiler and J.-D. Deuschel. [BibTeX].