Abstract: Bilevel optimization is widely used in machine learning, where an outer objective depends on the minimization of an inner problem. Most studies assume strong convexity of the inner objective with respect to finite-dimensional parameters, a restrictive setting for modern ML. We introduce a functional perspective on bilevel optimization, enabling richer models like neural networks and kernel methods while ensuring theoretical rigor and practical efficiency. We propose scalable algorithms for functional bilevel problems and demonstrate their benefits in instrumental regression and reinforcement learning. Theoretically, we establish novel generalization error bounds when the functional space is a Reproducing Kernel Hilbert Spaces, using empirical process theory and maximal inequalities for U-process, providing insights into the statistical accuracy of gradient-based methods for bilevel optimization.