The proliferation of data and the advances in statistical and machine learning methods create tremendous opportunities for optimization and optimal control under uncertainty to address fundamental problems involving stochastic systems at scale. We consider a general approach, building on data and on statistical/machine learning and moving from stochastic models of uncertainty to optimization and/or optimal control, that provides a mathematical foundation for the optimal design and control of complex stochastic systems. A few different examples will be presented to illustrate instances of our general approach, each involving general classes of resource allocation problems with broad applications. In one case, we devise a simulation-based framework that yields optimal resource allocations in an efficient and effective manner, whereas in another case we derive an optimal control policy that includes easily and efficiently implementable algorithms for governing dynamic resource allocations over time. Computational experiments demonstrate the significant benefits of our approach over previously published results. This research is part of a recently formed Center for Optimization under Uncertainty Research (COUR, pronounced "kor") across all of IBM Research.