Mingsong Ye, Ph.D. Candidate in Data Science

Bio

Mingsong Ye is a Ph.D. candidate in data science at Stevens Institute of Technology. He specializes in optimization (mixed integer programming), Bayesian inference, and uncertainty quantization.

Skillset

He is advance in programming in Python, utilizing PyTorch (deep learning library) and Gurobi (optimization software) for research and teaching database software in PostgreSQL.

Dissertation Summary

The use of Machine Learning and AI to Improve Computational Performance in Large-scale Optimization and Time Series Applications

The three essays in my dissertation proposal examine the use of machine learning and artificial intelligence (ML/AI) for performance improvement in large-scale combinatorial problems and time series forecasting.

The first essay, “Using ML/AI to Improve Computational Performance in Large-scale Optimization Problems” surveys recent research on the use of ML/AI techniques to improve computational performance in large-scale combinatorial optimization (CO) problems. These problems may be NP-hard or beyond. Exact and heuristic optimization algorithms have been designed to solve CO problems. However, many CO problems cannot be solved in a reasonable time by traditional approaches. I survey research on ML/AI approaches to this problem. I develop a framework for categorizing the various approaches based on whether they involve algorithm imitation or algorithm generation. The survey concludes that ML/AI approaches have promise but that a general approach to solving a broad class of CO problems has yet to be discovered.

The second essay is “Configuring Optimization Solvers using Large Language Models.” Modern optimization solvers combine multiple algorithms to process a single optimization problem instance and have hundreds of parameters to force/disallow certain techniques and control the computation process. The chosen configuration can heavily impact optimization performance. Parameter tuning has been the subject of much previous research however the problem is difficult because of the complexity of CO problems and the weak transferability among problem instances. I examine the potential of large language model (LLM) integration to automatically set the parameters and determine the best configuration. My experimental results focusing on cutting-plane selection demonstrate the feasibility of applying LLM to parameter tuning.

The third essay, “Uncertainty Quantification”, explores uncertainty quantification in Big Data. Uncertainty (variability) is composed of two sources: aleatoric uncertainty and epistemic uncertainty. The former is due to inherent randomness in the data, while the latter captures the uncertainty due to missing data and unmeasured confounders. We extend linear models to more complex systems to decompose uncertainty additively into epistemic and aleatoric components. A measure of aleatoric uncertainty is captured by an ensemble model fit over a series of bootstrapped samples. The bootstrapped samples are generated from a baseline dataset (original sample). Similarly, the epistemic uncertainty is measured by the residual error in the hypothesized model obtained by a neural network. Initial experiments show encouraging results.

Academic Advisor

Ted Stohr