AI-Powered Optimization Accelerates Engineering Design

Engineers often face the challenge of navigating numerous variables with limited opportunities for testing. A new approach leveraging Bayesian optimization and tabular foundation models is significantly accelerating the process of finding optimal solutions in complex engineering problems, from power grid design to vehicle safety.

The Challenge of High-Dimensional Optimization

Traditional optimization tools can struggle when dealing with problems involving hundreds of variables. Each evaluation, such as crash-testing a vehicle, can be costly and time-consuming. MIT researchers have developed a technique to address this challenge, improving the speed and efficiency of finding top solutions.

Bayesian Optimization and Foundation Models

The research rethinks the application of Bayesian optimization, an iterative method that builds a surrogate model to estimate the best configurations for a complex system. The key innovation lies in utilizing a tabular foundation model as the surrogate. These models, similar to large language models like ChatGPT but trained on tabular data (spreadsheets), can adapt to different applications without constant retraining.

How Tabular Foundation Models Enhance Optimization

Unlike traditional surrogate models that require retraining after each iteration, the tabular foundation model remains consistent throughout the optimization process. This significantly increases efficiency, particularly for large and complex problems. The model also identifies the most critical variables influencing the outcome, allowing the algorithm to focus its search and avoid wasting time on less impactful factors. For example, in car safety design, the algorithm can pinpoint features like the size of the front crumple zone as key determinants of safety performance.

Performance Gains and Applications

In tests conducted on 60 benchmark problems, including power grid design and car crash testing, the new method consistently found the best solutions 10 to 100 times faster than state-of-the-art optimization algorithms. While the method didn’t outperform baselines on all problems – such as robotic path planning, potentially due to limitations in the model’s training data – it demonstrated significant speedups for more complicated scenarios.

Future Directions

Researchers are exploring methods to further enhance the performance of tabular foundation models and apply the technique to problems with even higher dimensionality, such as the design of naval ships. This perform represents a broader shift towards using foundation models not just for perception or language, but as core algorithmic engines within scientific and engineering tools.

Expert Perspective

Wei Chen, the Wilson-Cook Professor in Engineering Design and chair of the Department of Mechanical Engineering at Northwestern University, notes that the approach “is a creative and promising way to reduce the heavy data requirements of simulation-based design.” He further states that the work is “a practical and powerful step toward making advanced design optimization more accessible and easier to apply in real-world settings.”¹

This research was presented at the International Conference on Learning Representations.

Source: MIT News