SmartSolveAI CSAIL · MIT
MIT · CSAIL

AI-Guided
Algorithm Discovery
for Fast
Linear Algebra.

Numerical linear algebra is the computational foundation of nearly all modern scientific computing and AI, shaping performance from large-scale multiphysics simulations to the training and inference of today's largest AI models. Yet its design space — including algorithm classes, mixed-precision strategies, sparsity structures, memory layouts, hardware backends, and task scheduling — is far too large for any human team to explore comprehensively. SmartSolveAI investigates how multi-agent AI can help navigate this high-dimensional design space to discover specialized, high-performance algorithms that outperform today's hand-tuned systems.

Coming soon — a new agentic version is on the way. Stay tuned.
01 Methodology

From benchmark to algorithms.

SmartSolveAI's methodology begins by benchmarking choices across the linear-algebra design space — including algorithm classes, mixed-precision strategies, sparsity structures, memory layouts, and hardware backends. It then uses the best-performing choices — evaluated by metrics such as accuracy, runtime, and memory usage — as input to an AI-assisted process that generates (1) selection heuristics to dispatch the optimal configuration for a given input, such as a specific preconditioner–solver pair, and (2) specialized variants of state-of-the-art solvers by leveraging information such as the input matrix's pattern and the target hardware.

SmartSolveAI methodology pipeline diagram INPUT Matrix A 01 · BENCHMARK Sampling the design space algorithms · precision · sparsity · HW 02 · DATABASE Best-performing choices accuracy · runtime · memory FORK 03A · AI-ASSISTED Selection heuristics e.g. optimal preconditioner–solver pair 03B · AI-ASSISTED Specialized solvers leveraging matrix pattern, hardware, etc. SMART DISCOVERY NEW LINEAR ALGEBRA
02 Publications

Selected research output.

Recent contributions advancing dynamic algorithmic dispatch, structure-aware linear algebra, and data-dependency-aware scheduling in Julia.

In Prep.

Parallel Agentic AI for Discovering Specialization and Task-Distribution Algorithms

SmartSolveAI Team
Manuscript in preparation
In Preparation
2025

Data-Driven Dynamic Algorithm Dispatch with Large Language Models ★ Outstanding Short Paper

Rushil Shah, Emmanuel Lujan, Rabab Alomairy, Alan Edelman
IEEE High Performance Extreme Computing Conference (HPEC), 2025
View →
2025

When Structure is Silent: Opportunities for Algorithmic Dispatch in Linear Algebra

Emmanuel Lujan, Alan Edelman
IEEE High Performance Extreme Computing Conference (HPEC), 2025
View →
2024

Dynamic Task Scheduling with Data Dependency Awareness Using Julia

Rabab Alomairy, Felipe Tome, Julian Samaroo, Alan Edelman
IEEE High Performance Extreme Computing Conference (HPEC), 2024
View →
03 Talks

Presentations & keynotes.

A selection of recent talks and presentations by members of the SmartSolveAI team.

JuliaCon · 2025 · Lightning Talk

Automated Algorithm Selection Discovery via LLMs

Rushil Shah, Emmanuel Lujan, Rabab Alomairy ▶ Watch
Householder Symposium · 2025 · Presentation

Julia, Portable Numerical Linear Algebra and Beyond

Alan Edelman et al. →
WAMTA · 2025 · Keynote

Improving the HPC Experience: Did Julia Get It Right, or Will AI Hide the Problem (or Both)?

Alan Edelman →
04 Software

Open, reproducible software.

Three open-source components developed under the SmartSolveAI project, with public source code, scripts, and reproducibility instructions.

01 SmartSolve.jl Multi-agent toolbox
The SmartSolve.jl agent loop: a Proposer agent generates a candidate implementation; a Reviewer agent evaluates it and either returns revisions to the Proposer or accepts the candidate as the final algorithm. AGENT Proposer generates a candidate AGENT Reviewer evaluates & decides candidate revisions accept OUT PROPOSE · REVIEW · ITERATE

A new multi-agent implementation of SmartSolve.jl. It exposes a minimal interface: install the package, describe the algorithm you want, and let the toolbox generate a high-performance implementation grounded in benchmarked dispatch strategies. The example shown produces an LU factorization with iterative refinement from a single natural-language prompt and a reference.

View on GitHub↗ Cite via Zenodo↗
Lujan, E., Shah, R., Alomairy, R., & Edelman, A. (2025).
SmartSolve.jl: AI for Algorithmic Discovery, v0.1.0-alpha. Zenodo.
doi:10.5281/zenodo.15784217
Install
repl · pkg mode 
]add https://github.com/Smart-Solve-AI/SmartSolve.jl
Run
smartsolve_demo.jl 
using SmartSolve

prompt = """
Generate a high performance implementation of LU
with iterative refinement using the following reference:
https://nhigham.com/2023/03/13/what-is-iterative-refinement
"""

secret_key = ENV["OPENAI_API_KEY"]

code, hist, conv = generate(prompt, secret_key)
02 SmartDiscovery PETSc benchmarking & selection
SmartDiscovery workflow: input matrix benchmarked across PETSc solver–preconditioner combinations, returning the optimal configuration. INPUT Matrix A BENCHMARK · PETSc solver × preconditioner GMRES BCGS CG MINRES ILU JACOBI GAMG SOR ASM HYPRE OPTIMAL CG + JACOBI EXHAUSTIVE SEARCH

A benchmarking and selection tool for sparse linear solves. For a given input matrix, SmartDiscovery exhaustively benchmarks combinations of solvers and preconditioners — measuring accuracy, runtime, and memory usage — and returns the optimal configuration for the system at hand.

A PETSc-tailored version is currently under development, using matrices from the MIT Center for the Exascale Simulation of Coupled High Enthalpy Fluid-Solid Interactions for development and testing. An early implementation of SmartDiscovery is already available in SmartSolve.jl.

View on GitHub↗
03 NeuralLinearSolve.jl Neural selection · SciML
NeuralLinearSolve.jl pipeline: sparse matrix rendered as a 64×64 spy plot, processed through three convolutional blocks and three dense layers, producing a probability distribution over candidate solvers. INPUT Matrix A SPY 64×64 3× CONV 1→16→32→64 3× DENSE 4096→256→64→3 PREDICTION KLU 0.94 UMFPACK 0.04 Pardiso 0.02 LEARNED MODEL

NeuralLinearSolve.jl is a direct implementation of learned selection heuristics for sparse linear systems. Given an input matrix, a convolutional neural network analyzes its spy plot and predicts a probability distribution over candidate solvers, enabling selection of the solver expected to be fastest.

The current version is trained on roughly 1,000 matrices from the SuiteSparse Matrix Collection and achieves around 94% top-1 accuracy on a held-out test set. For CHEFSI finite-element simulations, the benchmarking infrastructure behind SmartDiscovery for PETSc, and the database of optimal choices it generates, provide the training data needed to build a domain-specific learned heuristic.

Released as part of the SciML ecosystem.

View on GitHub↗
05 Team

Built at the Julia Lab, MIT.

SmartSolveAI is developed at the MIT Julia Lab↗ , a research group working at the intersection of numerical computing, programming languages, applied mathematics, and scientific machine learning.

06 Acknowledgements

With support from.

This work is made possible through the support of leading research institutions and programs in computational science.

Massachusetts Institute of Technology
Host institution
Defense Advanced Research Projects Agency
DIAL Program
National Nuclear Security Administration
U.S. Department of Energy