At the Gurobi Decision Intelligence Summit in September in Las Vegas, Irv participated in a panel discussion on Generative AI (GenAI) for Optimization, moderated by Gurobi’s Dan Steffy. Following are lightly edited excerpts.
Irv Lustig: Regarding GenAI, I would say I’m an observer, using my experience to evaluate different things that are going on in a technology that is moving incredibly fast without appropriate guardrails, in my opinion. There are various activities in GenAI at my firm—we have some great people conducting interesting experiments.
Dan Steffy: How do we know when something is delivering real value? Which things have you seen that you think are delivering the most tangible value today?
Irv: GenAI is the latest flashy hammer, and people are looking for nails. When I look at problems, I want to find the nails and then determine whether I need a hammer, screwdriver, wrench or other tool. To deliver value, understand the problem you’re trying to solve. Then you figure out if there are tools that can help you to deliver that value.
There was an interesting talk by Microsoft that was sponsored by the INFORMS Manufacturing & Service Operations Management Society. Briefly, a Microsoft team built an optimization application to help determine quantities and types of servers at each of their data centers. This was replacing an existing spreadsheet-based approach. To encourage adoption, the team built the capability for users to type a question in English that would help explain why the optimization application generated a particular solution. Typing that question – evaluated by a Large Language Model (LLM) – generated a scenario that would get optimized and enable comparisons of relevant KPIs. In their initial implementation, they found they could handle about 70 percent of the questions. If the tool couldn’t answer the question and translate it into a constraint or appropriate optimization, it would send a message back to the team saying, “Here’s a question you couldn’t handle.” The developers would then do some software engineering to handle the next question like that, which increased their success rate over time and gained the confidence of the users in applying the optimization. I thought it was a great idea to create this interactivity related to explainability, to allow a user to interact with the results of an optimization to better understand the solutions.
This was a really different way of building trust and using LLMs and GenAI to help do that in a way that everybody was confident that it was doing the right thing. At Princeton, we are exploring this kind of application to increase user acceptance of an optimization solution.
I should add that I think another reason the Microsoft application is successful is that it’s in a limited domain. It is not trying to say you can add a constraint about every potential thing—it is saying, “We have our application for determining how many data servers we need in data centers. Let’s just worry about interpreting that language.”
Dan: Do you feel like the impact here you’re getting more from having an added value on top of what you might have delivered in the past before these capabilities? Do you see this more as opening up increased user acceptance to be able to deliver things that people wouldn’t have been able to do or go into areas where now you can build more trust than before and expand?
Irv: The latter. I think it’s about gaining trust so that it helps substantiate the value that the optimization is already delivering and create trust in that delivery. How do we measure trust? I’m not sure I can quantify that, but if as a consulting firm we deliver that trust in Application 1, our client will want to do Applications 2, 3, 4, and 5. It will be much easier to get on to the next application because we have created that area of trust in what we’ve delivered.
There was a panel discussion about AI a year ago at the INFORMS meeting in Seattle. One of the panelists was Charles Isbell, who was then the provost at Wisconsin and is now the Chancellor of the University of Illinois, and he was joined by Michael Littman, who is now the AI provost at Brown University. The topic was AI in education, and the concern arose that students are going to write term papers using GenAI tools. Charles said, as consumers of what these AI tools are producing, we must learn that we must edit the output created by a GenAI tool. The moderator, an AI expert, said, “I know not to trust it.” Well, 99 percent of the people using this stuff trust it. You must learn how to edit. Charles added, “In order to know how to be a good editor, you have to be a good writer.” He was saying this in the context of writing in the English language, but I would say the same thing is true when writing code. There are lots of coding agents. I would say the same thing is true with writing a model for optimization. There are AI bots meant to help people who don’t know much about modeling—but how will they know whether the output is right or wrong if they don’t know how to write a model in the first place? The consumers of the outputs of AI tools need to be trained to do the editing, and they can only learn how to be a good editor if they knew how to write the stuff.
Dan: I’ve heard from colleagues about questions like, “Can we replace Gurobi with an LLM and just put the model into ChatGPT and it will somehow use magic to solve it?”
Irv: I recently read that someone put in a very simple traveling salesman problem (TSP) into an LLM and it generated paths that were not connecting all the nodes and that the paths were crossing each other. It can’t solve a TSP.
Dan: Where do you think GenAI will have more impact: as a democratizing force helping non-experts build optimization models, or as a force empowering the experts to do things faster?
Irv: I think we’ll build trust. Earlier in my career, I was on teams trying to sell custom solutions and we would show examples of incredible return on investment and a low time to pay back. It was actually a barrier because if the executive we were selling to knew nothing about optimization, they became very skeptical about investing a little and making a lot. High returns on low investments caused them to assume there was a lot of risk. It’s human nature to think that way. Now with GenAI, we have the opportunity to effectively communicate about a proposed solution and reduce the perception of risk. This is where I see the impact: our ability to communicate and explain, and have people interact with our applications, and that is going to elevate the trust factor. The trust we build with that first application will yield two applications thereafter. It’s about getting more acceptance of the technology as a decision support tool.
Dan: What are you most excited about or maybe most afraid of?
Irv: I’m excited and afraid of the same word, which is trust. We already talked about that this technology is going to help us build trust in optimization. I’m afraid in a more general sense, not necessarily about our optimization community, but about the new people that get exposed to optimization and start using the tools. That they’re going to trust these tools without knowing that they should be thinking that the outputs might be wrong.
Irv welcomes conversations about the paradox of “trust”: there is the issue of whether you can trust GenAI output; there is also the opportunity to use GenAI to gain trust in optimization solutions. Email us to set up a conversation.
